Adaptive Neural Compilation
Rudy Bunel, Alban Desmaison, Pushmeet Kohli et al. · arXiv (Cornell University) · 2016
This paper proposes an adaptive neural-compilation framework to address the problem of efficient program learning. Traditional code optimisation strategies used in compilers are based on applying pre-specified set of transformations that make the code faster to execute without changing its semantics. In contrast, our work involves adapting programs to make them more efficient while considering correctness only on a target input distribution. Our approach is inspired by the recent works on differentiable representations of programs. We show that it is possible to compile programs written in a low-level language to a differentiable representation. We also show how programs in this representation can be optimised to make them efficient on a target distribution of inputs. Experimental results demonstrate that our approach enables learning specifically-tuned algorithms for given data distributions with a high success rate.
Cited by 27 works. doi:10.48550/arxiv.1605.07969 Open-access PDF
Authors
References
- AutoResearch Lite: a reproducible paper-inspired experiment runner (Adam mechanism reproduction and convergence-speed comparison)
- Simple statistical gradient-following algorithms for connectionist reinforcement learning
- Neural GPUs Learn Algorithms
- Adding Gradient Noise Improves Learning for Very Deep Networks
- Superoptimizer: a look at the smallest program
- Stochastic superoptimization
- Reinforcement Learning Neural Turing Machines - Revised
- Learning Simple Algorithms from Examples
- A neural compiler
- Neural programming language