Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
Armand Joulin, Tomáš Mikolov · arXiv (Cornell University) · 2015
Despite the recent achievements in machine learning, we are still very far from achieving real artificial intelligence. In this paper, we discuss the limitations of standard deep learning approaches and show that some of these limitations can be overcome by learning how to grow the complexity of a model in a structured way. Specifically, we study the simplest sequence prediction problems that are beyond the scope of what is learnable with standard recurrent networks, algorithmically generated sequences which can only be learned by models which have the capacity to count and to memorize sequences. We show that some basic algorithms can be learned from sequential data using a recurrent network associated with a trainable memory.
Cited by 165 works. doi:10.48550/arxiv.1503.01007 Open-access PDF
Authors
References
- ImageNet classification with deep convolutional neural networks
- Gradient-based learning applied to document recognition
- Long Short-Term Memory
- Random Forests
- Learning Internal Representations by Error Propagation
- Book Review: Pattern Recognition and Machine Learning
- Finding Structure in Time
- Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition
- Scaling learning algorithms towards AI
- Pattern Recognition and Machine Learning (Information Science and Statistics)