Massively Parallel Methods for Deep Reinforcement Learning
Nair, Arun, Srinivasan, Praveen, Sam Blackwell et al. · arXiv (Cornell University) · 2015
We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the Deep Q-Network algorithm (DQN). Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment, using identical hyperparameters. Our performance surpassed non-distributed DQN in 41 of the 49 games and also reduced the wall-time required to achieve these results by an order of magnitude on most games.
Cited by 379 works. doi:10.48550/arxiv.1507.04296 Open-access PDF
Authors
- Sam Blackwell
- Cagdas Alcicek
- Rory Fearon
- Vedavyas Panneershelvam
- Mustafa Suleyman
- Stig Petersen
- Shane Legg
- Volodymyr Mnih
- Koray Kavukcuoglu
- David Silver
References
- ImageNet classification with deep convolutional neural networks
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Reinforcement Learning: An Introduction
- Human-level control through deep reinforcement learning
- Large Scale Distributed Deep Networks
- Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
- Going Deeper with Convolutions
- Speech recognition with deep recurrent neural networks
- Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition
- Playing Atari with Deep Reinforcement Learning