Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2015
We adapt the ideas underlying the success of Deep Q-Learning to the continuous action domain. We present an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. Using the same learning algorithm, network architecture and hyper-parameters, our algorithm robustly solves more than 20 simulated physics tasks, including classic problems such as cartpole swing-up, dexterous manipulation, legged locomotion and car driving. Our algorithm is able to find policies whose performance is competitive with those found by a planning algorithm with full access to the dynamics of the domain and its derivatives. We further demonstrate that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs.
Cited by 9,554 works. doi:10.48550/arxiv.1509.02971 Open-access PDF
Authors
- Timothy Lillicrap
- Jonathan J. Hunt
- Alexander Pritzel
- Nicolas Heess
- Tom Erez
- Yuval Tassa
- David Silver
- Daan Wierstra
References
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Human-level control through deep reinforcement learning
- On the Theory of the Brownian Motion
- Playing Atari with Deep Reinforcement Learning
- MuJoCo: A physics engine for model-based control
- Deterministic Policy Gradient Algorithms
- Adaptive critic designs
- End-to-End Training of Deep Visuomotor Policies
- Synthesis and stabilization of complex behaviors through online trajectory optimization