Deterministic Policy Gradient Algorithms
David Silver, Guy Lever, Nicolas Heess et al. · HAL (Le Centre pour la Communication Scientifique Directe) · 2014
2014 In this paper we consider deterministic policy gradient algorithms for reinforcement learning with continuous actions. The deterministic policy gradient has a particularly appealing form: it is the expected gradient of the action-value function. This simple form means that the deterministic policy gradient can be estimated much more efficiently than the usual stochastic policy gradient. To ensure adequate exploration, we introduce an off-policy actor-critic algorithm that learns a deterministic target policy from an exploratory behaviour policy. We demonstrate that deterministic policy gradient algorithms can significantly outperform their stochastic counterparts in high-dimensional action spaces.
Cited by 1,573 works.
Authors
References
- Reinforcement Learning: An Introduction
- Introduction to Reinforcement Learning
- Simple statistical gradient-following algorithms for connectionist reinforcement learning
- Policy Gradient Methods for Reinforcement Learning with Function Approximation
- A Natural Policy Gradient
- A Menu of Designs for Reinforcement Learning Over Time
- Reinforcement learning in feedback control
- Natural Actor-Critic
- 10.1162/1532443041827907
- Fast gradient-descent methods for temporal-difference learning with linear function approximation