Sample-Based Planning for Continuous Action Markov Decision Processes
Christopher Ryan Mansley, Ari Weinstein, Michael L. Littman · Proceedings of the International Conference on Automated Planning and Scheduling · 2011
In this paper, we present a new algorithm that integrates recent advances in solving continuous bandit problems with sample-based rollout methods for planning in Markov Decision Processes (MDPs). Our algorithm, Hierarchical Optimistic Optimization applied to Trees (HOOT) addresses planning in continuous-action MDPs. Empirical results are given that show that the performance of our algorithm meets or exceeds that of a similar discrete action planner by eliminating the problem of manual discretization of the action space.
Cited by 87 works. doi:10.1609/icaps.v21i1.13484 Open-access PDF
Authors
References
- Finite-time Analysis of the Multiarmed Bandit Problem
- Bandit Based Monte-Carlo Planning
- Learning to Drive a Bicycle Using Reinforcement Learning and Shaping
- A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes
- Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces
- Associative Reinforcement Learning: Functions in k-DNF
- Online Optimization in X-Armed Bandits
- Receding Horizon Differential Dynamic Programming
- Online exploration in least-squares policy iteration
- Binary action search for learning continuous-action control policies