Maximum margin planning
Nathan Ratliff, James Andrew Bagnell, Martin Zinkevich · International Conference on Machine Learning · 2006
Imitation learning of sequential, goal-directed behavior by standard supervised techniques is often difficult. We frame learning such behaviors as a maximum margin structured prediction problem over a space of policies. In this approach, we learn mappings from features to cost so an optimal policy in an MDP with these cost mimics the expert's behavior. Further, we demonstrate a simple, provably efficient approach to structured maximum margin learning, based on the subgradient method, that leverages existing fast algorithms for inference. Although the technique is general, it is particularly relevant in problems where A* and dynamic programming approaches make learning policies tractable in problems beyond the limitations of a QP formulation. We demonstrate our approach applied to route planning for outdoor mobile robots, where the behavior a designer wishes a planner to execute is often clear, while specifying cost functions that engender this behavior is a much more difficult task.
Cited by 606 works. doi:10.1145/1143844.1143936
Authors
References
- The Nature of Statistical Learning Theory
- Gradient-based learning applied to document recognition
- Markov Decision Processes: Discrete Stochastic Dynamic Programming.
- Online convex programming and generalized infinitesimal gradient ascent
- Multiclass Support Vector Machines
- Max-Margin Markov Networks
- Minimization Methods for Non-Differentiable Functions
- Large Margin Methods for Structured and Interdependent Output Variables
- Apprenticeship learning via inverse reinforcement learning
- Solving large scale linear prediction problems using stochastic gradient descent algorithms