Catch & Carry
Josh Merel, Saran Tunyasuvunakool, Arun Ahuja et al. · ACM Transactions on Graphics · 2020
We address the longstanding challenge of producing flexible, realistic humanoid character controllers that can perform diverse whole-body tasks involving object interactions. This challenge is central to a variety of fields, from graphics and animation to robotics and motor neuroscience. Our physics-based environment uses realistic actuation and first-person perception - including touch sensors and egocentric vision - with a view to producing active-sensing behaviors (e.g. gaze direction), transferability to real robots, and comparisons to the biology. We develop an integrated neural-network based approach consisting of a motor primitive module, human demonstrations, and an instructed reinforcement learning regime with curricula and task variations. We demonstrate the utility of our approach for several tasks, including goal-conditioned box carrying and ball catching, and we characterize its behavioral robustness. The resulting controllers can be deployed in real-time on a standard PC. 1
Cited by 101 works. doi:10.1145/3386569.3392474 Open-access PDF
Authors
- Josh Merel
- Saran Tunyasuvunakool
- Arun Ahuja
- Yuval Tassa
- Leonard Hasenclever
- Vu Pham
- Tom Erez
- Greg Wayne
- Nicolas Heess
References
- Deep Residual Learning for Image Recognition
- Long Short-Term Memory
- Human-level control through deep reinforcement learning
- MuJoCo: A physics engine for model-based control
- Behavior-based robotics
- Understanding intelligence
- A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
- Pharmacokinetics of a novel formulation of ivermectin after administration to goats
- Domain randomization for transferring deep neural networks from simulation to the real world
- Interactive control of avatars animated with human motion data