Target-driven visual navigation in indoor scenes using deep reinforcement learning
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve et al. · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · 2017
Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new goals, and (2) data inefficiency, i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows better generalization. To address the second issue, we propose the AI2-THOR framework, which provides an environment with high-quality 3D scenes and a physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently. We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment.
Cited by 1,498 works. doi:10.1109/icra.2017.7989381 Open-access PDF
Authors
References
- Deep Residual Learning for Image Recognition
- Human-level control through deep reinforcement learning
- Visualizing Data using t-SNE
- Mastering the game of Go with deep neural networks and tree search
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Learning a Similarity Metric Discriminatively, with Application to Face Verification
- The vector field histogram-fast obstacle avoidance for mobile robots
- Real-time simultaneous localisation and mapping with a single camera
- Real-time obstacle avoidance for fast mobile robots
- Reinforcement learning in robotics: A survey