Learning models for visual 3D localization with implicit mapping
Dan Rosenbaum, Frederic Besse, Fabio Viola et al. · arXiv (Cornell University) · 2018
We consider learning based methods for visual localization that do not require the construction of explicit maps in the form of point clouds or voxels. The goal is to learn an implicit representation of the environment at a higher, more abstract level. We propose to use a generative approach based on Generative Query Networks (GQNs, Eslami et al. 2018), asking the following questions: 1) Can GQN capture more complex scenes than those it was originally demonstrated on? 2) Can GQN be used for localization in those scenes? To study this approach we consider procedurally generated Minecraft worlds, for which we can generate images of complex 3D scenes along with camera pose coordinates. We first show that GQNs, enhanced with a novel attention mechanism can capture the structure of 3D scenes in Minecraft, as evidenced by their samples. We then apply the models to the localization problem, comparing the results to a discriminative baseline, and comparing the ways each approach captures the task uncertainty.
Cited by 16 works. doi:10.48550/arxiv.1807.03149 Open-access PDF
Authors
References
- Distinctive Image Features from Scale-Invariant Keypoints
- Adam: A Method for Stochastic Optimization
- Probabilistic robotics
- Bundle Adjustment — A Modern Synthesis
- ORB-SLAM: A Versatile and Accurate Monocular SLAM System
- Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
- Unsupervised Learning of Depth and Ego-Motion from Video
- PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
- 3D Semantic Parsing of Large-Scale Indoor Spaces
- Matching Networks for One Shot Learning