FlowNet: Learning Optical Flow with Convolutional Networks
Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg et al. · IEEE International Conference on Computer Vision (ICCV) · 2015
Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the tasks CNNs succeeded at. In this paper we construct CNNs which are capable of solving the optical flow estimation problem as a supervised learning task. We propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations. Since existing ground truth data sets are not sufficiently large to train a CNN, we generate a large synthetic Flying Chairs dataset. We show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI, achieving competitive accuracy at frame rates of 5 to 10 fps.
Cited by 4,113 works. doi:10.1109/iccv.2015.316
Authors
- Alexey Dosovitskiy
- Eddy Ilg
- Philip Haeusser
- Caner Hazırbaş
- Vladimir Golkov
- Patrick van der Smagt
- Daniel Cremers
- Thomas Brox
References
- ImageNet classification with deep convolutional neural networks
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
- Adam: A Method for Stochastic Optimization
- Backpropagation Applied to Handwritten Zip Code Recognition
- Fully convolutional networks for semantic segmentation
- AutoResearch Lite: a reproducible paper-inspired experiment runner (Adam mechanism reproduction and convergence-speed comparison)
- Caffe
- Visualizing and Understanding Convolutional Networks
- GAN(Generative Adversarial Nets)
- Determining optical flow