Inverting Visual Representations with Convolutional Networks
Alexey Dosovitskiy, Thomas Brox · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2016
Feature representations, both hand-designed and learned ones, are often hard to analyze and interpret, even when they are extracted from visual data. We propose a new approach to study image representations by inverting them with an up-convolutional neural network. We apply the method to shallow representations (HOG, SIFT, LBP), as well as to deep networks. For shallow representations our approach provides significantly better reconstructions than existing methods, revealing that there is surprisingly rich information contained in these features. Inverting a deep network trained on ImageNet provides several insights into the properties of the feature representation learned by the network. Most strikingly, the colors and the rough contours of an image can be reconstructed from activations in higher network layers and even from the predicted class probabilities.
Cited by 603 works. doi:10.1109/cvpr.2016.522 Open-access PDF
Authors
References
- ImageNet: A large-scale hierarchical image database
- Distinctive Image Features from Scale-Invariant Keypoints
- Backpropagation Applied to Handwritten Zip Code Recognition
- Histograms of Oriented Gradients for Human Detection
- Fully convolutional networks for semantic segmentation
- AutoResearch Lite: a reproducible paper-inspired experiment runner (Adam mechanism reproduction and convergence-speed comparison)
- Neural networks for pattern recognition
- Caffe
- Object Detection with Discriminatively Trained Part-Based Models
- Visualizing and Understanding Convolutional Networks