Conditional Random Fields as Recurrent Neural Networks
Shuai Zheng, Sadeep Jayasumana, Bernardino Romera‐Paredes et al. · IEEE International Conference on Computer Vision (ICCV) · 2015
Pixel-level labelling tasks, such as semantic segmentation, play a central role in image understanding. Recent approaches have attempted to harness the capabilities of deep learning techniques for image recognition to tackle pixel-level labelling tasks. One central issue in this methodology is the limited capacity of deep learning techniques to delineate visual objects. To solve this problem, we introduce a new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To this end, we formulate Conditional Random Fields with Gaussian pairwise potentials and mean-field approximate inference as Recurrent Neural Networks. This network, called CRF-RNN, is then plugged in as a part of a CNN to obtain a deep network that has desirable properties of both CNNs and CRFs. Importantly, our system fully integrates CRF modelling with CNNs, making it possible to train the whole deep network end-to-end with the usual back-propagation algorithm, avoiding offline post-processing methods for object delineation. We apply the proposed method to the problem of semantic image segmentation, obtaining top results on the challenging Pascal VOC 2012 segmentation benchmark.
Cited by 2,330 works. doi:10.1109/iccv.2015.179 Open-access PDF
Authors
- Shuai Zheng
- Sadeep Jayasumana
- Bernardino Romera‐Paredes
- Vibhav Vineet
- Zhizhong Su
- Dalong Du
- Chang Huang
- Philip H. S. Torr
References
- ImageNet classification with deep convolutional neural networks
- Gradient-based learning applied to document recognition
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
- Fully convolutional networks for semantic segmentation
- Microsoft COCO: Common Objects in Context
- Caffe
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Learning long-term dependencies with gradient descent is difficult
- The Pascal Visual Object Classes Challenge: A Retrospective