Instance-Aware Semantic Segmentation via Multi-task Network Cascades
Jifeng Dai, Kaiming He, Jian Sun · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2016
Semantic segmentation research has recently witnessed rapid progress, but many leading methods are unable to identify object instances. In this paper, we present Multitask Network Cascades for instance-aware semantic segmentation. Our model consists of three networks, respectively differentiating instances, estimating masks, and categorizing objects. These networks form a cascaded structure, and are designed to share their convolutional features. We develop an algorithm for the nontrivial end-to-end training of this causal, cascaded structure. Our solution is a clean, single-step training framework and can be generalized to cascades that have more stages. We demonstrate state-of-the-art instance-aware semantic segmentation accuracy on PASCAL VOC. Meanwhile, our method takes only 360ms testing an image using VGG-16, which is two orders of magnitude faster than previous systems for this challenging problem. As a by product, our method also achieves compelling object detection results which surpass the competitive Fast/Faster R-CNN systems. The method described in this paper is the foundation of our submissions to the MS COCO 2015 segmentation competition, where we won the 1st place.
Cited by 1,244 works. doi:10.1109/cvpr.2016.343 Open-access PDF
Authors
References
- Deep Residual Learning for Image Recognition
- ImageNet classification with deep convolutional neural networks
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
- Backpropagation Applied to Handwritten Zip Code Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Fully convolutional networks for semantic segmentation
- Microsoft COCO: Common Objects in Context
- Caffe
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification