Deeplysupervised nets
Chen‐Yu Lee, Saining Xie, Patrick Gallagher et al. · 2015
Our proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent. We make an attempt to boost the classification performance by study-ing a new formulation in deep networks. Three aspects in convolutional neural networks (CNN) style architectures are being looked at: (1) transparency of the intermediate layers to the overall classification; (2) discriminativeness and robust-ness of learned features, especially in the early layers; (3) effectiveness in training due to the presence of the exploding and vanishing gradients. We introduce “com-panion objective ” to the individual hidden layers, in addition to the overall objec-tive at the output layer (a different strategy to layer-wise pre-training). We extend techniques from stochastic gradient methods to analyze our algorithm. The advan-tage of our method is evident and our experimental result on benchmark datasets shows significant performance gain over existing methods (e.g. all state-of-the-art results on MNIST, CIFAR-10, CIFAR-100, and SVHN). 1
Cited by 776 works.
Authors
References
- ImageNet classification with deep convolutional neural networks
- Going deeper with convolutions
- Backpropagation Applied to Handwritten Zip Code Recognition
- A Fast Learning Algorithm for Deep Belief Nets
- Improving neural networks by preventing co-adaptation of feature detectors
- Visualizing and Understanding Convolutional Networks
- Understanding the difficulty of training deep feedforward neural networks
- What is the best multi-stage architecture for object recognition?
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- Greedy Layer-Wise Training of Deep Networks