A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
Mark Collier, Basil Mustafa, Efi Kokiopoulou et al. · arXiv (Cornell University) · 2020
Datasets with noisy labels are a common occurrence in practical applications of classification methods. We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label noise. We assume an underlying heteroscedastic generative process for noisy labels. To make gradient based training feasible we use a temperature parameterized softmax as a smooth approximation to the assumed generative process. We illustrate that the softmax temperature controls a bias-variance trade-off for the approximation. By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise. For image segmentation, our method increases the mean IoU on the PASCAL VOC and Cityscapes datasets by more than 1% over the state-of-the-art model.
Cited by 12 works. doi:10.48550/arxiv.2003.06778 Open-access PDF
Authors
References
- Deep Residual Learning for Image Recognition
- ImageNet: A large-scale hierarchical image database
- Learning Multiple Layers of Features from Tiny Images
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Statistical Analysis With Missing Data
- Conditional logit analysis of qualitative choice behavior
- The Pascal Visual Object Classes Challenge: A Retrospective
- Auto-Encoding Variational Bayes
- Xception: Deep Learning with Depthwise Separable Convolutions
- Gaussian Processes for Machine Learning