iCaRL: Incremental Classifier and Representation Learning
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2017
A major open problem on the road to artificial intelligence is the development of incrementally learning systems that learn about more and more concepts over time from a stream of data. In this work, we introduce a new training strategy, iCaRL, that allows learning in such a class-incremental way: only the training data for a small number of classes has to be present at the same time and new classes can be added progressively. iCaRL learns strong classifiers and a data representation simultaneously. This distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures. We show by experiments on CIFAR-100 and ImageNet ILSVRC 2012 data that iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail.
Cited by 3,784 works. doi:10.1109/cvpr.2017.587 Open-access PDF
Authors
References
- Deep Residual Learning for Image Recognition
- Gradient-based learning applied to document recognition
- ImageNet Large Scale Visual Recognition Challenge
- AutoResearch Lite: a reproducible paper-inspired experiment runner (Adam mechanism reproduction and convergence-speed comparison)
- Learning Multiple Layers of Features from Tiny Images
- Dropout: a simple way to prevent neural networks from overfitting
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Representation Learning: A Review and New Perspectives
- Distilling the Knowledge in a Neural Network
- Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost