Scalable multi-label annotation
Jia Hao Deng, Olga Russakovsky, Jonathan Krause et al. · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · 2014
We study strategies for scalable multi-label annotation, or for efficiently acquiring multiple labels from humans for a collection of items. We propose an algorithm that exploits correlation, hierarchy, and sparsity of the label distribution. A case study of labeling 200 objects using 20,000 images demonstrates the effectiveness of our approach. The algorithm results in up to 6x reduction in human computation time compared to the naive method of querying a human annotator for the presence of every object in every image.
Cited by 152 works. doi:10.1145/2556288.2557011
Authors
References
- ImageNet: A large-scale hierarchical image database
- WordNet: An electronic lexical database . Ed. by Christiane Fellbaum. Cambridge, MA: MIT Press, 1998. Pp. xxii, 423.
- Learning realistic human actions from movies
- BoosTexter: A Boosting-based System for Text Categorization
- Speed of processing in the human visual system
- Fast, Accurate Detection of 100,000 Object Classes on a Single Machine
- Get another label? improving data quality and data mining using multiple, noisy labelers
- Learning multi-label scene classification
- The multidimensional wisdom of crowds
- A Kernel Method for Multi-Labelled Classification