Detecting visual text
Jesse Dodge, Amit Goyal, Xufeng Han et al. · 2013
When people describe a scene, they often include information that is not visually apparent; sometimes based on background knowledge, sometimes to tell a story. We aim to separate visual text—descriptions of what is being seen—from non-visual text in natural images and their descriptions. To do so, we first concretely define what it means to be visual, annotate visual text and then develop algorithms to automatically classify noun phrases as visual or non-visual. We find that using text alone, we are able to achieve high accuracies at this task, and that incorporating features derived from computer vision algorithms improves performance. Finally, we show that we can reliably mine visual nouns and adjectives from large corpora and that we can use these effectively in the classification task. 1
Cited by 42 works.
Authors
- Jesse Dodge
- Xufeng Han
- Alyssa Mensch
- Margaret Mitchell
- Karl Stratos
- Kota Yamaguchi
- Yejin Choi
- Hal Daumé
- Alex Berg
- Tamara L. Berg
References
- Histograms of Oriented Gradients for Human Detection
- The Pascal Visual Object Classes (VOC) Challenge
- Object Detection with Discriminatively Trained Part-Based Models
- Word association norms, mutual information, and lexicography
- SUN database: Large-scale scene recognition from abbey to zoo
- Describing objects by their attributes
- The 2005 PASCAL Visual Object Classes Challenge
- Attribute and simile classifiers for face verification
- What is an object?
- Recognition using regions