Learning Multilingual Word Embeddings Using Image-Text Data
Karan Singhal, Karthik Raman, Balder ten Cate · 2019
There has been significant interest recently in learning multilingual word embeddings -- in which semantically similar words across languages have similar embeddings. State-of-the-art approaches have relied on expensive labeled data, which is unavailable for low-resource languages, or have involved post-hoc unification of monolingual embeddings. In the present paper, we investigate the efficacy of multilingual embeddings learned from weakly-supervised image-text data. In particular, we propose methods for learning multilingual embeddings using image-text data, by enforcing similarity between the representations of the image and that of the text. Our experiments reveal that even without using any expensive labeled data, a bag-of-words-based embedding model trained on image-text data achieves performance comparable to the state-of-the-art on crosslingual semantic similarity tasks.
Cited by 7 works. doi:10.18653/v1/w19-1807
Authors
References
- Deep Residual Learning for Image Recognition
- Going deeper with convolutions
- ImageNet Large Scale Visual Recognition Challenge
- Efficient Estimation of Word Representations in Vector Space
- Glove: Global Vectors for Word Representation
- Distributed Representations of Words and Phrases and their Compositionality
- CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
- DeViSE: A Deep Visual-Semantic Embedding Model
- Contextual correlates of synonymy
- Exploiting Similarities among Languages for Machine Translation