Learning and Evaluating Sparse Interpretable Sentence Embeddings
Valentin Trifonov, Octavian-Eugen Ganea, Anna Potapenko et al. · arXiv (Cornell University) · 2018
Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have the benefit of increased interpretability properties: to some degree, each dimension can be understood by a human and associated with a recognizable feature in the data. In this paper, we transfer this idea to sentence embeddings and explore several approaches to obtain a sparse representation. We further introduce a novel, quantitative and automated evaluation metric for sentence embedding interpretability, based on topic coherence methods. We observe an increase in interpretability compared to dense models, on a dataset of movie dialogs and on the scene descriptions from the MS COCO dataset.
Cited by 0 works. doi:10.48550/arxiv.1809.08621 Open-access PDF
Authors
References
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- SentEval: An Evaluation Toolkit for Universal Sentence Representations
- Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
- Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
- Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding
- k-Sparse Autoencoders
- Linear Algebraic Structure of Word Senses, with Applications to Polysemy
- Sparse Overcomplete Word Vector Representations
- SPINE: SParse Interpretable Neural Embeddings