SPINE: SParse Interpretable Neural Embeddings
Anant Subramanian, Danish Pruthi, Harsh Jhamtani et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2018
Prediction without justification has limited utility. Much of the success of neural models can be attributed to their ability to learn rich, dense and expressive representations. While these representations capture the underlying complexity and latent trends in the data, they are far from being interpretable. We propose a novel variant of denoising k-sparse autoencoders that generates highly efficient and interpretable distributed word representations (word embeddings), beginning with existing word representations from state-of-the-art methods like GloVe and word2vec. Through large scale human evaluation, we report that our resulting word embedddings are much more interpretable than the original GloVe and word2vec embeddings. Moreover, our embeddings outperform existing popular word embeddings on a diverse suite of benchmark downstream tasks.
Cited by 24 works. doi:10.1609/aaai.v32i1.11935
Authors
References
- Distributed Representations of Words and Phrases and their Compositionality
- Learning the parts of objects by non-negative matrix factorization
- Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
- Reading Tea Leaves: How Humans Interpret Topic Models
- Semantic feature production norms for a large set of living and nonliving things
- Tailoring Continuous Word Representations for Dependency Parsing
- Machine Reading Tea Leaves: Automatically Evaluating Topic Coherence and Topic Model Quality
- Learning question classifiers: the role of semantic information
- Building Large Monolingual Dictionaries at the Leipzig Corpora Collection: From 100 to 200 Languages
- Prototypicality, distinctiveness, and intercorrelation: Analyses of the semantic attributes of living and nonliving concepts