Language Models as Representations for Weakly-Supervised NLP Tasks
Fei Huang, Er Yates, Arun Ahuja et al. · 2012
Finding the right representation for words is critical for building accurate NLP systems when domain-specific labeled data for the task is scarce. This paper investigates language model representations, in which language models trained on unlabeled corpora are used to generate real-valued feature vectors for words. We investigate ngram models and probabilistic graphical models, including a novel lattice-structured Markov Random Field. Experiments indicate that language model representations outperform traditional representations, and that graphical model representations outperform ngram models, especially on sparse and polysemous words. 1
Cited by 16 works.
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- A tutorial on hidden Markov models and selected applications in speech recognition
- Indexing by latent semantic analysis
- Introduction to Modern Information Retrieval
- Latent dirichlet allocation
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Building a Large Annotated Corpus of English: The Penn Treebank
- A unified architecture for natural language processing
- Class-based n -gram models of natural language
- On the limited memory BFGS method for large scale optimization