Natural Language Processing (almost) from Scratch
Ronan Collobert, Jason Weston, Léon Bottou et al. · arXiv (Cornell University) · 2011
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.
Cited by 4,962 works. doi:10.48550/arxiv.1103.0398 Open-access PDF
Authors
References
- Gradient-based learning applied to document recognition
- A Fast Learning Algorithm for Deep Belief Nets
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- A tutorial on hidden Markov models and selected applications in speech recognition
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Greedy Layer-Wise Training of Deep Networks
- Phoneme recognition using time-delay neural networks
- Prediction and Entropy of Printed English
- An algorithm for suffix stripping
- Efficient BackProp