Natural Language Processing (almost) from Scratch
Ronan Collobert, Jason Weston, Léon Bottou et al. · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 2011
Editor: We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling, achieving or exceeding state-of-theart performance in each on four benchmark tasks. Our goal was to design a flexible architecture that can learn representations useful for the tasks, thus avoiding excessive taskspecific feature engineering (and therefore disregarding a lot of prior knowledge). Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabelled training data. This work is then used as a basis for building a freely available tagging system with excellent performance while requiring minimal computational resources. Keywords:
Cited by 3,898 works. Open-access PDF
Authors
References
- Gradient-based learning applied to document recognition
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Greedy Layer-Wise Training of Deep Networks
- Phoneme recognition using time-delay neural networks
- Prediction and Entropy of Printed English
- Class-based n -gram models of natural language
- Three models for the description of language
- Continuous speech recognition by statistical methods
- Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm
- Word Representations: A Simple and General Method for Semi-Supervised Learning