Statistics-Based Summarization - Step One: Sentence Compression
Kevin K. Knight, Daniel Marcu · 2000
When humans produce summaries of documents, they do not simply extract sentences and concatenate them. Rather, they create new sentences that are grammatical, that cohere with one another, and that capture the most salient pieces of information in the original document. Given that large collections of text/abstract pairs are available online, it is now possible to envision algorithms that are trained to mimic this process. In this paper, we focus on sentence compression, a simpler version of this larger challenge. We aim to achieve two goals simultaneously: our compressions should be grammatical, and they should retain the most important pieces of information. These two goals can conflict. We devise both noisy-channel and decision-tree approaches to the problem, and we evaluate results against manual compressions and a simple baseline. Introduction Most of the research in automatic summarization has focused on extraction, i.e., on identifying the most important claus...
Cited by 392 works.
Authors
References
- C4.5: Programs for Machine Learning
- The mathematics of statistical machine translation: parameter estimation
- A stochastic parts program and noun phrase parser for unrestricted text
- Statistical Methods for Speech Recognition
- Statistical decision-tree models for parsing
- Machine transliteration
- Information Retrieval as Statistical Translation
- Advances in Automatic Text Summarization
- Information fusion in the context of multi-document summarization
- Towards multidocument summarization by reformulation: progress and prospects