Large Margin Methods for Structured and Interdependent Output Variables
Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann et al. · MPG.PuRe (Max Planck Society) · 2005
Learning general functional dependencies between arbitrary input and output spaces is one of the key challenges in computational intelligence. While recent progress in machine learning has mainly focused on designing flexible and powerful input representations, this paper addresses the complementary issue of designing classification algorithms that can deal with more complex outputs, such as trees, sequences, or sets. More generally, we consider problems involving multiple dependent output variables, structured output spaces, and classification problems with class attributes. In order to accomplish this, we propose to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation. While this leads to a quadratic program with a potentially prohibitive, i.e. exponential, number of constraints, we present a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems. The proposed method has important applications in areas such as computational biology, natural language processing, information retrieval/extraction, and optical character recognition. Experiments from various domains involving different types of output spaces emphasize the breadth and generality of our approach.
Cited by 1,866 works. Open-access PDF
Authors
References
- Statistical Learning Theory
- The viterbi algorithm
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- A new polynomial-time algorithm for linear programming
- Foundations of statistical natural language processing
- The ellipsoid method and its consequences in combinatorial optimization
- Optimizing search engines using clickthrough data
- The Cutting-Plane Method for Solving Convex Programs
- Discriminative training methods for hidden Markov models
- BoosTexter: A Boosting-based System for Text Categorization