Efficient Multitemplate Learning for Structured Prediction
Qi Mao, Ivor Wai-Hung Tsang · IEEE Transactions on Neural Networks and Learning Systems · 2012
Conditional random fields (CRF) and structural support vector machines (structural SVM) are two state-of-the-art methods for structured prediction that captures the interdependencies among output variables. The success of these methods is attributed to the fact that their discriminative models are able to account for overlapping features on all input observations. These features are usually generated by applying a given set of templates on labeled data, but improper templates may lead to degraded performance. To alleviate this issue, in this paper we propose a novel multiple template learning paradigm to learn structured prediction and the importance of each template simultaneously, so that hundreds of arbitrary templates could be added into the learning model without caution. This paradigm can be formulated as a special multiple kernel learning problem with an exponential number of constraints. Then we introduce an efficient cutting-plane algorithm to solve this problem in the primal and present its convergence. We also evaluate the proposed learning paradigm on two widely studied structured prediction tasks, i.e., sequence labeling and dependency parsing. Extensive experimental results show that the proposed method outperforms CRFs and structural SVMs because of exploiting the importance of each template. Complexity analysis and empirical results also show that the proposed method is more efficient than Online multikernel learning on very sparse and high-dimensional data. We further extend this paradigm for structured prediction using generalized p-block norm regularization with p >; 1, and experiments show competitive performances when p ∈ [1,2).
Cited by 15 works. doi:10.1109/tnnls.2012.2228228 Open-access PDF
Authors
References
- Convex Optimization
- Error bounds for convolutional codes and an asymptotically optimum decoding algorithm
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Multitask Learning
- Multitask learning
- Max-Margin Markov Networks
- Large Margin Methods for Structured and Interdependent Output Variables
- Multiple kernel learning, conic duality, and the SMO algorithm
- Large Scale Multiple Kernel Learning
- Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons