Learning with idealized kernels
James Tin-Yau Kwok, Ivor Wai-Hung Tsang · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2003
The kernel function plays a central role in kernel methods. Existing methods typically fix the functional form of the kernel in advance and then only adapt the associated kernel parameters based on empirical data. In this paper, we consider the problem of adapting the kernel so that it becomes more similar to the so-called ideal kernel. We formulate this as a distance metric learning problem that searches for a suitable linear transform (feature weighting) in the kernel-induced feature space. This formulation is applicable even when the training set can only provide examples of similar and dissimilar pairs, but not explicit class label information. Computationally, this leads to a local-optima-free quadratic programming problem, with the number of variables independent of the number of features. Performance of this method is evaluated on classification and clustering tasks on both toy and real-world data sets.
Cited by 143 works.
Authors
References
- Statistical Learning Theory
- Nonlinear Dimensionality Reduction by Locally Linear Embedding
- An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
- An Introduction to Support Vector Machines
- New Support Vector Algorithms
- Distance Metric Learning with Application to Clustering with Side-Information
- Choosing Multiple Parameters for Support Vector Machines
- On Kernel Target Alignment
- A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms
- Improving support vector machine classifiers by modifying kernel functions