The Kernel Recursive Least-Squares Algorithm
Yaakov Engel, Shie Mannor, Ron Meir · IEEE Transactions on Signal Processing · 2004
We present a nonlinear version of the recursive least squares (RLS) algorithm. Our algorithm performs linear regression in a high-dimensional feature space induced by a Mercer kernel and can therefore be used to recursively construct minimum mean-squared-error solutions to nonlinear least-squares problems that are frequently encountered in signal processing applications. In order to regularize solutions and keep the complexity of the algorithm bounded, we use a sequential sparsification process that admits into the kernel representation a new input sample only if its feature space image cannot be sufficiently well approximated by combining the images of previously admitted samples. This sparsification procedure allows the algorithm to operate online, often in real time. We analyze the behavior of the algorithm, compare its scaling properties to those of support vector machines, and demonstrate its utility in solving two signal processing problems-time-series prediction and channel equalization.
Cited by 1,104 works. doi:10.1109/tsp.2004.830985
Authors
References
- The Nature of Statistical Learning Theory
- Statistical Learning Theory
- System Identification: Theory for the User
- A Wavelet Tour of Signal Processing
- Adaptive Filter Theory
- Learning with Kernels
- Multivariate Adaptive Regression Splines
- Adaptive filter theory
- Nonlinear Component Analysis as a Kernel Eigenvalue Problem
- Fast Training of Support Vector Machines Using Sequential Minimal Optimization