An optimization approach for fitting canonical tensor decompositions.
Daniel Dunlavy, Evrim Acar, Tamara G. Kolda · 2009
Tensor decompositions are higher-order analogues of matrix decompositions and have proven to be powerful tools for data analysis. In particular, we are interested in the canonical tensor decomposition, otherwise known as the CANDECOMP/PARAFAC decomposition (CPD), which expresses a tensor as the sum of component rank-one tensors and is used in a multitude of applications such as chemometrics, signal processing, neuroscience, and web analysis. The task of computing the CPD, however, can be difficult. The typical approach is based on alternating least squares (ALS) optimization, which can be remarkably fast but is not very accurate. Previously, nonlinear least squares (NLS) methods have also been recommended; existing NLS methods are accurate but slow. In this paper, we propose the use of gradient-based optimization methods. We discuss the mathematical calculation of the derivatives and further show that they can be computed efficiently, at the same cost as one iteration of ALS. Computational experiments demonstrate that the gradient-based optimization methods are much more accurate than ALS and orders of magnitude faster than NLS.
Cited by 22 works. doi:10.2172/978916 Open-access PDF
Authors
References
- Matrix Computations
- Numerical optimization
- Analysis of Individual Differences in Multidimensional Scaling Via an N-way Generalization of “Eckart-Young” Decomposition
- Tensor Decompositions and Applications
- Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis
- Least squares formulation of robust non-negative factor analysis
- Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics
- Multi‐Way Analysis with Applications in the Chemical Sciences
- Line search algorithms with guaranteed sufficient decrease
- Generalized rank annihilation factor analysis