EM Algorithms for PCA and SPCA
Sam T. Roweis · 1997
I present an expectation-maximization (EM) algorithm for principal component analysis (PCA). The algorithm allows a few eigenvectors and eigenvalues to be extracted from large collections of high dimensional data. It is computationally very efficient in space and time. It also naturally accommodates missing information. I also introduce a new variant of PCA called sensible principal component analysis (SPCA) which defines a proper density model in the data space. Learning for SPCA is also done with an EM algorithm. I report results on synthetic and real data showing that these EM algorithms correctly and efficiently find the leading eigenvectors of the covariance of datasets in a few iterations using up to hundreds of thousands of datapoints in thousands of dimensions. 1 Why EM for PCA? Principal component analysis (PCA) is a widely used dimensionality reduction technique in data analysis. Its popularity comes from three important properties. First, it is the optimal
Cited by 709 works.
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Matrix Computations
- The algebraic eigenvalue problem
- The Algebraic Eigenvalue Problem
- Probabilistic Principal Component Analysis
- Turbulence and the dynamics of coherent structures. I. Coherent structures
- ARPACK Users' Guide
- Supervised learning from incomplete data via an EM approach
- The EM algorithm for mixtures of factor analyzers