Reducing the Dimensionality of Data with Neural Networks
Geoffrey E. Hinton, Ruslan Salakhutdinov · Science · 2006
High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.
Cited by 20,839 works. doi:10.1126/science.1127647
Authors
References
- Neural networks and physical systems with emergent collective computational abilities.
- A Fast Learning Algorithm for Deep Belief Nets
- Indexing by latent semantic analysis
- Nonlinear Dimensionality Reduction by Locally Linear Embedding
- A Global Geometric Framework for Nonlinear Dimensionality Reduction
- Dimension Reduction by Local Principal Component Analysis
- Replicator Neural Networks for Universal Optimal Source Coding
- Learning sets of filters using back-propagation