Replicator Neural Networks for Universal Optimal Source Coding
Robert Hecht-Nielsen · Science · 1995
Replicator neural networks self-organize by using their inputs as desired outputs; they internally form a compressed representation for the input data. A theorem shows that a class of replicator networks can, through the minimization of mean squared reconstruction error (for instance, by training on raw data examples), carry out optimal data compression for arbitrary data vector sources. Data manifolds, a new general model of data sources, are then introduced and a second theorem shows that, in a practically important limiting case, optimal-compression replicator networks operate by creating an essentially unique natural coordinate system for the manifold.
Cited by 133 works. doi:10.1126/science.269.5232.1860
Authors
References
- Multilayer feedforward networks are universal approximators
- Self-Organizing Maps
- A Learning Algorithm for Boltzmann Machines*
- The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
- Spectra of Quantized Signals
- Nonlinear principal component analysis using autoassociative neural networks
- Principal Curves
- Parallel models of associative memory
- Universal approximation bounds for superpositions of a sigmoidal function
- Neural networks and principal component analysis: Learning from examples without local minima