On overfitting, generalization, and randomly expanded training sets
George N. Karystinos, Dimitris A. Pados · IEEE Transactions on Neural Networks · 2000
An algorithmic procedure is developed for the random expansion of a given training set to combat overfitting and improve the generalization ability of backpropagation trained multilayer perceptrons (MLPs). The training set is K-means clustered and locally most entropic colored Gaussian joint input-output probability density function (pdf) estimates are formed per cluster. The number of clusters is chosen such that the resulting overall colored Gaussian mixture exhibits minimum differential entropy upon global cross-validated shaping. Numerical studies on real data and synthetic data examples drawn from the literature illustrate and support these theoretical developments.
Cited by 146 works. doi:10.1109/72.870038
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Pattern classification and scene analysis
- Elements of Information Theory
- Learning Internal Representations by Error Propagation
- Backpropagation Applied to Handwritten Zip Code Recognition
- Probability, Random Variables, and Stochastic Processes.
- Estimating the Dimension of a Model
- On Estimation of a Probability Density Function and Mode
- A Stochastic Approximation Method
- Pattern Classification and Scene Analysis