Bayesian Interpolation
David Mackay · Neural Computation · 1992
Although Bayesian analysis has been in use since Laplace, the Bayesian method of model-comparison has only recently been developed in depth. In this paper, the Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data. The concepts and methods described are quite general and can be applied to many other data modeling problems. Regularizing constants are set by examining their posterior probability distribution. Alternative regularizers (priors) and alternative basis sets are objectively compared by evaluating the evidence for them. “Occam's razor” is automatically embodied by this process. The way in which Bayes infers the values of regularizing constants and noise levels has an elegant interpretation in terms of the effective number of parameters determined by the data set. This framework is due to Gull and Skilling.
Cited by 4,388 works. doi:10.1162/neco.1992.4.3.415
Authors
References
- Estimating the Dimension of a Model
- Modeling by shortest data description
- A Practical Bayesian Framework for Backpropagation Networks
- Statistical predictor identification
- An Information Measure for Classification
- Probability, Frequency and Reasonable Expectation
- On the Asymptotic Behaviour of Posterior Distributions
- Estimation and Inference by Compact Coding
- Common Structure of Smoothing Techniques in Statistics
- Laplace's 1774 Memoir on Inverse Probability