Variational Algorithms for Approximate Bayesian Inference
Matthew J. Beal · 2003
The Bayesian framework for machine learning allows for the incorporation of prior knowledge in a coherent way, avoids overfitting problems, and provides a principled basis for selecting between alternative models. Unfortunately the computations required are usually intractable. This thesis presents a unified variational Bayesian (VB) framework which approximates these computations in models with latent variables using a lower bound on the marginal likelihood. Chapter 1 presents background material on Bayesian inference, graphical models, and propaga-tion algorithms. Chapter 2 forms the theoretical core of the thesis, generalising the expectation-maximisation (EM) algorithm for learning maximum likelihood parameters to the VB EM al-gorithm which integrates over model parameters. The algorithm is then specialised to the large family of conjugate-exponential (CE) graphical models, and several theorems are presented to pave the road for automated VB derivation procedures in both directed and undirected graphs (Bayesian and Markov networks, respectively). Chapters 3-5 derive and apply the VB EM algorithm to three commonly-used and important models: mixtures of factor analysers, linear dynamical systems, and hidden Markov models.
Cited by 1,670 works.
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Optimization by Simulated Annealing
- Equation of State Calculations by Fast Computing Machines
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Estimating the Dimension of a Model
- Neural networks for pattern recognition
- Error bounds for convolutional codes and an asymptotically optimum decoding algorithm
- Bayesian Data Analysis
- An Information-Maximization Approach to Blind Separation and Blind Deconvolution
- Monte Carlo sampling methods using Markov chains and their applications