Stochastic variational inference
Matthew D. Hoffman, David M. Blei, Chong Wang et al. · Journal of Machine Learning Research · 2013
We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets.
Cited by 1,333 works. doi:10.5555/2567709.2502622
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Equation of State Calculations by Fast Computing Machines
- A New Approach to Linear Filtering and Prediction Problems
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
- Generalized Linear Models
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- A tutorial on hidden Markov models and selected applications in speech recognition
- On Information and Sufficiency
- Latent dirichlet allocation
- A Stochastic Approximation Method