Deep Probabilistic Programming
Dustin Tran, Matthew D. Hoffman, Rif A. Saurous et al. · arXiv (Cornell University) · 2017
We propose Edward, a Turing-complete probabilistic programming language. Edward defines two compositional representations---random variables and inference. By treating inference as a first class citizen, on a par with modeling, we show that probabilistic programming can be as flexible and computationally efficient as traditional deep learning. For flexibility, Edward makes it easy to fit the same model using a variety of composable inference methods, ranging from point estimation to variational inference to MCMC. In addition, Edward can reuse the modeling representation as part of inference, facilitating the design of rich variational models and generative adversarial networks. For efficiency, Edward is integrated into TensorFlow, providing significant speedups over existing probabilistic systems. For example, we show on a benchmark logistic regression task that Edward is at least 35x faster than Stan and 6x faster than PyMC3. Further, Edward incurs no runtime overhead: it is as fast as handwritten TensorFlow.
Cited by 107 works. doi:10.48550/arxiv.1701.03757 Open-access PDF
Authors
References
- Training Products of Experts by Minimizing Contrastive Divergence
- On the quantitative analysis of deep belief networks
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Sampling-Based Approaches to Calculating Marginal Densities
- TensorFlow: A system for large-scale machine learning
- An Introduction to Variational Methods for Graphical Models
- On sequential Monte Carlo sampling methods for Bayesian filtering
- Large-Scale Machine Learning with Stochastic Gradient Descent
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks