Automatic Reparameterisation of Probabilistic Programs
Maria I. Gorinova, Dave Moore, Matthew D. Hoffman · arXiv (Cornell University) · 2019
Probabilistic programming has emerged as a powerful paradigm in statistics, applied science, and machine learning: by decoupling modelling from inference, it promises to allow modellers to directly reason about the processes generating data. However, the performance of inference algorithms can be dramatically affected by the parameterisation used to express a model, requiring users to transform their programs in non-intuitive ways. We argue for automating these transformations, and demonstrate that mechanisms available in recent modeling frameworks can implement non-centring and related reparameterisations. This enables new inference algorithms, and we propose two: a simple approach using interleaved sampling and a novel variational formulation that searches over a continuous space of parameterisations. We show that these approaches enable robust inference across a range of models, and can yield more efficient samplers than the best fixed parameterisation.
Cited by 4 works. doi:10.48550/arxiv.1906.03028 Open-access PDF
Authors
References
- Adam: A Method for Stochastic Optimization
- Book Review: Pattern Recognition and Machine Learning
- UCI Machine Learning Repository
- Data Analysis Using Regression and Multilevel/Hierarchical Models
- Slice Sampling
- A tutorial on adaptive MCMC
- Estimation in Parallel Randomized Experiments
- A General Framework for the Parametrization of Hierarchical Models
- Hamiltonian Monte Carlo for Hierarchical Models
- Ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC estimation of stochastic volatility models