Inference from Iterative Simulation Using Multiple Sequences
Andrew Gelman, Donald B. Rubin · Statistical Science · 1992
The Gibbs sampler, the algorithm of Metropolis and similar iterative simulation methods are potentially very helpful for summarizing multivariate distributions. Used naively, however, iterative simulation can give misleading answers. Our methods are simple and generally applicable to the output of any iterative simulation; they are designed for researchers primarily interested in the science underlying the data and models they are analyzing, rather than for researchers interested in the probability theory underlying the iterative simulations themselves. Our recommended strategy is to use several independent sequences, with starting points sampled from an overdispersed distribution. At each step of the iterative simulation, we obtain, for each univariate estimand of interest, a distributional estimate and an estimate of how much sharper the distributional estimate might become if the simulations were continued indefinitely. Because our focus is on applied inference for Bayesian posterior distributions in real problems, which often tend toward normality after transformations and marginalization, we derive our results as normal-theory approximations to exact Bayesian inference, conditional on the observed simulations. The methods are illustrated on a random-effects mixture model applied to experimental measurements of reaction times of normal and schizophrenic patients.
Cited by 16,491 works. doi:10.1214/ss/1177011136 Open-access PDF
Authors
References
- Equation of State Calculations by Fast Computing Machines
- Design of Experiments
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
- Multiple Imputation for Nonresponse in Surveys
- Monte Carlo sampling methods using Markov chains and their applications
- An Approximate Distribution of Estimates of Variance Components
- Sampling-Based Approaches to Calculating Marginal Densities
- The Calculation of Posterior Distributions by Data Augmentation
- Stochastic Simulation
- The Monte Carlo Method