A Tutorial on MM Algorithms
David R. Hunter, Kenneth L. Lange · The American Statistician · 2004
Most problems in frequentist statistics involve optimization of a function such as a likelihood or a sum of squares. EM algorithms are among the most effective algorithms for maximum likelihood estimation because they consistently drive the likelihood uphill by maximizing a simple surrogate function for the log-likelihood. Iterative optimization of a surrogate function as exemplified by an EM algorithm does not necessarily require missing data. Indeed, every EM algorithm is a special case of the more general class of MM optimization algorithms, which typically exploit convexity rather than missing data in majorizing or minorizing an objective function. In our opinion, MM algorithms deserve to be part of the standard toolkit of professional statisticians. This article explains the principle behind MM algorithms, suggests some methods for constructing them, and discusses some of their attractive features. We include numerous examples throughout the article to illustrate the concepts described. In addition to surveying previous work on MM algorithms, this article introduces some new material on constrained optimization and standard error estimation.
Cited by 1,783 works. doi:10.1198/0003130042836
Authors
References
- Maximum Likelihood from Incomplete Data Via the EM Algorithm
- Applied logistic regression
- Iterative Solution of Nonlinear Equations in Several Variables
- Linear and nonlinear programming.
- Inequalities: Theory of Majorization and its Applications.
- The EM Algorithm and Extensions
- Regression Quantiles
- On the Convergence Properties of the EM Algorithm
- Maximum Likelihood Estimation via the ECM Algorithm: A General Framework
- Iterative solution of nonlinear equations