Proximal Algorithms
Neal Parikh, Stephen Boyd · Foundations and Trends® in Optimization · 2014
Proximal Algorithms discusses proximal operators and proximal algorithms, and illustrates their applicability to standard and distributed convex optimization in general and many applications of recent interest in particular. Much like Newton's method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, large-scale, or distributed versions of these problems. They are very generally applicable, but are especially well-suited to problems of substantial recent interest involving large or high-dimensional datasets. Proximal methods sit at a higher level of abstraction than classical algorithms like Newton's method: the base operation is evaluating the proximal operator of a function, which itself involves solving a small convex optimization problem. These subproblems, which generalize the problem of projecting a point onto a convex set, often admit closed-form solutions or can be solved very quickly with standard or simple specialized methods. Proximal Algorithms discusses different interpretations of proximal operators and algorithms, looks at their connections to many other topics in optimization and applied mathematics, surveys some popular algorithms, and provides a large number of examples of proximal operators that commonly arise in practice.
Cited by 3,932 works. doi:10.1561/2400000003
Authors
References
- Convex Optimization
- An Algorithm for Least-Squares Estimation of Nonlinear Parameters
- Regression Shrinkage and Selection Via the Lasso
- A method for the solution of certain non-linear problems in least squares
- Numerical optimization
- Robust Estimation of a Location Parameter
- De-noising by soft-thresholding
- Regularization and Variable Selection Via the Elastic Net
- Partitioning procedures for solving mixed-variables programming problems
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers