Gradient methods for minimizing composite objective function
Yurii E. Nesterov · 2007
In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two convex terms: one is smooth and given by a black-box oracle, and another is general but simple and its structure is known. Despite to the bad properties of the sum, such problems, both in convex and nonconvex cases, can be solved with efficiency typical for the good part of the objective. For convex problems of the above structure, we consider primal and dual variants of the gradient method (converge as O (1/k)), and an accelerated multistep version with convergence rate O (1/k2), where k is\nthe iteration counter. For all methods, we suggest some efficient "line search" procedures and show that the additional computational work necessary for estimating the unknown problem class parameters can only multiply the complexity of each iteration by a small constant factor. We present also the results of preliminary computational experiments, which confirm the superiority of the accelerated scheme.
Cited by 863 works.
Authors
References
- Regression Shrinkage and Selection Via the Lasso
- Iterative Solution of Nonlinear Equations in Several Variables
- Atomic Decomposition by Basis Pursuit
- Subset Selection in Regression
- Introductory Lectures on Convex Optimization
- Just relax: convex programming methods for identifying sparse signals in noise
- Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems
- Smooth minimization of non-smooth functions
- Deconvolution with the l 1 norm
- Robust Modeling With Erratic Data