Algorithme de Frank-Wolfe

The Frank–Wolfe algorithm is an iterative first-order optimization algorithm for constrained convex optimization. Also known as the conditional gradient method, reduced gradient algorithm and the convex combination algorithm, the method was originally proposed by Marguerite Frank and Philip Wolfe in 1956. In each iteration, the Frank–Wolfe algorithm considers a linear approximation of the objective function, and moves towards a minimizer of this linear function (taken over the same domain). Suppose is a compact convex set in a vector space and is a convex, differentiable real-valued function. The Frank–Wolfe algorithm solves the optimization problem Minimize subject to . Initialization: Let , and let be any point in . Step 1. Direction-finding subproblem: Find solving Minimize Subject to (Interpretation: Minimize the linear approximation of the problem given by the first-order Taylor approximation of around constrained to stay within .) Step 2. Step size determination: Set , or alternatively find that minimizes subject to . Step 3. Update: Let , let and go to Step 1. While competing methods such as gradient descent for constrained optimization require a projection step back to the feasible set in each iteration, the Frank–Wolfe algorithm only needs the solution of a linear problem over the same set in each iteration, and automatically stays in the feasible set. The convergence of the Frank–Wolfe algorithm is sublinear in general: the error in the objective function to the optimum is after k iterations, so long as the gradient is Lipschitz continuous with respect to some norm. The same convergence rate can also be shown if the sub-problems are only solved approximately. The iterates of the algorithm can always be represented as a sparse convex combination of the extreme points of the feasible set, which has helped to the popularity of the algorithm for sparse greedy optimization in machine learning and signal processing problems, as well as for example the optimization of minimum–cost flows in transportation networks.