Lecture

Truncated Conjugate Gradients for Trust-Region Subproblem

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-512: Optimization on manifolds

We develop, analyze and implement numerical algorithms to solve optimization problems of the form min f(x) where x is a point on a smooth manifold. To this end, we first study differential and Riemannian geometry (with a focus dictated by computational con ...

Description

This lecture covers the concept of truncated conjugate gradients for solving the trust-region subproblem in optimization on manifolds. The instructor introduces the trust-region subproblem and explains the strategy of running conjugate gradients cautiously and opportunistically. The lecture details the iterative process of truncated CG, emphasizing the importance of monitoring signs and boundaries to efficiently find approximate solutions. Various modifications to the CG algorithm are discussed, including early stopping criteria and checking for convergence, ensuring an effective optimization process.

Instructor

Nicolas Boumal

Teaching & PhD PhD Students Andreea-Alexandra Musat Past EPFL PhD Students Christopher Criscitiello, Quentin Rebjock Courses Continuous optimization MATH-329 This course introduces students to continuous, nonlinear optimization. We study the theory of optimization with continuous variables (with full proofs), and we analyze and implement important algorithms to solve constrained and unconstrained problems. Linear Algebra MATH-111(e) The purpose of the course is to introduce

Official source

Related concepts (29)

Iterative method

In computational mathematics, an iterative method is a mathematical procedure that uses an initial value to generate a sequence of improving approximate solutions for a class of problems, in which the n-th approximation is derived from the previous ones. A specific implementation with termination criteria for a given iterative method like gradient descent, hill climbing, Newton's method, or quasi-Newton methods like BFGS, is an algorithm of the iterative method.

Differentiable manifold

In mathematics, a differentiable manifold (also differential manifold) is a type of manifold that is locally similar enough to a vector space to allow one to apply calculus. Any manifold can be described by a collection of charts (atlas). One may then apply ideas from calculus while working within the individual charts, since each chart lies within a vector space to which the usual rules of calculus apply. If the charts are suitably compatible (namely, the transition from one chart to another is differentiable), then computations done in one chart are valid in any other differentiable chart.

Jacobi method

In numerical linear algebra, the Jacobi method (a.k.a. the Jacobi iteration method) is an iterative algorithm for determining the solutions of a strictly diagonally dominant system of linear equations. Each diagonal element is solved for, and an approximate value is plugged in. The process is then iterated until it converges. This algorithm is a stripped-down version of the Jacobi transformation method of matrix diagonalization. The method is named after Carl Gustav Jacob Jacobi.

Complex manifold

In differential geometry and complex geometry, a complex manifold is a manifold with an atlas of charts to the open unit disc in , such that the transition maps are holomorphic. The term complex manifold is variously used to mean a complex manifold in the sense above (which can be specified as an integrable complex manifold), and an almost complex manifold. Since holomorphic functions are much more rigid than smooth functions, the theories of smooth and complex manifolds have very different flavors: compact complex manifolds are much closer to algebraic varieties than to differentiable manifolds.

Kähler manifold

In mathematics and especially differential geometry, a Kähler manifold is a manifold with three mutually compatible structures: a complex structure, a Riemannian structure, and a symplectic structure. The concept was first studied by Jan Arnoldus Schouten and David van Dantzig in 1930, and then introduced by Erich Kähler in 1933. The terminology has been fixed by André Weil.

Related lectures (64)

Computing the Newton Step: GD as a Matrix-Free Way

Explores matrix-based and matrix-free approaches for computing the Newton step in optimization on manifolds.

Transporters: a proxy for parallel transport

Explores transporters as a practical alternative to parallel transport, discussing minimal requirements, examples with matrices, pragmatic choices, and optimization algorithms.

Momentum Methods and Nonlinear Conjugate Gradients

The session explains how momentum accelerates gradient descent, presents the theory of nonlinear conjugate gradients on manifolds, compares several beta‑rule updates, discusses line‑search and preconditioner choices, and shows how to employ the Manopt library for Riemannian optimization without explicit Hessians.

Optimization on Manifolds: Context and Applications MOOC: Introduction to optimization on smooth manifolds: first order methods

Introduces optimization on manifolds, covering classical and modern techniques in the field.

Geodesics in Wasserstein Space