Lecture

The Conjugate Gradients Method (CG)

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-512: Optimization on manifolds

We develop, analyze and implement numerical algorithms to solve optimization problems of the form min f(x) where x is a point on a smooth manifold. To this end, we first study differential and Riemannian geometry (with a focus dictated by computational con ...

Description

This lecture introduces the Conjugate Gradients method, a technique for solving linear systems with a positive definite operator. Starting with the definition and properties of CG, the instructor explains the iterative process, the computation of residuals, and the concept of conjugate directions. The lecture covers the convergence properties and the necessary conditions for quadratic convergence, emphasizing the importance of linear independence among the conjugate directions. The instructor also discusses the representation of Hessians and the Riemannian distance on a manifold, providing insights into the practical implementation of CG.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Nicolas Boumal

Teaching & PhD PhD Students Andreea-Alexandra Musat Past EPFL PhD Students Christopher Criscitiello, Quentin Rebjock Courses Continuous optimization MATH-329 This course introduces students to continuous, nonlinear optimization. We study the theory of optimization with continuous variables (with full proofs), and we analyze and implement important algorithms to solve constrained and unconstrained problems. Linear Algebra MATH-111(e) The purpose of the course is to introduce

Official source

Related concepts (32)

System of linear equations

In mathematics, a system of linear equations (or linear system) is a collection of one or more linear equations involving the same variables. For example, is a system of three equations in the three variables x, y, z. A solution to a linear system is an assignment of values to the variables such that all the equations are simultaneously satisfied. A solution to the system above is given by the ordered triple since it makes all three equations valid. The word "system" indicates that the equations should be considered collectively, rather than individually.

Iterative method

In computational mathematics, an iterative method is a mathematical procedure that uses an initial value to generate a sequence of improving approximate solutions for a class of problems, in which the n-th approximation is derived from the previous ones. A specific implementation with termination criteria for a given iterative method like gradient descent, hill climbing, Newton's method, or quasi-Newton methods like BFGS, is an algorithm of the iterative method.

Linear algebra

Linear algebra is the branch of mathematics concerning linear equations such as: linear maps such as: and their representations in vector spaces and through matrices. Linear algebra is central to almost all areas of mathematics. For instance, linear algebra is fundamental in modern presentations of geometry, including for defining basic objects such as lines, planes and rotations. Also, functional analysis, a branch of mathematical analysis, may be viewed as the application of linear algebra to spaces of functions.

Linear independence

In the theory of vector spaces, a set of vectors is said to be if there exists no nontrivial linear combination of the vectors that equals the zero vector. If such a linear combination exists, then the vectors are said to be . These concepts are central to the definition of dimension. A vector space can be of finite dimension or infinite dimension depending on the maximum number of linearly independent vectors. The definition of linear dependence and the ability to determine whether a subset of vectors in a vector space is linearly dependent are central to determining the dimension of a vector space.

Linear combination

In mathematics, a linear combination is an expression constructed from a set of terms by multiplying each term by a constant and adding the results (e.g. a linear combination of x and y would be any expression of the form ax + by, where a and b are constants). The concept of linear combinations is central to linear algebra and related fields of mathematics. Most of this article deals with linear combinations in the context of a vector space over a field, with some generalizations given at the end of the article.

Related lectures (64)

Truncated Conjugate Gradients for Trust-Region Subproblem

Explores truncated conjugate gradients for solving the trust-region subproblem in optimization on manifolds efficiently.

Vectorization in Python: Efficient Computation with Numpy

Covers vectorization in Python using Numpy for efficient scientific computing, emphasizing the benefits of avoiding for loops and demonstrating practical applications.

Conjugate Gradient Method

Covers the Conjugate Gradient method for solving linear systems efficiently.

Linear Systems: Convergence and Methods

Explores linear systems, convergence, and solving methods with a focus on CPU time and memory requirements.

Computing the Newton Step: GD as a Matrix-Free Way