Lecture

Gradient Descent

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-512: Optimization on manifolds

We develop, analyze and implement numerical algorithms to solve optimization problems of the form min f(x) where x is a point on a smooth manifold. To this end, we first study differential and Riemannian geometry (with a focus dictated by computational con ...

Description

This lecture covers optimization on manifolds, focusing on a family of gradient descent methods to minimize a smooth function on a manifold. The algorithm template for (Riemannian) gradient descent is discussed, emphasizing the aim to guarantee small gradients. The lecture also delves into Taylor expansions, first-order, and the concept of retraction and Riemannian metric on the manifold. The instructor explains the algorithm's steps and highlights the importance of ensuring small gradients for convergence. The lecture concludes with insights on global convergence and the regularity assumption, particularly the Lipschitz-type assumption. Various mathematical expressions and theorems are presented to support the concepts discussed.

In MOOC

Introduction to optimization on smooth manifolds: first order methods

Optimization on manifolds is the result of smooth geometry and optimization merging into one elegant modern framework. We start the course at "What is a manifold?", and give the students a firm understanding of submanifolds embedded in real space. This covers numerous applications in engineering and the sciences. All definitions and theorems are motivated to build time-tested optimization algorithms. The math is precise, to promote understanding and enable computation. We build our way up to Riemannian g

Instructor

Nicolas Boumal

Teaching & PhD PhD Students Andreea-Alexandra Musat Past EPFL PhD Students Christopher Criscitiello, Quentin Rebjock Courses Continuous optimization MATH-329 This course introduces students to continuous, nonlinear optimization. We study the theory of optimization with continuous variables (with full proofs), and we analyze and implement important algorithms to solve constrained and unconstrained problems. Linear Algebra MATH-111(e) The purpose of the course is to introduce

Official source

Related concepts (32)

Differentiable manifold

In mathematics, a differentiable manifold (also differential manifold) is a type of manifold that is locally similar enough to a vector space to allow one to apply calculus. Any manifold can be described by a collection of charts (atlas). One may then apply ideas from calculus while working within the individual charts, since each chart lies within a vector space to which the usual rules of calculus apply. If the charts are suitably compatible (namely, the transition from one chart to another is differentiable), then computations done in one chart are valid in any other differentiable chart.

Pseudo-Riemannian manifold

In differential geometry, a pseudo-Riemannian manifold, also called a semi-Riemannian manifold, is a differentiable manifold with a metric tensor that is everywhere nondegenerate. This is a generalization of a Riemannian manifold in which the requirement of positive-definiteness is relaxed. Every tangent space of a pseudo-Riemannian manifold is a pseudo-Euclidean vector space. A special case used in general relativity is a four-dimensional Lorentzian manifold for modeling spacetime, where tangent vectors can be classified as timelike, null, and spacelike.

Manifold

In mathematics, a manifold is a topological space that locally resembles Euclidean space near each point. More precisely, an -dimensional manifold, or -manifold for short, is a topological space with the property that each point has a neighborhood that is homeomorphic to an open subset of -dimensional Euclidean space. One-dimensional manifolds include lines and circles, but not lemniscates. Two-dimensional manifolds are also called surfaces. Examples include the plane, the sphere, and the torus, and also the Klein bottle and real projective plane.

Riemannian manifold

In differential geometry, a Riemannian manifold or Riemannian space (M, g), so called after the German mathematician Bernhard Riemann, is a real, smooth manifold M equipped with a positive-definite inner product gp on the tangent space TpM at each point p. The family gp of inner products is called a Riemannian metric (or Riemannian metric tensor). Riemannian geometry is the study of Riemannian manifolds. A common convention is to take g to be smooth, which means that for any smooth coordinate chart (U, x) on M, the n2 functions are smooth functions.

Riemannian geometry

Riemannian geometry is the branch of differential geometry that studies Riemannian manifolds, defined as smooth manifolds with a Riemannian metric (an inner product on the tangent space at each point that varies smoothly from point to point). This gives, in particular, local notions of angle, length of curves, surface area and volume. From those, some other global quantities can be derived by integrating local contributions.

Related lectures (64)

Differential Forms on Manifolds

Introduces differential forms on manifolds, covering tangent bundles and intersection pairings.

Retractions vector fields and tangent bundles: Tangent bundles MOOC: Introduction to optimization on smooth manifolds: first order methods

Covers retractions, tangent bundles, and embedded submanifolds on manifolds with proofs and examples.

Differentiating vector fields: Why do it?

Explores the importance of differentiating vector fields and the correct methodology to achieve it, emphasizing the significance of going beyond the first order.

Riemannian metrics and gradients: Examples and Riemannian submanifolds MOOC: Introduction to optimization on smooth manifolds: first order methods

Explores Riemannian metrics on manifolds and the concept of Riemannian submanifolds in Euclidean spaces.

Riemannian metrics and gradients: Why and definition of Riemannian manifolds MOOC: Introduction to optimization on smooth manifolds: first order methods