Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Trust Region Methods: Why, with an Example

In course

MATH-512: Optimization on manifolds

We develop, analyze and implement numerical algorithms to solve optimization problems of the form min f(x) where x is a point on a smooth manifold. To this end, we first study differential and Riemannian geometry (with a focus dictated by computational con ...

Description

This lecture introduces trust region methods, explaining their historical development from Levenberg to modern Riemannian manifolds. The instructor discusses the transition from gradient descent to Newton's method and presents an example of Max-Cut Burer-Monteiro rank 2 optimization.

Instructor

Nicolas Boumal

Official source

Related concepts (32)

Riemannian manifold

In differential geometry, a Riemannian manifold or Riemannian space (M, g), so called after the German mathematician Bernhard Riemann, is a real, smooth manifold M equipped with a positive-definite inner product gp on the tangent space TpM at each point p. The family gp of inner products is called a Riemannian metric (or Riemannian metric tensor). Riemannian geometry is the study of Riemannian manifolds. A common convention is to take g to be smooth, which means that for any smooth coordinate chart (U, x) on M, the n2 functions are smooth functions.

Pseudo-Riemannian manifold

In differential geometry, a pseudo-Riemannian manifold, also called a semi-Riemannian manifold, is a differentiable manifold with a metric tensor that is everywhere nondegenerate. This is a generalization of a Riemannian manifold in which the requirement of positive-definiteness is relaxed. Every tangent space of a pseudo-Riemannian manifold is a pseudo-Euclidean vector space. A special case used in general relativity is a four-dimensional Lorentzian manifold for modeling spacetime, where tangent vectors can be classified as timelike, null, and spacelike.

Mathematical optimization

Mathematical optimization (alternatively spelled optimisation) or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfields: discrete optimization and continuous optimization. Optimization problems arise in all quantitative disciplines from computer science and engineering to operations research and economics, and the development of solution methods has been of interest in mathematics for centuries.

Curvature of Riemannian manifolds

In mathematics, specifically differential geometry, the infinitesimal geometry of Riemannian manifolds with dimension greater than 2 is too complicated to be described by a single number at a given point. Riemann introduced an abstract and rigorous way to define curvature for these manifolds, now known as the Riemann curvature tensor. Similar notions have found applications everywhere in differential geometry of surfaces and other objects. The curvature of a pseudo-Riemannian manifold can be expressed in the same way with only slight modifications.

Non-linear least squares

Non-linear least squares is the form of least squares analysis used to fit a set of m observations with a model that is non-linear in n unknown parameters (m ≥ n). It is used in some forms of nonlinear regression. The basis of the method is to approximate the model by a linear one and to refine the parameters by successive iterations. There are many similarities to linear least squares, but also some significant differences.

Related lectures (28)

Geodesic Convexity: Basic Facts and Definitions

Explores geodesic convexity, focusing on properties of convex functions on manifolds.

Optimization on Manifolds: Context and Applications MOOC: Introduction to optimization on smooth manifolds: first order methods

Introduces optimization on manifolds, covering classical and modern techniques in the field.

Optimization Techniques: Gradient Method Overview

Discusses the gradient method for optimization, focusing on its application in machine learning and the conditions for convergence.

Geodesic Convexity: Basic Definitions

Introduces geodesic convexity on Riemannian manifolds and explores its properties.

Linear convergence with Polyak-Łojasiewicz: Mechanical proof