Lecture

Descent methods and line search: Second Wolfe condition

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture delves into the second Wolfe condition, which aims to avoid steps that are too short along a descent direction. By requiring a sufficient increase in the directional derivative, this condition provides flexibility to the algorithm, allowing for different step sizes based on the parameter beta_2. Through illustrations and examples, the instructor explains how the second Wolfe condition characterizes the progress of the algorithm and influences the acceptance or rejection of steps, ultimately impacting the optimization process.

In MOOC

Optimization: principles and algorithms - Unconstrained nonlinear optimization

What you'll learn Formulation: you will learn from simple examples how to formulate, transform and characterize an optimization problem. Objective function: you will review the mathematical properties of the objective function that are important in optimization. Optimality conditions: you will learn sufficient and necessary conditions for an optimal solution. Solving equations, Newton: this is a reminder about Newton's method to solve nonlinear equations. Newton's local method: you will see how to interpret

Official source

Related concepts (13)

Directional derivative

A directional derivative is a concept in multivariable calculus that measures the rate at which a function changes in a particular direction at a given point. The directional derivative of a multivariable differentiable (scalar) function along a given vector v at a given point x intuitively represents the instantaneous rate of change of the function, moving through x with a velocity specified by v. The directional derivative of a scalar function f with respect to a vector v at a point (e.g.

Derivative

In mathematics, the derivative shows the sensitivity of change of a function's output with respect to the input. Derivatives are a fundamental tool of calculus. For example, the derivative of the position of a moving object with respect to time is the object's velocity: this measures how quickly the position of the object changes when time advances. The derivative of a function of a single variable at a chosen input value, when it exists, is the slope of the tangent line to the graph of the function at that point.

Covariant derivative

In mathematics, the covariant derivative is a way of specifying a derivative along tangent vectors of a manifold. Alternatively, the covariant derivative is a way of introducing and working with a connection on a manifold by means of a differential operator, to be contrasted with the approach given by a principal connection on the frame bundle – see affine connection. In the special case of a manifold isometrically embedded into a higher-dimensional Euclidean space, the covariant derivative can be viewed as the orthogonal projection of the Euclidean directional derivative onto the manifold's tangent space.

Fréchet derivative

In mathematics, the Fréchet derivative is a derivative defined on normed spaces. Named after Maurice Fréchet, it is commonly used to generalize the derivative of a real-valued function of a single real variable to the case of a vector-valued function of multiple real variables, and to define the functional derivative used widely in the calculus of variations. Generally, it extends the idea of the derivative from real-valued functions of one real variable to functions on normed spaces.

Exterior derivative

On a differentiable manifold, the exterior derivative extends the concept of the differential of a function to differential forms of higher degree. The exterior derivative was first described in its current form by Élie Cartan in 1899. The resulting calculus, known as exterior calculus, allows for a natural, metric-independent generalization of Stokes' theorem, Gauss's theorem, and Green's theorem from vector calculus.

Related lectures (50)

Theorem and Explanations MOOC: Analysis I (part 5): Continuous and derivable functions, the derivative function MOOC: Analysis I

Covers the theorem of finite increments in derivative functions.

Optimization Techniques

Covers optimization techniques and their application in solving real-world problems.

Rescuing Data Center Processors

Explores challenges and solutions for data center processors, focusing on efficiency, cache issues, branch prediction, and architectural optimizations.

Descent methods and line search: First Wolfe condition MOOC: Optimization: principles and algorithms - Unconstrained nonlinear optimization

Introduces the First Wolfe condition to ensure a proportional decrease in the objective function relative to the step length.

Partial Derivatives: Functions and Definitions