Gradient Descent: Proximal Operator and Step-Size Strategies

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (32)

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Linear Systems: Convergence and Methods

Explores linear systems, convergence, and solving methods with a focus on CPU time and memory requirements.

Gradient Descent

Covers the concept of gradient descent in scalar cases, focusing on finding the minimum of a function by iteratively moving in the direction of the negative gradient.

Optimization Methods

Covers optimization methods without constraints, including gradient and line search in the quadratic case.

Convergence Analysis: Stochastic Gradient Algorithms

Explores the convergence analysis of stochastic gradient algorithms under various operational modes and step-size sequences.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Newton Method: Convergence and Quadratic Care

Covers the Newton method and its convergence properties near the optimal point.

Uniform Integrability and Convergence

Explores uniform integrability, convergence theorems, and the importance of bounded sequences in understanding the convergence of random variables.

Descent Methods and Newton Method with Line Search MOOC: Optimization: principles and algorithms - Unconstrained nonlinear optimization

Covers the comparison between descent methods and Newton's method, modifications, line search, and convergence analysis.

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Gradient Descent

Covers the algorithm of gradient descent, aiming to minimize a function by iteratively moving in the direction of the steepest decrease.

Optimization Techniques: Gradient Method Overview

Discusses the gradient method for optimization, focusing on its application in machine learning and the conditions for convergence.

Coordinate Descent: Efficient Optimization Techniques

Covers coordinate descent, a method for optimizing functions by updating one coordinate at a time.

Proximal Operator and Gradient Descent

Covers proximal operators, gradient descent, optimality conditions, and convergence analysis in optimization problems.

Adaptive Gradient Methods: Part 1

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Proximal and Subgradient Descent: Optimization Techniques

Discusses proximal and subgradient descent methods for optimization in machine learning.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex minimization problems using accelerated gradient descent methods.

Proximal Gradient Descent: Optimization Techniques in Machine Learning Log in to Mediaspace to watch this video

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Linear Systems: Iterative Methods Log in to Mediaspace to watch this video

Explores linear systems and iterative methods like gradient descent and conjugate gradient for efficient solutions.

Page 1 of 2