Lectures related to Gradient Descent Methods

Gradient Descent Methods: Theory and Computation

Explores gradient descent methods for smooth convex and non-convex problems, covering iterative strategies, convergence rates, and challenges in optimization.

Optimization Methods: Convexity and Gradient Descent

Explores optimization methods, including convexity, gradient descent, and non-convex minimization, with examples like maximum likelihood estimation and ridge regression.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Stochastic Gradient Descent: Optimization and Convergence

Explores stochastic gradient descent, covering convergence rates, acceleration, and practical applications in optimization problems.

From Stochastic Gradient Descent to Non-Smooth Optimization

Covers stochastic optimization, sparsity, and non-smooth minimization via subgradient descent.

Optimization Methods: Convergence and Trade-offs

Covers optimization methods, convergence guarantees, trade-offs, and variance reduction techniques in numerical optimization.

Proximal and Subgradient Descent: Optimization Techniques

Discusses proximal and subgradient descent methods for optimization in machine learning.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Optimization Basics: Norms, Convexity, Differentiability

Explores optimization basics such as norms, convexity, and differentiability, along with practical applications and convergence rates.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Structured Sparsity: Atomic Norms and Convex Optimization

Explores atomic norms, convex optimization, and structured sparsity in mathematical data analysis.

Faster and Projected Gradient Descent: Optimization Techniques

Discusses advanced optimization techniques, focusing on faster and projected gradient descent methods in machine learning.

Proximal Operators and Constrained Optimization

Introduces proximal operators, gradient methods, and constrained optimization, exploring their convergence and practical applications.

Optimization Techniques: Gradient Method Overview

Discusses the gradient method for optimization, focusing on its application in machine learning and the conditions for convergence.

Stochastic Gradient Descent: Optimization Techniques

Explores Stochastic Gradient Descent with Averaging, comparing it with Gradient Descent, and discusses challenges in non-convex optimization and sparse recovery techniques.

Non-Convex Optimization: Techniques and Applications Log in to Mediaspace to watch this video

Covers non-convex optimization techniques and their applications in machine learning.

Proximal Gradient Descent: Optimization Techniques in Machine Learning Log in to Mediaspace to watch this video

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Faster Gradient Descent: Projected Optimization Techniques Log in to Mediaspace to watch this video

Covers faster gradient descent methods and projected gradient descent for constrained optimization in machine learning.

Optimization Techniques: Gradient Descent and Convex Functions Log in to Mediaspace to watch this video

Provides an overview of optimization techniques, focusing on gradient descent and properties of convex functions in machine learning.