Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Related lectures (44)

Gradient Descent Methods: Theory and Computation

Explores gradient descent methods for smooth convex and non-convex problems, covering iterative strategies, convergence rates, and challenges in optimization.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.

Gradient Descent Methods

Covers gradient descent methods for convex and nonconvex problems, including smooth unconstrained convex minimization, maximum likelihood estimation, and examples like ridge regression and image classification.

Structures in Non-Convex Optimization

Delves into structures in non-convex optimization, emphasizing scalable optimization for deep learning.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Optimization Methods: Convergence and Trade-offs

Covers optimization methods, convergence guarantees, trade-offs, and variance reduction techniques in numerical optimization.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Optimization Techniques: Convexity in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity and its implications for efficient problem-solving.

Structures in Non-Convex Optimization

Explores non-convex optimization in deep learning, covering critical points, SGD convergence, saddle points, and adaptive gradient methods.

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.

Optimization Methods: Convexity and Gradient Descent

Explores optimization methods, including convexity, gradient descent, and non-convex minimization, with examples like maximum likelihood estimation and ridge regression.

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

From Stochastic Gradient Descent to Non-Smooth Optimization

Covers stochastic optimization, sparsity, and non-smooth minimization via subgradient descent.

Adversarial Machine Learning: Fundamentals and Techniques

Explores adversarial machine learning, covering the generation of adversarial examples, robustness challenges, and techniques like Fast Gradient Sign Method.

Optimality of Convergence Rate: Acceleration in Gradient Descent

Explores the optimality of convergence rate in gradient descent and acceleration techniques for convex and non-convex problems.

Stochastic Gradient Descent: Optimization Techniques

Explores Stochastic Gradient Descent with Averaging, comparing it with Gradient Descent, and discusses challenges in non-convex optimization and sparse recovery techniques.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Gradient Descent: Principles and Applications

Covers gradient descent, its principles, applications, and convergence rates in optimization for machine learning.

Deep Learning III

Delves into deep learning optimization, challenges, SGD variants, critical points, overparametrized networks, and adaptive methods.