Lecture

Optimization: Gradient Descent and Subgradients

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Linear Models: Recap and Logistic Regression

Covers linear models, binary classification, logistic regression, and model evaluation metrics.

Gradient Descent

Covers the concept of gradient descent in scalar cases, focusing on finding the minimum of a function by iteratively moving in the direction of the negative gradient.

Optimization Basics

Introduces optimization basics, covering logistic regression, derivatives, convex functions, gradient descent, and second-order methods.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.

Gradient Descent: Principles and Applications

Covers gradient descent, its principles, applications, and convergence rates in optimization for machine learning.

Proximal Operators: Optimization Methods

Explores proximal operators, subgradient methods, and composite minimization in optimization.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Linear Models for Classification

Explores linear models for classification, including parametric models, regression, and logistic regression, along with model evaluation metrics and maximum margin classifiers.

Linear Models and Overfitting

Explores linear models, overfitting, and the importance of feature expansion and adding more data to reduce overfitting.

Optimality of Convergence Rate: Acceleration in Gradient Descent

Explores the optimality of convergence rate in gradient descent and acceleration techniques for convex and non-convex problems.

Optimization Techniques: Gradient Method Overview

Discusses the gradient method for optimization, focusing on its application in machine learning and the conditions for convergence.

Multilayer Perceptron: Training and Optimization

Explores the multilayer perceptron model, training, optimization, data preprocessing, activation functions, backpropagation, and regularization.

Linear Models: Continued

Explores linear models, logistic regression, gradient descent, and multi-class logistic regression with practical applications and examples.

Iterative Descent Methods: Optimization Principles

Explores iterative descent methods in optimization, gradient descent, local minima, and convergence principles.

Linear and Logistic Regression

Introduces linear and logistic regression, covering parametric models, multi-output prediction, non-linearity, gradient descent, and classification applications.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Adaptive Gradient Methods: Part 1

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Proximal gradient descent and intro linear models

Covers proximal gradient descent and linear models in regression analysis.

Regression Diagnostics

Covers regression diagnostics for linear models, emphasizing the importance of checking assumptions and identifying outliers and influential observations.