Lecture

From Stochastic Gradient Descent to Non-Smooth Optimization

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Linear Regression: Statistical Inference and Regularization

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Gradient Descent and Linear Regression

Covers stochastic gradient descent, linear regression, regularization, supervised learning, and the iterative nature of gradient descent.

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Probabilistic Models for Linear Regression

Covers the probabilistic model for linear regression and its applications in nuclear magnetic resonance and X-ray imaging.

Concise Signal Models and Compressive Sensing

Explores concise signal models, compressive sensing, sparsity, atomic norms, and non-smooth minimization using subgradient descent.

Gradient Descent Methods

Covers gradient descent methods for convex and nonconvex problems, including smooth unconstrained convex minimization, maximum likelihood estimation, and examples like ridge regression and image classification.

Gradient Descent Methods: Theory and Computation

Explores gradient descent methods for smooth convex and non-convex problems, covering iterative strategies, convergence rates, and challenges in optimization.

Probabilistic Linear Regression

Explores probabilistic linear regression, covering joint and conditional probability, ridge regression, and overfitting mitigation.

Linear Regression and Logistic Regression

Covers linear and logistic regression for regression and classification tasks, focusing on loss functions and model training.

Overfitting in Supervised Learning: Case Studies and Techniques

Addresses overfitting in supervised learning through polynomial regression case studies and model selection techniques.

Loss Functions and Optimization

Explores loss functions, gradient descent, and step size impact on optimization in machine learning models, highlighting the delicate balance required for efficient convergence.

Lecture 02

Non-smoothness and Compressive Sensing

Explores non-smooth minimization, compressive sensing, sparse signal recovery, and simple representations using atomic sets and atoms.

Linear Regression: Basics and Gradient Descent

Covers the basics of linear regression, including feature engineering, supervised vs. unsupervised learning, and minimizing the cost function.

Linear Regression: Foundations and Applications

Introduces linear regression, covering its fundamentals, applications, and evaluation metrics in machine learning.

Understanding Data Attributes

Covers the analysis of various data attributes and linear regression models.

Linear and Logistic Regression

Introduces linear and logistic regression, covering parametric models, multi-output prediction, non-linearity, gradient descent, and classification applications.

Convex Optimization: Examples of Convex Functions

Explores convex optimization, convex functions, and their properties, including strict convexity and strong convexity, as well as different types of convex functions like linear affine functions and norms.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.