Lecture

Generalization in Deep Learning

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Deep Learning Modus Operandi

Explores the benefits of deeper networks in deep learning and the importance of over-parameterization and generalization.

Nonlinear ML Algorithms

Introduces nonlinear ML algorithms, covering nearest neighbor, k-NN, polynomial curve fitting, model complexity, overfitting, and regularization.

Deep Learning III

Delves into deep learning optimization, challenges, SGD variants, critical points, overparametrized networks, and adaptive methods.

Subquadratic Attention Mechanisms: State Space Models Overview

Covers subquadratic attention mechanisms and state space models, focusing on their theoretical foundations and practical implementations in machine learning.

Deep Learning Building Blocks: Linear Layers

Explains the fundamental building blocks of deep learning, focusing on linear layers and activation functions.

Deep Learning: No Free Lunch Theorem and Inductive Bias

Covers the No Free Lunch Theorem and the role of inductive bias in deep learning and reinforcement learning.

Universal Approximation Theorem: MLP

Covers Multi-Layer Perceptrons (MLP) and their application from classification to regression, including the Universal Approximation Theorem and challenges with gradients.

Convolutional Neural Networks

Introduces Convolutional Neural Networks (CNNs) for autonomous vehicles, covering architecture, applications, and regularization techniques.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Linear Algebra: Efficiency and Complexity

Explores constraints, efficiency, and complexity in linear algebra, emphasizing convexity and worst-case complexity in algorithm analysis.

Primal-dual Optimization: Extra-Gradient Method

Explores the Extra-Gradient method for Primal-dual optimization, covering nonconvex-concave problems, convergence rates, and practical performance.

Curse of Dimensionality in Deep Learning

Delves into the challenges of deep learning, exploring dimensionality, performance, and overfitting phenomena in neural networks.

Deep Learning Building Blocks

Covers tensors, loss functions, autograd, and convolutional layers in deep learning.

Model Complexity and Overfitting in Machine Learning

Covers model complexity, overfitting, and strategies to select appropriate machine learning models.

Neural Networks Optimization

Explores neural networks optimization, including backpropagation, batch normalization, weight initialization, and hyperparameter search strategies.

Machine Learning Fundamentals: Overfitting and Regularization

Covers overfitting, regularization, and cross-validation in machine learning, exploring polynomial curve fitting, feature expansion, kernel functions, and model selection.

Structures in Non-Convex Optimization

Delves into structures in non-convex optimization, emphasizing scalable optimization for deep learning.

Provable Benefits of Overparameterization in Model Compression

Explores the provable benefits of overparameterization in model compression, emphasizing the efficiency of deep neural networks and the importance of retraining for improved performance.

No Free Lunch Theorem: Deep Learning

Explores the No Free Lunch Theorem and how deep networks match real-world problem structures.

The Wrong Method: Polynomial Interpolation MOOC: Numerical Analysis for Engineers

Explores the inefficiencies of the incorrect method for polynomial interpolation.