Lecture

Regularization Methods: Training and Validation Base

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Model Selection and Evaluation

Discusses the experimental framework for selecting and evaluating supervised learning models to prevent overfitting.

Gradient-Based Algorithms in High-Dimensional Learning

Provides insights on gradient-based algorithms, deep learning mysteries, and the challenges of non-convex problems.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Linear Regression and Logistic Regression

Covers linear and logistic regression for regression and classification tasks, focusing on loss functions and model training.

Careful Cross-validation

Emphasizes the significance of careful cross-validation in deep neural networks, including the split of data and the concept of K-fold cross-validation.

Specification Testing and Machine Learning

Explores specification testing, machine learning, overfitting, regularization, prediction tests, and variable selection.

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

Neural Networks: Multilayer Learning

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.

Deep Learning: Convolutional Neural Networks

Introduces Convolutional Neural Networks, explaining their architecture, training process, and applications in semantic segmentation tasks.

Training Neural Networks

Covers the training of neural networks using stochastic gradient descent, chain rules, gradients computation, weight decay, and dropout.

Bayesian Statistics: Regularization and Divergence

Covers Kullback-Leibler divergence, regularization, and Bayesian statistics to combat overfitting in machine learning models.

Kernel Methods

Covers overfitting, model selection, validation methods, kernel functions, and SVM concepts.

Building Neural Networks: Assembly Strategies

Focuses on assembling neural network building blocks and dealing with data sparseness using various strategies and assumptions.

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Universal Approximation Theorem: MLP

Covers Multi-Layer Perceptrons (MLP) and their application from classification to regression, including the Universal Approximation Theorem and challenges with gradients.

Neural Networks: Basics and History

Covers the history and fundamental concepts of neural networks, including the mathematical model of a neuron, gradient descent, and the multilayer perceptron.

Comparing L1 and L0 + Greedy algorithms

Compares L1 and L0 penalization in linear regression with orthogonal designs using greedy algorithms and empirical comparisons.

Convolutional Neural Networks

Introduces Convolutional Neural Networks (CNNs) for autonomous vehicles, covering architecture, applications, and regularization techniques.

Quantum Many-body Dynamics with Artificial Neural Networks

Explores quantum many-body dynamics using artificial neural networks, focusing on experimental simulations and theoretical challenges.

Generalization in Deep Learning

Explores generalization in deep learning, covering model complexity, implicit bias, and the double descent phenomenon.