Lecture

Overfitting vs Underfitting

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture discusses common mistakes in machine learning, focusing on overfitting and underfitting. It explains how model complexity affects error rates, using polynomial functions as examples. The instructor demonstrates how to find the optimal complexity for a model and provides solutions for handling overfitting.

In MOOC

IoT Systems and Industrial Applications with Design Thinking

Internet of Things (IoT) and smart connected devices have radically changed the way our world works and how companies operate and create new businesses. Do you want to learn about IoT fundamentals? This is a course for you! In this course we provide an overview of the underlaying IoT technologies and its business applications. This course is an introduction to IoT and explains why it is a continuously evolving concept with many industrial applications. You will learn the fundamental building blocks of IoT

Official source

Related concepts (25)

Overfitting

In mathematical modeling, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably". An overfitted model is a mathematical model that contains more parameters than can be justified by the data. In a mathematical sense, these parameters represent the degree of a polynomial. The essence of overfitting is to have unknowingly extracted some of the residual variation (i.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Hermite polynomials

In mathematics, the Hermite polynomials are a classical orthogonal polynomial sequence. The polynomials arise in: signal processing as Hermitian wavelets for wavelet transform analysis probability, such as the Edgeworth series, as well as in connection with Brownian motion; combinatorics, as an example of an Appell sequence, obeying the umbral calculus; numerical analysis as Gaussian quadrature; physics, where they give rise to the eigenstates of the quantum harmonic oscillator; and they also occur in some cases of the heat equation (when the term is present); systems theory in connection with nonlinear operations on Gaussian noise.

Generalization error

For supervised learning applications in machine learning and statistical learning theory, generalization error (also known as the out-of-sample error or the risk) is a measure of how accurately an algorithm is able to predict outcome values for previously unseen data. Because learning algorithms are evaluated on finite samples, the evaluation of a learning algorithm may be sensitive to sampling error. As a result, measurements of prediction error on the current data may not provide much information about predictive ability on new data.

Bias–variance tradeoff

In statistics and machine learning, the bias–variance tradeoff is the property of a model that the variance of the parameter estimated across samples can be reduced by increasing the bias in the estimated parameters. The bias–variance dilemma or bias–variance problem is the conflict in trying to simultaneously minimize these two sources of error that prevent supervised learning algorithms from generalizing beyond their training set: The bias error is an error from erroneous assumptions in the learning algorithm.

Related lectures (64)

Model Assessment and Hyperparameter Tuning

Explores model assessment, hyperparameter tuning, and resampling strategies in machine learning.

Complexity: Approximation-Estimation Trade-off

Explores the control of complexity in hypothesis spaces and the trade-off between approximation and estimation in risk decomposition.

Generalization in Deep Learning

Delves into the trade-off between model complexity and risk, generalization bounds, and the dangers of overfitting complex function classes.

Generalization Theory

Explores generalization theory in machine learning, addressing challenges in higher-dimensional spaces and the bias-variance tradeoff.

Overfitting: Symptoms and Characteristics