Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Data Dredging

Description

This lecture discusses the concept of data dredging in machine learning, where one iterates between training, validation, and test sets to achieve a good model, highlighting the risks of overfitting and lack of generalizability.

In MOOC

IoT Systems and Industrial Applications with Design Thinking

Official source

Related concepts (15)

Training, validation, and test data sets

In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data. Such algorithms function by making data-driven predictions or decisions, through building a mathematical model from input data. These input data used to build the model are usually divided into multiple data sets. In particular, three data sets are commonly used in different stages of the creation of the model: training, validation, and test sets.

Overfitting

In mathematical modeling, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably". An overfitted model is a mathematical model that contains more parameters than can be justified by the data. In a mathematical sense, these parameters represent the degree of a polynomial. The essence of overfitting is to have unknowingly extracted some of the residual variation (i.

Cross-validation (statistics)

Cross-validation, sometimes called rotation estimation or out-of-sample testing, is any of various similar model validation techniques for assessing how the results of a statistical analysis will generalize to an independent data set. Cross-validation is a resampling method that uses different portions of the data to test and train a model on different iterations. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice.

Statistical model validation

In statistics, model validation is the task of evaluating whether a chosen statistical model is appropriate or not. Oftentimes in statistical inference, inferences from models that appear to fit their data may be flukes, resulting in a misunderstanding by researchers of the actual relevance of their model. To combat this, model validation is used to test whether a statistical model can hold up to permutations in the data.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Related lectures (35)

Model Assessment and Hyperparameter Tuning

Explores model assessment, hyperparameter tuning, and resampling strategies in machine learning.

Nonlinear ML Algorithms

Introduces nonlinear ML algorithms, covering nearest neighbor, k-NN, polynomial curve fitting, model complexity, overfitting, and regularization.

Model Selection: Generalization and Validation

Explores generalization, model selection, and validation in machine learning, emphasizing the importance of unbiased model evaluation.

Overfitting, Cross-validation, Regularization

Explores overfitting, cross-validation, and regularization in machine learning, emphasizing model complexity and the importance of regularization strength.

Specification Testing and Machine Learning