Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Introduction to Supervised Learning and Decision Theory

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture introduces the concepts of supervised learning and decision theory, focusing on key elements such as loss function, risk, target function, and excess risk. It covers the formalization of supervised learning, decision theoretic framework, generalization, expected behavior, and the goal of minimizing risk.

Instructor

Yoav Zemel

Official source

Related concepts (31)

Decision theory

Decision theory (or the theory of choice; not to be confused with choice theory) is a branch of applied probability theory and analytic philosophy concerned with the theory of making decisions based on assigning probabilities to various factors and assigning numerical consequences to the outcome. There are three branches of decision theory: Normative decision theory: Concerned with the identification of optimal decisions, where optimality is often determined by considering an ideal decision-maker who is able to calculate with perfect accuracy and is in some sense fully rational.

Decision-making

In psychology, decision-making (also spelled decision making and decisionmaking) is regarded as the cognitive process resulting in the selection of a belief or a course of action among several possible alternative options. It could be either rational or irrational. The decision-making process is a reasoning process based on assumptions of values, preferences and beliefs of the decision-maker. Every decision-making process produces a final choice, which may or may not prompt action.

Loss function

In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost" associated with the event. An optimization problem seeks to minimize a loss function. An objective function is either a loss function or its opposite (in specific domains, variously called a reward function, a profit function, a utility function, a fitness function, etc.

Info-gap decision theory

Info-gap decision theory seeks to optimize robustness to failure under severe uncertainty, in particular applying sensitivity analysis of the stability radius type to perturbations in the value of a given estimate of the parameter of interest. It has some connections with Wald's maximin model; some authors distinguish them, others consider them instances of the same principle. It has been developed by Yakov Ben-Haim, and has found many applications and described as a theory for decision-making under "severe uncertainty".

Regret (decision theory)

In decision theory, on making decisions under uncertainty—should information about the best course of action arrive after taking a fixed decision—the human emotional response of regret is often experienced, and can be measured as the value of difference between a made decision and the optimal decision. The theory of regret aversion or anticipated regret proposes that when facing a decision, individuals might anticipate regret and thus incorporate in their choice their desire to eliminate or reduce this possibility.

Related lectures (51)

Decision Theory: Risk and Inference

Explores decision theory, risk functions, and inference in statistical analysis.

Decision Theory: Risk and Hypothesis Testing

Covers decision theory, risk functions, and hypothesis testing in statistical inference.

Discriminant Analysis: Bayes Rule

Covers the Bayes discriminant rule for allocating individuals to populations based on measurements and prior probabilities.

Supervised Learning: Linear Regression

Covers supervised learning with a focus on linear regression, including topics like digit classification, spam detection, and wind speed prediction.

Examples of decision models for Supervised learning