Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Supervised Learning with kNN: Regression Model

In course

EE-411: Fundamentals of inference and learning

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and prac ...

Description

This lecture presents a simple mathematical model for supervised learning with k-nearest neighbors (kNN) in the context of regression. The instructor explains the assumptions, the distribution knowledge, and the process to find the best function that minimizes the error. Various concepts such as Bayes, risk minimization, and population are covered.

Instructor

Florent Gérard Krzakala

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Normal distribution

In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution (and also its median and mode), while the parameter is its standard deviation. The variance of the distribution is . A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one or more independent variables (often called 'predictors', 'covariates', 'explanatory variables' or 'features'). The most common form of regression analysis is linear regression, in which one finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

Population growth

Population growth is the increase in the number of people in a population or dispersed group. Actual global human population growth amounts to around 83 million annually, or 1.1% per year. The global population has grown from 1 billion in 1800 to 7.9 billion in 2020. The UN projected population to keep growing, and estimates have put the total population at 8.6 billion by mid-2030, 9.8 billion by mid-2050 and 11.2 billion by 2100.

World population

In demographics, the world population is the total number of humans currently living. It was estimated by the United Nations to have exceeded eight billion in mid-November 2022. It took over 200,000 years of human prehistory and history for the human population to reach one billion and only 219 years more to reach 8 billion. The human population has experienced continuous growth following the Great Famine of 1315–1317 and the end of the Black Death in 1350, when it was nearly 370,000,000.

Related lectures (40)

Validation and k-Nearest Neighbors Method

Introduces supervised learning concepts and the k-Nearest Neighbors method for classification and regression tasks.

Machine Learning Basics: Supervised Learning

Introduces the basics of supervised machine learning, covering types, techniques, bias-variance tradeoff, and model evaluation.

Introduction to Machine Learning: Supervised Learning

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Deep Learning Modus Operandi

Explores the benefits of deeper networks in deep learning and the importance of over-parameterization and generalization.

Supervised Learning: Classification and Regression