Lecture

Supervised Learning with kNN: Regression Model

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

EE-411: Fundamentals of inference and learning

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and prac ...

Description

This lecture presents a simple mathematical model for supervised learning with k-nearest neighbors (kNN) in the context of regression. The instructor explains the assumptions, the distribution knowledge, and the process to find the best function that minimizes the error. Various concepts such as Bayes, risk minimization, and population are covered.

Instructor

Florent Gérard Krzakala

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Cauchy distribution

The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) function, or Breit–Wigner distribution. The Cauchy distribution is the distribution of the x-intercept of a ray issuing from with a uniformly distributed angle. It is also the distribution of the ratio of two independent normally distributed random variables with mean zero.

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Low back pain

Low back pain (LBP) or lumbago is a common disorder involving the muscles, nerves, and bones of the back, in between the lower edge of the ribs and the lower fold of the buttocks. Pain can vary from a dull constant ache to a sudden sharp feeling. Low back pain may be classified by duration as acute (pain lasting less than 6 weeks), sub-chronic (6 to 12 weeks), or chronic (more than 12 weeks). The condition may be further classified by the underlying cause as either mechanical, non-mechanical, or referred pain.

Stable distribution

In probability theory, a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters. A random variable is said to be stable if its distribution is stable. The stable distribution family is also sometimes referred to as the Lévy alpha-stable distribution, after Paul Lévy, the first mathematician to have studied it. Of the four parameters defining the family, most attention has been focused on the stability parameter, (see panel).

F-distribution

In probability theory and statistics, the F-distribution or F-ratio, also known as Snedecor's F distribution or the Fisher–Snedecor distribution (after Ronald Fisher and George W. Snedecor), is a continuous probability distribution that arises frequently as the null distribution of a test statistic, most notably in the analysis of variance (ANOVA) and other F-tests. The F-distribution with d1 and d2 degrees of freedom is the distribution of where and are independent random variables with chi-square distributions with respective degrees of freedom and .

Related lectures (64)

Regression Trees and Ensemble Methods in Machine Learning

Discusses regression trees, ensemble methods, and their applications in predicting used car prices and stock returns.

Validation and k-Nearest Neighbors Method

Introduces supervised learning concepts and the k-Nearest Neighbors method for classification and regression tasks.

Linear Models: Part 1

Covers linear models, including regression, derivatives, gradients, hyperplanes, and classification transition, with a focus on minimizing risk and evaluation metrics.

Machine Learning Basics: Supervised Learning

Introduces the basics of supervised machine learning, covering types, techniques, bias-variance tradeoff, and model evaluation.

Machine Learning: Supervised and Unsupervised Learning Techniques