Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

KNN Classifier: Nearest Neighbor Approach

In course

MICRO-455: Machine learning I

Description

This lecture covers the K-Nearest Neighbors (KNN) classifier, a simple machine learning algorithm that assigns a label to a new point based on the labels of its closest known points. It explains the probabilistic approach of KNN to smooth away noise in labels, the classification process with KNN, the impact of different distance metrics on decision surfaces, and the advantages of KNN such as simplicity and not requiring assumptions about data distribution.

Instructor

Aude Billard

Official source

Related concepts (16)

K-nearest neighbors algorithm

In statistics, the k-nearest neighbors algorithm (k-NN) is a non-parametric supervised learning method first developed by Evelyn Fix and Joseph Hodges in 1951, and later expanded by Thomas Cover. It is used for classification and regression. In both cases, the input consists of the k closest training examples in a data set. The output depends on whether k-NN is used for classification or regression: In k-NN classification, the output is a class membership.

Nearest neighbor search

Nearest neighbor search (NNS), as a form of proximity search, is the optimization problem of finding the point in a given set that is closest (or most similar) to a given point. Closeness is typically expressed in terms of a dissimilarity function: the less similar the objects, the larger the function values. Formally, the nearest-neighbor (NN) search problem is defined as follows: given a set S of points in a space M and a query point q ∈ M, find the closest point in S to q. Donald Knuth in vol.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Statistical classification

In statistics, classification is the problem of identifying which of a set of categories (sub-populations) an observation (or observations) belongs to. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagnosis to a given patient based on observed characteristics of the patient (sex, blood pressure, presence or absence of certain symptoms, etc.). Often, the individual observations are analyzed into a set of quantifiable properties, known variously as explanatory variables or features.

Multiclass classification

In machine learning and statistical classification, multiclass classification or multinomial classification is the problem of classifying instances into one of three or more classes (classifying instances into one of two classes is called binary classification). While many classification algorithms (notably multinomial logistic regression) naturally permit the use of more than two classes, some are by nature binary algorithms; these can, however, be turned into multinomial classifiers by a variety of strategies.

Related lectures (31)

Nearest Neighbor Rules: Part 2

Explores the Nearest Neighbor Rules, k-NN algorithm challenges, Bayes classifier, and k-means algorithm for clustering.

Supervised Learning: k-NN and Decision Trees

Introduces supervised learning with k-NN and decision trees, covering techniques, examples, and ensemble methods.

Textual Data Analysis: Classification & Dimensionality Reduction

Explores textual data classification, focusing on methods like Naive Bayes and dimensionality reduction techniques like Principal Component Analysis.

K-Nearest Neighbors & Feature Expansion

Introduces k-Nearest Neighbors for classification and feature expansion to handle nonlinear data through transformed inputs.

Introduction to Machine Learning