Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Classification with GMM and kNN

Description

This lecture covers the classification process using Gaussian Mixture Models (GMM) and k-Nearest Neighbors (kNN). The instructor explains how to model classes with GMM and determine boundaries using ML discriminant rules. The lecture also delves into the Curse of Dimensionality, computational costs, and the comparison results of different models. Practical exercises involve finding boundaries with GMM, understanding the impact of covariance matrices, and analyzing classification errors. The importance of cross-validation and the choice of training/testing ratios are highlighted, along with the application of kNN for classification. The lecture concludes with a demonstration of drawing boundaries with different k values in kNN.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

K-nearest neighbors algorithm

In statistics, the k-nearest neighbors algorithm (k-NN) is a non-parametric supervised learning method first developed by Evelyn Fix and Joseph Hodges in 1951, and later expanded by Thomas Cover. It is used for classification and regression. In both cases, the input consists of the k closest training examples in a data set. The output depends on whether k-NN is used for classification or regression: In k-NN classification, the output is a class membership.

Statistical classification

In statistics, classification is the problem of identifying which of a set of categories (sub-populations) an observation (or observations) belongs to. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagnosis to a given patient based on observed characteristics of the patient (sex, blood pressure, presence or absence of certain symptoms, etc.). Often, the individual observations are analyzed into a set of quantifiable properties, known variously as explanatory variables or features.

Training, validation, and test data sets

In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data. Such algorithms function by making data-driven predictions or decisions, through building a mathematical model from input data. These input data used to build the model are usually divided into multiple data sets. In particular, three data sets are commonly used in different stages of the creation of the model: training, validation, and test sets.

Nearest neighbor search

Nearest neighbor search (NNS), as a form of proximity search, is the optimization problem of finding the point in a given set that is closest (or most similar) to a given point. Closeness is typically expressed in terms of a dissimilarity function: the less similar the objects, the larger the function values. Formally, the nearest-neighbor (NN) search problem is defined as follows: given a set S of points in a space M and a query point q ∈ M, find the closest point in S to q. Donald Knuth in vol.

Curse of dimensionality

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic programming. Dimensionally cursed phenomena occur in domains such as numerical analysis, sampling, combinatorics, machine learning, data mining and databases.

Related lectures (36)

Nearest Neighbor Rules: Part 2

Explores the Nearest Neighbor Rules, k-NN algorithm challenges, Bayes classifier, and k-means algorithm for clustering.

Model Evaluation: K-Nearest Neighbor

Explores model evaluation with K-Nearest Neighbor, covering optimal k selection, similarity metrics, and performance metrics for classification models.

Quantifying Performance: Misclassification and F-Measure

Covers quantifying performance through true positives, false negatives, and false positives in machine learning.

Metrics for Classification

Covers sampling, cross-validation, quantifying performance, optimal model determination, overfitting detection, and classification sensitivity.

Nonlinear ML Algorithms