Lecture

Clustering: Principles and Methods

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture introduces the principles of clustering in machine learning, where algorithms group datapoints without knowing true labels or the number of groups. Clustering is used for feature extraction and data compression, with cluster prototypes representing typical datapoints or centroids. The lecture covers similarity measures, PCA projection, and the use of K-means and soft-K-means algorithms. It also discusses the impact of initialization on clustering performance and evaluates different clustering methods using internal and external measures. The instructor demonstrates how to determine the optimal number of clusters using RSS and explains the sensitivity of K-means to initialization.

Official source

Related concepts (32)

K-means clustering

k-means clustering is a method of vector quantization, originally from signal processing, that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid), serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells. k-means clustering minimizes within-cluster variances (squared Euclidean distances), but not regular Euclidean distances, which would be the more difficult Weber problem: the mean optimizes squared errors, whereas only the geometric median minimizes Euclidean distances.

Cluster analysis

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of exploratory data analysis, and a common technique for statistical data analysis, used in many fields, including pattern recognition, , information retrieval, bioinformatics, data compression, computer graphics and machine learning.

Hierarchical clustering

In data mining and statistics, hierarchical clustering (also called hierarchical cluster analysis or HCA) is a method of cluster analysis that seeks to build a hierarchy of clusters. Strategies for hierarchical clustering generally fall into two categories: Agglomerative: This is a "bottom-up" approach: Each observation starts in its own cluster, and pairs of clusters are merged as one moves up the hierarchy. Divisive: This is a "top-down" approach: All observations start in one cluster, and splits are performed recursively as one moves down the hierarchy.

Data compression

In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by removing unnecessary or less important information.

Lossy compression

In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size for storing, handling, and transmitting content. The different versions of the photo of the cat on this page show how higher degrees of approximation create coarser images as more details are removed. This is opposed to lossless data compression (reversible data compression) which does not degrade the data.

Related lectures (64)

Clustering & Density Estimation

Covers clustering, PCA, LDA, K-means, GMM, KDE, and Mean Shift algorithms for density estimation and clustering.

Principal Component Analysis: Dimensionality Reduction

Covers Principal Component Analysis for dimensionality reduction, exploring its applications, limitations, and importance of choosing the right components.

Clustering & Density Estimation

Covers dimensionality reduction, PCA, clustering techniques, and density estimation methods.

Unsupervised Learning: Clustering & Dimensionality Reduction

Introduces unsupervised learning through clustering with K-means and dimensionality reduction using PCA, along with practical examples.

K-Means Clustering: Image Compression