Lecture

Clustering & Density Estimation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers the concepts of dimensionality reduction, focusing on Principal Component Analysis (PCA) and Fisher Linear Discriminant Analysis (LDA). It also delves into nonlinear dimensionality reduction methods like Kernel PCA and t-SNE. The lecture then transitions into the topic of clustering, explaining K-means clustering and its properties. It further explores Gaussian Mixture Models (GMM) and non-parametric density estimation techniques like histograms and Kernel Density Estimation (KDE). The Mean Shift algorithm for clustering is also discussed, emphasizing its iterative approach to finding density maxima. Real-world examples and visualizations are used to illustrate these concepts.

Official source

Ontological neighbourhood

Statistics

Statistical inference: Non-parametric statistics

Information engineering

Machine learning: Unsupervised learning

Related concepts (32)

Dimensionality reduction

Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension. Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

Kernel density estimation

In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on kernels as weights. KDE answers a fundamental data smoothing problem where inferences about the population are made, based on a finite data sample. In some fields such as signal processing and econometrics it is also termed the Parzen–Rosenblatt window method, after Emanuel Parzen and Murray Rosenblatt, who are usually credited with independently creating it in its current form.

Density estimation

In statistics, probability density estimation or simply density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density function is thought of as the density according to which a large population is distributed; the data are usually thought of as a random sample from that population. A variety of approaches to density estimation are used, including Parzen windows and a range of data clustering techniques, including vector quantization.

Multivariate kernel density estimation

Kernel density estimation is a nonparametric technique for density estimation i.e., estimation of probability density functions, which is one of the fundamental questions in statistics. It can be viewed as a generalisation of histogram density estimation with improved statistical properties. Apart from histograms, other types of density estimators include parametric, spline, wavelet and Fourier series. Kernel density estimators were first introduced in the scientific literature for univariate data in the 1950s and 1960s and subsequently have been widely adopted.

Nonlinear dimensionality reduction

Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa) itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis.

Related lectures (64)

Clustering & Density Estimation

Covers clustering, PCA, LDA, K-means, GMM, KDE, and Mean Shift algorithms for density estimation and clustering.

Clustering & Density Estimation

Covers dimensionality reduction, PCA, clustering techniques, and density estimation methods.

Unsupervised Behavior Clustering

Explores unsupervised behavior clustering and dimensionality reduction techniques, covering algorithms like K-Means, DBSCAN, and Gaussian Mixture Model.

Dimensionality Reduction: PCA and LDA

Covers dimensionality reduction techniques like PCA and LDA, clustering methods, density estimation, and data representation.

Unsupervised Learning: Clustering

Explores unsupervised learning through clustering techniques, algorithms, applications, and challenges in various fields.

Related concepts (32)

Dimensionality reduction

Kernel density estimation

Density estimation

Multivariate kernel density estimation

Nonlinear dimensionality reduction