Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Principal Component Analysis: Geometric Interpretation and Dimension Reduction

In course

EE-311: Fundamentals of machine learning

Ce cours présente une vue générale des techniques d'apprentissage automatique, passant en revue les algorithmes, le formalisme théorique et les protocoles expérimentaux. ...

Description

This lecture covers Principal Component Analysis (PCA), a method to reduce the dimensionality of data by maximizing variance along new dimensions. It explains how PCA represents data in a new orthonormal basis, the importance of centering and standardization, and the calculation of principal components. The lecture also discusses the geometric interpretation of PCA, the choice of the number of principal components, and the latent representation of data. Additionally, it touches on model selection and validation in the context of dimension reduction.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructors (2)

François Fleuret

Michael Stefan Daniel Liebling

Official source

Related concepts (32)

Principal component analysis

Principal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and enabling the visualization of multidimensional data. Formally, PCA is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.

Dimensionality reduction

Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension. Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

Nonlinear dimensionality reduction

Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa) itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis.

Feature learning

In machine learning, feature learning or representation learning is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data. This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task. Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process.

Kernel principal component analysis

In the field of multivariate statistics, kernel principal component analysis (kernel PCA) is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are performed in a reproducing kernel Hilbert space. Recall that conventional PCA operates on zero-centered data; that is, where is one of the multivariate observations.

Related lectures (54)

Principal Component Analysis: Dimensionality Reduction

Covers Principal Component Analysis for dimensionality reduction, exploring its applications, limitations, and importance of choosing the right components.

Unsupervised Learning: Dimensionality Reduction

Explores unsupervised learning techniques for reducing dimensions in data, emphasizing PCA, LDA, and Kernel PCA.

Principal Component Analysis: Dimensionality Reduction

Explores Principal Component Analysis for dimensionality reduction and unsupervised feature selection.

Introduction to Machine Learning: Linear Models

Introduces linear models for supervised learning, covering overfitting, regularization, and kernels, with applications in machine learning tasks.

Clustering: Theory and Practice