Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Dimensionality Reduction

In course

COM-308: Internet analytics

Internet analytics is the collection, modeling, and analysis of user data in large-scale online services, such as social networking, e-commerce, search, and advertisement. This class explores a number of the key functions of such online services that have ...

Description

This lecture covers the concepts of Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) for dimensionality reduction. It explains how to find low-dimensional representations of high-dimensional data, with applications in visualization, noise reduction, and efficiency. The lecture also delves into the spectral theorem, SVD existence, low-rank approximation, and best rank(r)-approximation. Additionally, it explores the interpretation of SVD, covariance vs correlation matrix in PCA, Multidimensional Scaling (MDS), non-linear embedding techniques like Isomap, and concludes with a summary of lessons learned.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Matthias Grossglauser

Official source

Ontological neighbourhood

Mathematics

Algebra: Linear algebra

Information engineering

Data science: Dimensionality reduction

Related concepts (32)

Singular value decomposition

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any matrix. It is related to the polar decomposition. Specifically, the singular value decomposition of an complex matrix M is a factorization of the form where U is an complex unitary matrix, is an rectangular diagonal matrix with non-negative real numbers on the diagonal, V is an complex unitary matrix, and is the conjugate transpose of V.

Dimensionality reduction

Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension. Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

Nonlinear dimensionality reduction

Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa) itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis.

Polar decomposition

In mathematics, the polar decomposition of a square real or complex matrix is a factorization of the form , where is a unitary matrix and is a positive semi-definite Hermitian matrix ( is an orthogonal matrix and is a positive semi-definite symmetric matrix in the real case), both square and of the same size. Intuitively, if a real matrix is interpreted as a linear transformation of -dimensional space , the polar decomposition separates it into a rotation or reflection of , and a scaling of the space along a set of orthogonal axes.

Matrix decomposition

In the mathematical discipline of linear algebra, a matrix decomposition or matrix factorization is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems. In numerical analysis, different decompositions are used to implement efficient matrix algorithms. For instance, when solving a system of linear equations , the matrix A can be decomposed via the LU decomposition.

Related lectures (50)

Principal Components: Properties & Applications

Explores principal components, covariance, correlation, choice, and applications in data analysis.

Principal Component Analysis: Dimension Reduction

Covers Principal Component Analysis for dimension reduction in biological data, focusing on visualization and pattern identification.

Principal Component Analysis: Dimensionality Reduction

Covers Principal Component Analysis for dimensionality reduction, exploring its applications, limitations, and importance of choosing the right components.

Canonical Correlation Analysis: Overview

Covers Canonical Correlation Analysis, a method to find relationships between two sets of variables.

Unsupervised learning: Young-Eckart-Mirsky theorem and intro to PCA