Lecture

Dimensionality Reduction

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

COM-308: Internet analytics

Internet analytics is the collection, modeling, and analysis of user data in large-scale online services, such as social networking, e-commerce, search, and advertisement. This class explores a number of the key functions of such online services that have ...

Description

This lecture covers the concepts of Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) for dimensionality reduction. It explains how to find low-dimensional representations of high-dimensional data, with applications in visualization, noise reduction, and efficiency. The lecture also delves into the spectral theorem, SVD existence, low-rank approximation, and best rank(r)-approximation. Additionally, it explores the interpretation of SVD, covariance vs correlation matrix in PCA, Multidimensional Scaling (MDS), non-linear embedding techniques like Isomap, and concludes with a summary of lessons learned.

Instructor

Matthias Grossglauser

Matthias Grossglauser is a Professor of Computer and Communication Sciences at EPFL in Lausanne, Switzerland, where he co-directs the Information and Network Dynamics lab. His current research interests center on machine learning, stochastic networks, and discrete choice models, and on their applications in artificial intelligence, reinforcement learning, network science, computational social sciences, and recommender systems. He is also a member of the Federal Communications Commission (ComCom), the indepe

Official source

Ontological neighbourhood

Mathematics

Algebra: Linear algebra

Information engineering

Data science: Dimensionality reduction

Related concepts (32)

Nonlinear dimensionality reduction

Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa) itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis.

Covariance matrix

In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a given random vector. Any covariance matrix is symmetric and positive semi-definite and its main diagonal contains variances (i.e., the covariance of each element with itself). Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions.

Curse of dimensionality

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic programming. Dimensionally cursed phenomena occur in domains such as numerical analysis, sampling, combinatorics, machine learning, data mining and databases.

Linear subspace

In mathematics, and more specifically in linear algebra, a linear subspace or vector subspace is a vector space that is a subset of some larger vector space. A linear subspace is usually simply called a subspace when the context serves to distinguish it from other types of subspaces. If V is a vector space over a field K and if W is a subset of V, then W is a linear subspace of V if under the operations of V, W is a vector space over K.

Canonical correlation

In statistics, canonical-correlation analysis (CCA), also called canonical variates analysis, is a way of inferring information from cross-covariance matrices. If we have two vectors X = (X1, ..., Xn) and Y = (Y1, ..., Ym) of random variables, and there are correlations among the variables, then canonical-correlation analysis will find linear combinations of X and Y which have maximum correlation with each other. T. R.

Related lectures (64)

Linear Algebra: Singular Value Decomposition

Delves into singular value decomposition and its applications in linear algebra.

Principal Components: Properties & Applications

Explores principal components, covariance, correlation, choice, and applications in data analysis.

Estimating the Term Structure: Principal Component Analysis

Covers Principal Component Analysis for yield curve shape estimation and dimension reduction in interest rate models.

Principal Component Analysis: Dimension Reduction

Covers Principal Component Analysis for dimension reduction in biological data, focusing on visualization and pattern identification.

Singular Value Decomposition: Image Compression and Applications