Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Principal Component Analysis: Theory and Applications

Description

This lecture covers the theory and applications of Principal Component Analysis (PCA), focusing on dimension reduction and eigenvectors. It explains how PCA can be used to find the optimal linear dimension reduction for random variables, as well as the properties of Gaussian vectors.

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Mathematics

Algebra: Linear algebra

Related concepts (32)

Eigendecomposition of a matrix

In linear algebra, eigendecomposition is the factorization of a matrix into a canonical form, whereby the matrix is represented in terms of its eigenvalues and eigenvectors. Only diagonalizable matrices can be factorized in this way. When the matrix being factorized is a normal or real symmetric matrix, the decomposition is called "spectral decomposition", derived from the spectral theorem. Eigenvalue, eigenvector and eigenspace A (nonzero) vector v of dimension N is an eigenvector of a square N × N matrix A if it satisfies a linear equation of the form for some scalar λ.

Covariance matrix

In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a given random vector. Any covariance matrix is symmetric and positive semi-definite and its main diagonal contains variances (i.e., the covariance of each element with itself). Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions.

Multivariate normal distribution

In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One definition is that a random vector is said to be k-variate normally distributed if every linear combination of its k components has a univariate normal distribution. Its importance derives mainly from the multivariate central limit theorem.

Eigenvalues and eigenvectors

In linear algebra, an eigenvector (ˈaɪgənˌvɛktər) or characteristic vector of a linear transformation is a nonzero vector that changes at most by a constant factor when that linear transformation is applied to it. The corresponding eigenvalue, often represented by , is the multiplying factor. Geometrically, a transformation matrix rotates, stretches, or shears the vectors it acts upon. The eigenvectors for a linear transformation matrix are the set of vectors that are only stretched, with no rotation or shear.

Principal component analysis

Principal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and enabling the visualization of multidimensional data. Formally, PCA is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.

Related lectures (46)

Principal Components: Properties & Applications

Explores principal components, covariance, correlation, choice, and applications in data analysis.

Multivariate Statistics: Introduction and Methods

Introduces multivariate statistics, focusing on uncovering associations between components in data in vector form.

Multivariate Statistics: Wishart and Hotelling T²

Explores the Wishart distribution, properties of Wishart matrices, and the Hotelling T² distribution, including the two-sample Hotelling T² statistic.

Market Response Functions

Explores market response functions, flash crashes, correlation estimation, and noise filtering in finance.

Non-Negative Definite Matrices and Covariance Matrices