Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

PCA: Derivation and Optimization

In course

MICRO-455: Machine learning I

Description

This lecture covers the derivation of the projection in PCA, the reconstruction process through error minimization, and the optimization to choose the optimal eigenvectors. It explains how to maximize variance, the properties of PCA projections, and the PCA algorithm step by step.

Instructor

Aude Billard

Official source

Related concepts (32)

Covariance matrix

In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a given random vector. Any covariance matrix is symmetric and positive semi-definite and its main diagonal contains variances (i.e., the covariance of each element with itself). Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions.

Map projection

In cartography, a map projection is any of a broad set of transformations employed to represent the curved two-dimensional surface of a globe on a plane. In a map projection, coordinates, often expressed as latitude and longitude, of locations from the surface of the globe are transformed to coordinates on a plane. Projection is a necessary step in creating a two-dimensional map and is one of the essential elements of cartography. All projections of a sphere on a plane necessarily distort the surface in some way and to some extent.

Eigenvalues and eigenvectors

In linear algebra, an eigenvector (ˈaɪgənˌvɛktər) or characteristic vector of a linear transformation is a nonzero vector that changes at most by a constant factor when that linear transformation is applied to it. The corresponding eigenvalue, often represented by , is the multiplying factor. Geometrically, a transformation matrix rotates, stretches, or shears the vectors it acts upon. The eigenvectors for a linear transformation matrix are the set of vectors that are only stretched, with no rotation or shear.

Generalized eigenvector

In linear algebra, a generalized eigenvector of an matrix is a vector which satisfies certain criteria which are more relaxed than those for an (ordinary) eigenvector. Let be an -dimensional vector space and let be the matrix representation of a linear map from to with respect to some ordered basis. There may not always exist a full set of linearly independent eigenvectors of that form a complete basis for . That is, the matrix may not be diagonalizable.

Mercator projection

The Mercator projection (mərˈkeɪtər) is a cylindrical map projection presented by Flemish geographer and cartographer Gerardus Mercator in 1569. It became the standard map projection for navigation because it is unique in representing north as up and south as down everywhere while preserving local directions and shapes. The map is thereby conformal. As a side effect, the Mercator projection inflates the size of objects away from the equator. This inflation is very small near the equator but accelerates with increasing latitude to become infinite at the poles.

Related lectures (53)

Principal Components: Properties & Applications

Explores principal components, covariance, correlation, choice, and applications in data analysis.

Oja's Rule

Covers Oja's rule in Neurorobotics, focusing on learning eigenvectors and eigenvalues for capturing maximal variance.

Multivariate Statistics: Wishart and Hotelling T²

Explores the Wishart distribution, properties of Wishart matrices, and the Hotelling T² distribution, including the two-sample Hotelling T² statistic.

Non-Negative Definite Matrices and Covariance Matrices

Covers non-negative definite matrices, covariance matrices, and Principal Component Analysis for optimal dimension reduction.

Market Response Functions