Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Principal Component Analysis: Covariance Matrix and Eigenvalues

In course

MGT-448: Statistical inference and machine learning

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data ...

Description

This lecture covers the concept of Principal Component Analysis (PCA) applied to data with a focus on maximizing variance. Topics include the formulation of the augmented Lagrangian, empirical covariance matrix calculation, and finding the second PC component through eigenvalues. The lecture also discusses the eigen vector corresponding to the largest eigenvalue and its significance.

Instructor

Negar Kiyavash

Official source

Related concepts (9)

Covariance matrix

In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a given random vector. Any covariance matrix is symmetric and positive semi-definite and its main diagonal contains variances (i.e., the covariance of each element with itself). Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions.

Sample mean and covariance

The sample mean (sample average) or empirical mean (empirical average), and the sample covariance or empirical covariance are statistics computed from a sample of data on one or more random variables. The sample mean is the average value (or mean value) of a sample of numbers taken from a larger population of numbers, where "population" indicates not number of people but the entirety of relevant data, whether collected or not. A sample of 40 companies' sales from the Fortune 500 might be used for convenience instead of looking at the population, all 500 companies' sales.

Cross-covariance matrix

In probability theory and statistics, a cross-covariance matrix is a matrix whose element in the i, j position is the covariance between the i-th element of a random vector and j-th element of another random vector. A random vector is a random variable with multiple dimensions. Each element of the vector is a scalar random variable. Each element has either a finite number of observed empirical values or a finite or infinite number of potential values. The potential values are specified by a theoretical joint probability distribution.

Covariance

In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the lesser values (that is, the variables tend to show similar behavior), the covariance is positive. In the opposite case, when the greater values of one variable mainly correspond to the lesser values of the other, (that is, the variables tend to show opposite behavior), the covariance is negative.

Estimation of covariance matrices

In statistics, sometimes the covariance matrix of a multivariate random variable is not known but has to be estimated. Estimation of covariance matrices then deals with the question of how to approximate the actual covariance matrix on the basis of a sample from the multivariate distribution. Simple cases, where observations are complete, can be dealt with by using the sample covariance matrix.

Related lectures (31)

Principal Components: Properties & Applications

Explores principal components, covariance, correlation, choice, and applications in data analysis.

Principal Component Analysis: Understanding Data Structure

Explores Principal Component Analysis, dimensionality reduction, data quality assessment, and error rate control.

Principal Component Analysis: Dimension Reduction

Covers Principal Component Analysis for dimension reduction in biological data, focusing on visualization and pattern identification.

Estimating the Term Structure: Principal Component Analysis

Covers Principal Component Analysis for yield curve shape estimation and dimension reduction in interest rate models.

Linear Dimensionality Reduction