Lecture

Covariance Cleaning and Estimators

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

FIN-525: Financial big data

The course introduces modern methods to acquire, clean, and analyze large quantities of financial data efficiently. The second part expands on how to apply these techniques and robust statistics to financial analysis, in particular to intraday data and inv ...

Description

This lecture covers the rationale behind covariance matrix cleaning, emphasizing the drawbacks of using the sample covariance matrix for portfolio optimization. It delves into rotationally invariant estimators, hierarchical clustering, and bootstraps. The instructor discusses the empirical spectrum of correlation matrices, eigenvalue clipping, and a formal approach to correlation/covariance cleaning. The lecture explores the concept of a good covariance estimator, focusing on minimizing the distance with the true matrix and building an optimal estimator for portfolio optimization. Additionally, it introduces the linear shrinkage method and the concept of optimal rotationally invariant estimators.

Instructor

Damien Challet

Teaching & PhD Courses Financial big data FIN-525 The course introduces modern methods to acquire, clean, and analyze large quantities of financial data efficiently. The second part expands on how to apply these techniques and robust statistics to financial analysis, in particular to intraday data and investment strategies.

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Bias of an estimator

In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called unbiased. In statistics, "bias" is an property of an estimator. Bias is a distinct concept from consistency: consistent estimators converge in probability to the true value of the parameter, but may be biased or unbiased; see bias versus consistency for more.

Invariant estimator

In statistics, the concept of being an invariant estimator is a criterion that can be used to compare the properties of different estimators for the same quantity. It is a way of formalising the idea that an estimator should have certain intuitively appealing qualities. Strictly speaking, "invariant" would mean that the estimates themselves are unchanged when both the measurements and the parameters are transformed in a compatible way, but the meaning has been extended to allow the estimates to change in appropriate ways with such transformations.

Estimator

In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the sample mean is a commonly used estimator of the population mean. There are point and interval estimators. The point estimators yield single-valued results. This is in contrast to an interval estimator, where the result would be a range of plausible values.

Covariance

In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the lesser values (that is, the variables tend to show similar behavior), the covariance is positive. In the opposite case, when the greater values of one variable mainly correspond to the lesser values of the other, (that is, the variables tend to show opposite behavior), the covariance is negative.

Sample mean and covariance

The sample mean (sample average) or empirical mean (empirical average), and the sample covariance or empirical covariance are statistics computed from a sample of data on one or more random variables. The sample mean is the average value (or mean value) of a sample of numbers taken from a larger population of numbers, where "population" indicates not number of people but the entirety of relevant data, whether collected or not. A sample of 40 companies' sales from the Fortune 500 might be used for convenience instead of looking at the population, all 500 companies' sales.

Related lectures (64)

Principal Components: Properties & Applications

Explores principal components, covariance, correlation, choice, and applications in data analysis.

Risk Measures and Multivariate Distributions

Covers risk measures, VaR, ES computation, backtesting, and multivariate distributions for portfolio risk assessment.

Dependence in Random Vectors

Explores dependence in random vectors, covering joint density, conditional independence, covariance, and moment generating functions.

Red bus/Blue bus paradox

Explores the Red bus/Blue bus paradox, nested logit models, and multivariate extreme value models in transportation.

Shrinkage Estimation of Large Covariance Matrices