Lecture

Optimality and Asymptotics

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-341: Linear models

Regression modelling is a fundamental tool of statistics, because it describes how the law of a random variable of interest may depend on other variables. This course aims to familiarize students with linear models and some of their extensions, which lie a ...

Description

This lecture discusses the optimality of the Least Squares Estimator (LSE) in the Gaussian Linear Model, showing that it is the best unbiased estimator. It explores the Gauss-Markov Theorem, which states that the LSE is the best linear unbiased estimator. The concept of optimality is further examined under weaker assumptions, focusing on uncorrelatedness. The lecture also delves into the large sample distribution of the estimator, emphasizing its behavior as the sample size grows. Various theorems and proofs are presented to support the discussion, shedding light on the distribution properties of the estimator under different conditions.

Instructors (2)

Victor Panaretos

Curriculum vitae Background & EducationPublications & PreprintsHonours & AwardsEditorial ActivitiesInvited Talks & Seminars Teaching & PhD PhD Students Ulysse Naepels, Nikitas Georgakis, Ho Yun Past EPFL PhD Students Andrea Kraus, Shahin Tavakoli, Mikael Kuusela, Yoav Zemel, Marie-Hélène Descary, Valentina Masarotto, Tomas Rubin, Tomas Masák, Laya Ghodrati, Kartik Waghmare, Leonardo Santoro Past EPFL PhD Students as codirector Matthieu Simeoni, Ophelia Miralles, Sonia Alouini, Timmy Rong Tian Tse, Krali M

Tomas Masák

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Data analysis: Regression analysis

Related concepts (32)

Generalized linear model

In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value. Generalized linear models were formulated by John Nelder and Robert Wedderburn as a way of unifying various other statistical models, including linear regression, logistic regression and Poisson regression.

Sampling distribution

In statistics, a sampling distribution or finite-sample distribution is the probability distribution of a given random-sample-based statistic. If an arbitrarily large number of samples, each involving multiple observations (data points), were separately used in order to compute one value of a statistic (such as, for example, the sample mean or sample variance) for each sample, then the sampling distribution is the probability distribution of the values that the statistic takes on.

Estimator

In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the sample mean is a commonly used estimator of the population mean. There are point and interval estimators. The point estimators yield single-valued results. This is in contrast to an interval estimator, where the result would be a range of plausible values.

Poisson distribution

In probability theory and statistics, the Poisson distribution is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time or space if these events occur with a known constant mean rate and independently of the time since the last event. It is named after French mathematician Siméon Denis Poisson ('pwɑːsɒn; pwasɔ̃). The Poisson distribution can also be used for the number of events in other specified interval types such as distance, area, or volume.

Cauchy distribution

The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) function, or Breit–Wigner distribution. The Cauchy distribution is the distribution of the x-intercept of a ray issuing from with a uniformly distributed angle. It is also the distribution of the ratio of two independent normally distributed random variables with mean zero.

Related lectures (64)

Linear Regression: Statistical Inference and Regularization

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Gauss-Markov Theorem: Optimal Estimation

Explores the Gauss-Markov Theorem and the optimality of Least Squares Estimators in the Gaussian Linear model.

Nonlinear Machine Learning: k-Nearest Neighbors and Feature Expansion

Covers the transition from linear to nonlinear models, focusing on k-NN and feature expansion techniques.

Weighted Least Squares Estimation: IRLS Algorithm

Explores the IRLS algorithm for weighted least squares estimation in GLM.

Regularization in Machine Learning