Lecture

Statistical Theory: Maximum Likelihood Estimation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-442: Statistical theory

This course gives a mostly rigourous treatment of some statistical methods outside the context of standard likelihood theory. ...

Description

This lecture delves into the consistency of the Maximum Likelihood Estimator (MLE) and its asymptotic properties. It explores the relationship between the MLE and the Kullback-Leibler Divergence, highlighting the challenges in proving the MLE's consistency. The lecture discusses deterministic examples to illustrate the complexities of the MLE's behavior. It also covers the construction of asymptotically MLE-like estimators and the Newton-Raphson algorithm. The lecture concludes with a discussion on misspecified models and likelihood, emphasizing the importance of model approximation and the behavior of estimators in such scenarios.

Instructor

Yoav Zemel

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (31)

Statistical inference

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population. Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population.

Statistical model

A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process. When referring specifically to probabilities, the corresponding term is probabilistic model. A statistical model is usually specified as a mathematical relationship between one or more random variables and other non-random variables.

Estimator

In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the sample mean is a commonly used estimator of the population mean. There are point and interval estimators. The point estimators yield single-valued results. This is in contrast to an interval estimator, where the result would be a range of plausible values.

Statistical hypothesis testing

A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. While hypothesis testing was popularized early in the 20th century, early forms were used in the 1700s. The first use is credited to John Arbuthnot (1710), followed by Pierre-Simon Laplace (1770s), in analyzing the human sex ratio at birth; see .

Invariant estimator

In statistics, the concept of being an invariant estimator is a criterion that can be used to compare the properties of different estimators for the same quantity. It is a way of formalising the idea that an estimator should have certain intuitively appealing qualities. Strictly speaking, "invariant" would mean that the estimates themselves are unchanged when both the measurements and the parameters are transformed in a compatible way, but the meaning has been extended to allow the estimates to change in appropriate ways with such transformations.

Related lectures (64)

Confidence Intervals: Definition and Estimation

Explains confidence intervals, parameter estimation methods, and the central limit theorem in statistical inference.

Optimality in Decision Theory: Unbiased Estimation

Explores optimality in decision theory and unbiased estimation, emphasizing sufficiency, completeness, and lower bounds for risk.

Statistical Theory: Inference and Optimality

Explores constructing confidence regions, inverting hypothesis tests, and the pivotal method, emphasizing the importance of likelihood methods in statistical inference.

Estimators and Confidence Intervals

Explores bias, variance, unbiased estimators, and confidence intervals in statistical estimation.

Law of Large Numbers: Strong Convergence