Lecture

Maximum Likelihood Estimation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-442: Statistical theory

This course gives a mostly rigourous treatment of some statistical methods outside the context of standard likelihood theory. ...

Description

This lecture covers the problem of point estimation, focusing on maximum likelihood estimators and their relationship with Kullback-Leibler divergence. It discusses the asymptotic properties of the MLE, including consistency and asymptotic normality. The lecture also explores examples such as the geometric and uniform distributions, illustrating the properties of MLEs. The asymptotic theory for MLEs is presented, emphasizing the regularity conditions required for consistency. The lecture concludes with a discussion on the asymptotic distribution of the MLE and the critical aspect of proving its consistency.

Instructor

Yoav Zemel

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Maximum likelihood estimation

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.

Statistical inference

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population. Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population.

Asymptotic distribution

In mathematics and statistics, an asymptotic distribution is a probability distribution that is in a sense the "limiting" distribution of a sequence of distributions. One of the main uses of the idea of an asymptotic distribution is in providing approximations to the cumulative distribution functions of statistical estimators. A sequence of distributions corresponds to a sequence of random variables Zi for i = 1, 2, ..., I .

Asymptotic theory (statistics)

In statistics, asymptotic theory, or large sample theory, is a framework for assessing properties of estimators and statistical tests. Within this framework, it is often assumed that the sample size n may grow indefinitely; the properties of estimators and tests are then evaluated under the limit of n → ∞. In practice, a limit evaluation is considered to be approximately valid for large finite sample sizes too. Most statistical problems begin with a dataset of size n.

Estimation theory

Estimation theory is a branch of statistics that deals with estimating the values of parameters based on measured empirical data that has a random component. The parameters describe an underlying physical setting in such a way that their value affects the distribution of the measured data. An estimator attempts to approximate the unknown parameters using the measurements.

Related lectures (64)

Statistical Theory: Maximum Likelihood Estimation

Explores the consistency and asymptotic properties of the Maximum Likelihood Estimator, including challenges in proving its consistency and constructing MLE-like estimators.

Statistical Theory: Cramér-Rao Bound & Hypothesis Testing

Explores the Cramér-Rao bound, hypothesis testing, and optimality in statistical theory.

Statistical Theory: Inference and Optimality

Explores constructing confidence regions, inverting hypothesis tests, and the pivotal method, emphasizing the importance of likelihood methods in statistical inference.

Continuous Random Variables

Covers continuous random variables, probability density functions, and distributions, with practical examples.

Optimality in Decision Theory: Unbiased Estimation