Lecture

Maximum Likelihood Estimation: Theory and Examples

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture introduces the concept of maximum likelihood estimation, focusing on the proof of the Rao-Blackwell Theorem and its application in deriving unbiased estimators. The instructor explains the importance of sufficient statistics and demonstrates how to calculate maximum likelihood estimators through examples such as Bernoulli, Exponential, and Gaussian trials. The lecture concludes with an example of Poisson observations, emphasizing the process of determining the maximum likelihood estimator. Throughout the lecture, the instructor emphasizes the significance of selecting estimators that contain the most relevant information from the sample data.

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (32)

Sufficient statistic

In statistics, a statistic is sufficient with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the parameter". In particular, a statistic is sufficient for a family of probability distributions if the sample from which it is calculated gives no additional information than the statistic, as to which of those probability distributions is the sampling distribution.

Minimum-variance unbiased estimator

In statistics a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an unbiased estimator that has lower variance than any other unbiased estimator for all possible values of the parameter. For practical statistics problems, it is important to determine the MVUE if one exists, since less-than-optimal procedures would naturally be avoided, other things being equal. This has led to substantial development of statistical theory related to the problem of optimal estimation.

Bias of an estimator

In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called unbiased. In statistics, "bias" is an property of an estimator. Bias is a distinct concept from consistency: consistent estimators converge in probability to the true value of the parameter, but may be biased or unbiased; see bias versus consistency for more.

Maximum likelihood estimation

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.

Estimator

In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the sample mean is a commonly used estimator of the population mean. There are point and interval estimators. The point estimators yield single-valued results. This is in contrast to an interval estimator, where the result would be a range of plausible values.

Related lectures (64)

Maximum Likelihood Estimation: Properties and Applications

Covers Maximum Likelihood Estimation properties, applications, and assumptions, providing a comprehensive understanding of MLE concepts and their practical implications.

Optimality in Decision Theory: Unbiased Estimation

Explores optimality in decision theory and unbiased estimation, emphasizing sufficiency, completeness, and lower bounds for risk.

Confidence Intervals: Definition and Estimation

Explains confidence intervals, parameter estimation methods, and the central limit theorem in statistical inference.

Estimation: Measures of Performance

Explores estimation measures of performance, including the Cramér-Rao bound and maximum likelihood estimation.

Maximum Likelihood Estimation: Multivariate Statistics