Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Likelihood Estimation and Least Squares

In course

MATH-341: Linear models

Regression modelling is a fundamental tool of statistics, because it describes how the law of a random variable of interest may depend on other variables. This course aims to familiarize students with linear models and some of their extensions, which lie a ...

Description

This lecture covers the concepts of simple normal linear regression, multiple normal linear regression, and maximum likelihood estimation. It explains the jargon, linearity in parameters, data structure, and the process of finding the least squares estimator. Examples include a professor's van fuel consumption and cement heat evolution.

Instructors (2)

Tomas Masák

Victor Panaretos

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Statistical inference: Mathematical statistics

Related concepts (32)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Linear model

In statistics, the term linear model is used in different ways according to the context. The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model. However, the term is also used in time series analysis with a different meaning. In each case, the designation "linear" is used to identify a subclass of models for which substantial reduction in the complexity of the related statistical theory is possible.

Generalized linear model

In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value. Generalized linear models were formulated by John Nelder and Robert Wedderburn as a way of unifying various other statistical models, including linear regression, logistic regression and Poisson regression.

General linear model

The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regression models may be compactly written as where Y is a matrix with series of multivariate measurements (each column being a set of measurements on one of the dependent variables), X is a matrix of observations on independent variables that might be a design matrix (each column being a set of observations on one of the independent variables), B is a matrix containing parameters that are usually to be estimated and U is a matrix containing errors (noise).

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one or more independent variables (often called 'predictors', 'covariates', 'explanatory variables' or 'features'). The most common form of regression analysis is linear regression, in which one finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

Related lectures (51)

Regression: Linear Models

Introduces linear regression, generalized linear models, and mixed-effect models for regression analysis.

Weighted Least Squares Estimation: IRLS Algorithm

Explores the IRLS algorithm for weighted least squares estimation in GLM.

Linear Regression: Estimation and Inference

Explores linear regression estimation, linearity assumptions, and statistical tests in the context of model comparison.

Linear Regression: Maximum Likelihood Approach

Covers linear regression topics including confidence intervals, variance, and maximum likelihood approach.

Linear Regression: Statistical Inference and Regularization