Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Statistical Justification of Least Squares

In course

MGT-448: Statistical inference and machine learning

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data ...

Description

This lecture covers the statistical justification of the Least Squares method, explaining the error terms, independently and identically distributed random variables, likelihood function, logistic regression, Perceptron learning algorithm, and Generalized Linear Models. The instructor discusses the exponential family, natural parameters, and canonical parameters, emphasizing the relationship between data and parameters.

Instructor

Negar Kiyavash

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Data analysis: Regression analysis

Related concepts (32)

Exponential family

In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate expectations, covariances using differentiation based on some useful algebraic properties, as well as for generality, as exponential families are in a sense very natural sets of distributions to consider. The term exponential class is sometimes used in place of "exponential family", or the older term Koopman–Darmois family.

Generalized linear model

In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value. Generalized linear models were formulated by John Nelder and Robert Wedderburn as a way of unifying various other statistical models, including linear regression, logistic regression and Poisson regression.

Natural exponential family

In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF). The natural exponential families (NEF) are a subset of the exponential families. A NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in an exponential family with parameter θ can be written with probability density function (PDF) where and are known functions.

Likelihood function

In statistical inference, the likelihood function quantifies the plausibility of parameter values characterizing a statistical model in light of observed data. Its most typical usage is to compare possible parameter values (under a fixed set of observations and a particular model), where higher values of likelihood are preferred because they correspond to more probable parameter values.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Related lectures (51)

Generalised Linear Models: Regression with Exponential Family Responses

Covers regression with exponential family responses using Generalised Linear Models.

Weighted Least Squares Estimation: IRLS Algorithm

Explores the IRLS algorithm for weighted least squares estimation in GLM.

Logistic Regression: Statistical Inference and Machine Learning

Covers logistic regression, likelihood function, Newton's method, and classification error estimation.

Generalized Linear Regression: Classification

Explores Generalized Linear Regression, Classification, confusion matrices, ROC curves, and noise in data.

Generalized Linear Models: GLMs for Non-Gaussian Data