Lecture

Weighted Least Squares Estimation: IRLS Algorithm

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers the Iteratively Reweighted Least Squares (IRLS) algorithm for weighted least squares estimation. The instructor explains the heuristics behind the sampling distribution of the Maximum Likelihood Estimator (MLE) in Generalized Linear Models (GLM). The lecture delves into the iterative process, the score expression, and the importance of the weight matrix in the estimation process. Additionally, the lecture discusses the asymptotic normality of MLE in GLM, goodness of fit measures, nested models comparison using deviance, and diagnostic tools like leverage and Cook's statistic.

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Statistical inference: Mathematical statistics

Related concepts (32)

Multinomial logistic regression

In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes. That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables (which may be real-valued, binary-valued, categorical-valued, etc.).

Binary data

Binary data is data whose unit can take on only two possible states. These are often labelled as 0 and 1 in accordance with the binary numeral system and Boolean algebra. Binary data occurs in many different technical and scientific fields, where it can be called by different names including bit (binary digit) in computer science, truth value in mathematical logic and related domains and binary variable in statistics. A discrete variable that can take only one state contains zero information, and is the next natural number after 1.

Weighted least squares

Weighted least squares (WLS), also known as weighted linear regression, is a generalization of ordinary least squares and linear regression in which knowledge of the unequal variance of observations (heteroscedasticity) is incorporated into the regression. WLS is also a specialization of generalized least squares, when all the off-diagonal entries of the covariance matrix of the errors, are null.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Residual sum of squares

In statistics, the residual sum of squares (RSS), also known as the sum of squared residuals (SSR) or the sum of squared estimate of errors (SSE), is the sum of the squares of residuals (deviations predicted from actual empirical values of data). It is a measure of the discrepancy between the data and an estimation model, such as a linear regression. A small RSS indicates a tight fit of the model to the data. It is used as an optimality criterion in parameter selection and model selection.

Related lectures (64)

Generalised Linear Models: Regression with Exponential Family Responses

Covers regression with exponential family responses using Generalised Linear Models.

Generalized Linear Models: GLMs for Non-Gaussian Data

Explores Generalized Linear Models for non-Gaussian data, covering interpretation of natural link function, MLE asymptotic normality, deviance measures, residuals, and logistic regression.

Probabilistic Models for Linear Regression

Covers the probabilistic model for linear regression and its applications in nuclear magnetic resonance and X-ray imaging.

Linear Regression: Maximum Likelihood Approach

Covers linear regression topics including confidence intervals, variance, and maximum likelihood approach.

Linear Regression: Statistical Inference and Regularization