Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Method of Least Squares: Normal Equations

In course

MATH-111(f): Linear Algebra

L'objectif du cours est d'introduire les notions de base de l'algèbre linéaire et ses applications. ...

Description

This lecture covers the method of least squares and the concept of normal equations. It explains how to find the closest vector to a given one, seeking the solution that minimizes the error. The instructor demonstrates the process of establishing normal equations and provides examples to illustrate the concept. The lecture also discusses the finiteness of solutions and the uniqueness of solutions in the least squares sense.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Mikaël Cavallin

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (32)

Ordinary least squares

In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the principle of least squares: minimizing the sum of the squares of the differences between the observed dependent variable (values of the variable being observed) in the input dataset and the output of the (linear) function of the independent variable.

Least squares

The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the residuals (a residual being the difference between an observed value and the fitted value provided by a model) made in the results of each individual equation. The most important application is in data fitting.

Linear least squares

Linear least squares (LLS) is the least squares approximation of linear functions to data. It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and generalized (correlated) residuals. Numerical methods for linear least squares include inverting the matrix of the normal equations and orthogonal decomposition methods. The three main linear least squares formulations are: Ordinary least squares (OLS) is the most common estimator.

Generalized least squares

In statistics, generalized least squares (GLS) is a method used to estimate the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in the regression model. Least squares and weighted least squares may need to be more statistically efficient and prevent misleading inferences. GLS was first described by Alexander Aitken in 1935. In standard linear regression models one observes data on n statistical units.

Non-linear least squares

Non-linear least squares is the form of least squares analysis used to fit a set of m observations with a model that is non-linear in n unknown parameters (m ≥ n). It is used in some forms of nonlinear regression. The basis of the method is to approximate the model by a linear one and to refine the parameters by successive iterations. There are many similarities to linear least squares, but also some significant differences.

Related lectures (46)

Linear Equations: Least Squares Solution MOOC: Linear Algebra (Part 3)

Explains how to solve linear equations using the least squares method to minimize errors in the system.

Linear Regression Testing

Explores least squares in linear regression, hypothesis testing, outliers, and model assumptions.

Linear Regression: Maximum Likelihood Approach

Covers linear regression topics including confidence intervals, variance, and maximum likelihood approach.

Hadamard Design: Introduction and Construction

Introduces Hadamard and Plackett-Burman designs, explaining construction, analysis, and optimization techniques.