Lecture

Recursive Least-Squares: Weighted Formulation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

EE-566: Adaptation and learning

In this course, students learn to design and master algorithms and core concepts related to inference and learning from data and the foundations of adaptation and learning theories with applications.

Description

This lecture covers the Recursive Least-Squares (RLS) algorithm, focusing on the exponentially weighted formulation with 2-regularization. The algorithm enables real-time updating of the minimizer, making it suitable for streaming data. The lecture also discusses the derivation of the RLS algorithm, including the recursive update steps and the weighted least-squares problem. The presentation emphasizes the importance of forgetting factors, regularization parameters, and sample averaging in the algorithm's efficiency.

Instructor

Ali H. Sayed

Ali H. Sayed is Dean of Engineering at EPFL, Switzerland, where he also leads the Adaptive Systems Laboratory. He has also served as Distinguished Professor and Chairman of Electrical Engineering at UCLA. He is recognized as a Highly Cited Researcher and is a member of the US National Academy of Engineering. He is also a member of the World Academy of Sciences and served as President of the IEEE Signal Processing Society during 2018 and 2019. Dr. Sayed is an author/co-author of over 570 scholarly publications and six books. His research involves several areas including adaptation and learning theories, data and network sciences, statistical inference, and multiagent systems. His work has been recognized with several major awards including the 2022 IEEE Fourier Award, the 2020 Norbert Wiener Society Award and the 2015 Education Award from the IEEE Signal Processing Society, the 2014 Papoulis Award from the European Association for Signal Processing, the 2013 Meritorious Service Award and the 2012 Technical Achievement Award from the IEEE Signal Processing Society, the 2005 Terman Award from the American Society for Engineering Education, the 2005 Distinguished Lecturer from the IEEE Signal Processing Society, the 2003 Kuwait Prize, and the 1996 IEEE Donald G. Fink Prize. His publications have been awarded several Best Paper Awards from the IEEE (2002, 2005, 2012, 2014) and EURASIP (2015). He is a Fellow of IEEE, EURASIP, and the American Association for the Advancement of Science (AAAS); the publisher of the journal Science.

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (32)

Weighted least squares

Weighted least squares (WLS), also known as weighted linear regression, is a generalization of ordinary least squares and linear regression in which knowledge of the unequal variance of observations (heteroscedasticity) is incorporated into the regression. WLS is also a specialization of generalized least squares, when all the off-diagonal entries of the covariance matrix of the errors, are null.

Generalized least squares

In statistics, generalized least squares (GLS) is a method used to estimate the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in the regression model. Least squares and weighted least squares may need to be more statistically efficient and prevent misleading inferences. GLS was first described by Alexander Aitken in 1935. In standard linear regression models one observes data on n statistical units.

Sampling error

In statistics, sampling errors are incurred when the statistical characteristics of a population are estimated from a subset, or sample, of that population. It can produced biased results. Since the sample does not include all members of the population, statistics of the sample (often known as estimators), such as means and quartiles, generally differ from the statistics of the entire population (known as parameters). The difference between the sample statistic and population parameter is considered the sampling error.

Simple random sample

In statistics, a simple random sample (or SRS) is a subset of individuals (a sample) chosen from a larger set (a population) in which a subset of individuals are chosen randomly, all with the same probability. It is a process of selecting a sample in a random way. In SRS, each subset of k individuals has the same probability of being chosen for the sample as any other subset of k individuals. A simple random sample is an unbiased sampling technique. Simple random sampling is a basic type of sampling and can be a component of other more complex sampling methods.

Streaming media

Streaming media is multimedia that is delivered and consumed in a continuous manner from a source, with little or no intermediate storage in network elements. Streaming refers to the delivery method of content, rather than the content itself. Distinguishing delivery method from the media applies specifically to telecommunications networks, as most of the traditional media delivery systems are either inherently streaming (e.g. radio, television) or inherently non-streaming (e.g. books, videotapes, audio CDs).

Related lectures (64)

Sampling: Weighted Exogenous Maximum Likelihood Estimation MOOC: Selected Topics on Discrete Choice

Introduces the Weighted Exogenous Maximum Likelihood Estimator in discrete choice models.

Weighted Least Squares Estimation: IRLS Algorithm

Explores the IRLS algorithm for weighted least squares estimation in GLM.

Support Vector Machines: Non-Linear Data Mapping

Explores mapping non-linear data to higher dimensions using SVM and covers polynomial feature expansion, regularization, noise implications, and curve-fitting methods.

Linear Regression

Covers linear regression for estimating train speed using least squares and regularization.

Linear Regression: Statistical Inference and Regularization