Lecture

Linear Regression: Parametric Models and Optimization

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

EE-311: Fundamentals of machine learning

Ce cours présente une vue générale des techniques d'apprentissage automatique, passant en revue les algorithmes, le formalisme théorique et les protocoles expérimentaux. ...

Description

This lecture introduces linear regression as a supervised learning problem, focusing on parametric models and optimization techniques to find the optimal parameters. The instructor explains how to make predictions from labeled examples and demonstrates the process through a linear regression example. The lecture covers the concept of minimizing the sum of least squares to estimate the model parameters and emphasizes the importance of defining the model function. Additionally, it discusses the application of linear regression in predicting outcomes based on observations, highlighting the analytical approach to solving regression problems.

Instructors (2)

Michael Stefan Daniel Liebling

Teaching & PhD Past EPFL PhD Students as codirector Christian Jaques, Olivia Mariani, Adrian Shajkofci, François Thierry M Marelli Courses Fundamentals of machine learning EE-311 This course provides a general overview of machine learning, covering the main algorithms, theoretical formalisms, and experimental protocols.

François Fleuret

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (27)

Linear least squares

Linear least squares (LLS) is the least squares approximation of linear functions to data. It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and generalized (correlated) residuals. Numerical methods for linear least squares include inverting the matrix of the normal equations and orthogonal decomposition methods. The three main linear least squares formulations are: Ordinary least squares (OLS) is the most common estimator.

Bayesian linear regression

Bayesian linear regression is a type of conditional modeling in which the mean of one variable is described by a linear combination of other variables, with the goal of obtaining the posterior probability of the regression coefficients (as well as other parameters describing the distribution of the regressand) and ultimately allowing the out-of-sample prediction of the regressand (often labelled ) conditional on observed values of the regressors (usually ).

Generalized least squares

In statistics, generalized least squares (GLS) is a method used to estimate the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in the regression model. Least squares and weighted least squares may need to be more statistically efficient and prevent misleading inferences. GLS was first described by Alexander Aitken in 1935. In standard linear regression models one observes data on n statistical units.

Non-linear least squares

Non-linear least squares is the form of least squares analysis used to fit a set of m observations with a model that is non-linear in n unknown parameters (m ≥ n). It is used in some forms of nonlinear regression. The basis of the method is to approximate the model by a linear one and to refine the parameters by successive iterations. There are many similarities to linear least squares, but also some significant differences.

Segmented regression

Segmented regression, also known as piecewise regression or broken-stick regression, is a method in regression analysis in which the independent variable is partitioned into intervals and a separate line segment is fit to each interval. Segmented regression analysis can also be performed on multivariate data by partitioning the various independent variables. Segmented regression is useful when the independent variables, clustered into different groups, exhibit different relationships between the variables in these regions.

Related lectures (64)

Regression: Linear Models

Explores linear regression, least squares, residuals, and confidence intervals in regression models.

Linear Regression: Foundations and Applications

Introduces linear regression, covering its fundamentals, applications, and evaluation metrics in machine learning.

Linear Regression: Statistical Inference and Regularization

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Supervised Learning: Linear Regression

Covers supervised learning with a focus on linear regression, including topics like digit classification, spam detection, and wind speed prediction.

Overfitting in Supervised Learning: Case Studies and Techniques