Lecture

Linear Regression: Absence or Presence of Covariates

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers the fundamental concepts of linear regression, focusing on the absence or presence of covariates. It explains the distinction between marginal inference and regression, where observations are generated under different experimental conditions. The lecture delves into the bewildering variety of models that can be captured by the general specification of independent distributions. It also discusses the tools of the trade, starting from Gaussian linear regression and gradually generalizing to subspaces, projection matrices, and optimal dimension reduction. The concept of subspaces and spectra associated with real matrices is explored, along with orthogonal projections and the singular value decomposition theorem.

Official source

Ontological neighbourhood

Mathematics

Algebra: Linear algebra

Statistics

Data analysis: Regression analysis

Related concepts (32)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Eigenvalues and eigenvectors

In linear algebra, an eigenvector (ˈaɪgənˌvɛktər) or characteristic vector of a linear transformation is a nonzero vector that changes at most by a constant factor when that linear transformation is applied to it. The corresponding eigenvalue, often represented by , is the multiplying factor. Geometrically, a transformation matrix rotates, stretches, or shears the vectors it acts upon. The eigenvectors for a linear transformation matrix are the set of vectors that are only stretched, with no rotation or shear.

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one or more independent variables (often called 'predictors', 'covariates', 'explanatory variables' or 'features'). The most common form of regression analysis is linear regression, in which one finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

Normal distribution

In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution (and also its median and mode), while the parameter is its standard deviation. The variance of the distribution is . A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

Singular value decomposition

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any matrix. It is related to the polar decomposition. Specifically, the singular value decomposition of an complex matrix M is a factorization of the form where U is an complex unitary matrix, is an rectangular diagonal matrix with non-negative real numbers on the diagonal, V is an complex unitary matrix, and is the conjugate transpose of V.

Related lectures (64)

Linear Regression Testing

Explores least squares in linear regression, hypothesis testing, outliers, and model assumptions.

Canonical Correlation Analysis: Overview

Covers Canonical Correlation Analysis, a method to find relationships between two sets of variables.

Subspaces, Spectra, and Projections

Explores subspaces, spectra, and projections in linear algebra, including symmetric matrices and orthogonal projections.

Orthogonal Projections in Linear Algebra

Explores orthogonal projections, orthonormal bases, and QR factorization in linear algebra.

Linear Algebra: Singular Value Decomposition