Lecture

Local Averaging Predictors

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture introduces local averaging predictors, including K-nearest neighbors, histogram-based methods, and Nadaraya-Watson predictors. It covers the properties of local averaging predictors, the general framework of local empirical risk minimization, and local linear regression (LOWESS). The slides explain the principle of local averaging methods, different types of local averaging methods, convolution kernels, and Nadaraya-Watson estimators. It also discusses star velocity estimation with KNN, comments on local averaging methods, local empirical risk minimization, and local quadratic ERM. The lecture concludes with local linear regression, its applications, and generalizations to local polynomial and spline regression.

Instructor

Yoav Zemel

Official source

Related concepts (28)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Local regression

Local regression or local polynomial regression, also known as moving regression, is a generalization of the moving average and polynomial regression. Its most common methods, initially developed for scatterplot smoothing, are LOESS (locally estimated scatterplot smoothing) and LOWESS (locally weighted scatterplot smoothing), both pronounced ˈloʊɛs. They are two strongly related non-parametric regression methods that combine multiple regression models in a k-nearest-neighbor-based meta-model.

Polynomial regression

In statistics, polynomial regression is a form of regression analysis in which the relationship between the independent variable x and the dependent variable y is modelled as an nth degree polynomial in x. Polynomial regression fits a nonlinear relationship between the value of x and the corresponding conditional mean of y, denoted E(y |x). Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknown parameters that are estimated from the data.

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one or more independent variables (often called 'predictors', 'covariates', 'explanatory variables' or 'features'). The most common form of regression analysis is linear regression, in which one finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Related lectures (64)

Regression: Interactive Lecture

Covers linear regression, weighted regression, locally weighted regression, support vector regression, noise handling, and eye mapping using SVR.

Linear Regression Analysis

Introduces linear regression analysis, covering model building, predictors, coefficients, and outcome interpretation.

Non-parametric Regression: Smoothing Techniques

Explores non-parametric regression techniques, including splines, bias-variance tradeoff, orthogonal functions, wavelets, and modulation estimators.

Introduction to Machine Learning: Regression Fundamentals

Introduces the fundamentals of regression in machine learning, covering course logistics, key concepts, and the importance of loss functions in model evaluation.

Probabilistic Models for Linear Regression