Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Support Vector Regression: Principles and Optimization

In course

MICRO-455: Machine learning I

Description

This lecture introduces Support Vector Regression (SVR) as an extension of the support vector machine framework for classification to estimate continuous functions. It covers the linear case, the E-tube concept, the ε-margin, optimization problems, and the solution for both linear and non-linear regression. The lecture also delves into the interpretation of SVR solutions, the role of hyperparameters in SVR optimization, and their influence on the fit.

Instructor

Aude Billard

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (32)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one or more independent variables (often called 'predictors', 'covariates', 'explanatory variables' or 'features'). The most common form of regression analysis is linear regression, in which one finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Ridge regression

Ridge regression is a method of estimating the coefficients of multiple-regression models in scenarios where the independent variables are highly correlated. It has been used in many fields including econometrics, chemistry, and engineering. Also known as Tikhonov regularization, named for Andrey Tikhonov, it is a method of regularization of ill-posed problems. It is particularly useful to mitigate the problem of multicollinearity in linear regression, which commonly occurs in models with large numbers of parameters.

Related lectures (30)

Linear Regression: Statistical Inference and Regularization

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Linear Models for Classification

Covers linear models for classification, including SVM, decision boundaries, support vectors, and Lagrange duality.

Kernel Methods: Machine Learning

Explores kernel methods in machine learning, emphasizing their application in regression tasks and the prevention of overfitting.

Introduction to Machine Learning: Supervised Learning

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Supervised Learning: Classification and Regression