Lecture

Kernel Methods: SVM and Regression

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers the concepts of linear models, margin, maximum margin classifier, support vector machine, slack variables, curse of dimensionality, nearest neighbor method, k-nearest neighbors, polynomial curve fitting, polynomial feature expansion, kernel functions, kernel trick, kernel regression, Gaussian process regression, and the dual formulation of SVM. It also explains the importance of feature expansion, the Representer theorem, and the prediction process in kernel regression. The lecture concludes with examples of kernel SVM and the advantages and drawbacks of kernel methods.

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (32)

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Kernel (statistics)

The term kernel is used in statistical analysis to refer to a window function. The term "kernel" has several distinct meanings in different branches of statistics. In statistics, especially in Bayesian statistics, the kernel of a probability density function (pdf) or probability mass function (pmf) is the form of the pdf or pmf in which any factors that are not functions of any of the variables in the domain are omitted. Note that such factors may well be functions of the parameters of the pdf or pmf.

Nonparametric regression

Nonparametric regression is a category of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data. That is, no parametric form is assumed for the relationship between predictors and dependent variable. Nonparametric regression requires larger sample sizes than regression based on parametric models because the data must supply the model structure as well as the model estimates.

Kernel regression

In statistics, kernel regression is a non-parametric technique to estimate the conditional expectation of a random variable. The objective is to find a non-linear relation between a pair of random variables X and Y. In any nonparametric regression, the conditional expectation of a variable relative to a variable may be written: where is an unknown function. Nadaraya and Watson, both in 1964, proposed to estimate as a locally weighted average, using a kernel as a weighting function.

Related lectures (64)

Feature Expansion and Kernel Methods

Explores feature expansion, kernel methods, SVM, and nonlinear classification in machine learning.

Feature Expansion and Kernels

Covers feature expansion, kernels, SVM, and nonlinear classification in machine learning.

Introduction to Machine Learning: Supervised Learning

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Machine Learning Review

Covers a review of machine learning concepts, including supervised learning, classification vs regression, linear models, kernel functions, support vector machines, dimensionality reduction, deep generative models, and cross-validation.

Kernel Methods: Neural Networks