Lecture

Support Vector Machines: Kernel SVM

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers the concept of non-linear SVM using kernels, where linear functions may not be suitable for data separation. It introduces the idea of transforming data into a higher-dimensional space for linear separability, defining a redescription space for linear separation. The lecture explains the general case of redescription spaces in Hilbert spaces and the importance of completeness and separability. It delves into the optimization problem of training SVM in the redescription space and the use of kernels to avoid explicit knowledge of the transformation. The lecture also discusses the definition and properties of kernels, including quadratic, polynomial, and radial basis function kernels.

Official source

Related concepts (24)

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Polynomial kernel

In machine learning, the polynomial kernel is a kernel function commonly used with support vector machines (SVMs) and other kernelized models, that represents the similarity of vectors (training samples) in a feature space over polynomials of the original variables, allowing learning of non-linear models. Intuitively, the polynomial kernel looks not only at the given features of input samples to determine their similarity, but also combinations of these. In the context of regression analysis, such combinations are known as interaction features.

Reproducing kernel Hilbert space

In functional analysis (a branch of mathematics), a reproducing kernel Hilbert space (RKHS) is a Hilbert space of functions in which point evaluation is a continuous linear functional. Roughly speaking, this means that if two functions and in the RKHS are close in norm, i.e., is small, then and are also pointwise close, i.e., is small for all . The converse does not need to be true. Informally, this can be shown by looking at the supremum norm: the sequence of functions converges pointwise, but does not converge uniformly i.

Hilbert space

In mathematics, Hilbert spaces (named after David Hilbert) allow the methods of linear algebra and calculus to be generalized from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise naturally and frequently in mathematics and physics, typically as function spaces. Formally, a Hilbert space is a vector space equipped with an inner product that induces a distance function for which the space is a complete metric space.

Radial basis function kernel

In machine learning, the radial basis function kernel, or RBF kernel, is a popular kernel function used in various kernelized learning algorithms. In particular, it is commonly used in support vector machine classification. The RBF kernel on two samples and x', represented as feature vectors in some input space, is defined as may be recognized as the squared Euclidean distance between the two feature vectors. is a free parameter.

Related lectures (64)

Kernel Methods: RKHS and Kernels

Explores RKHS, positive definite kernels, and the Moore-Aronszajn theorem in kernel methods.

Nonlinear SVM: Kernels and Dual Optimization

Explores transforming data with nonlinear maps, kernels, dual optimization, and interpreting SVM results.

Kernel Methods: Machine Learning

Covers Kernel Methods in Machine Learning, focusing on overfitting, model selection, cross-validation, regularization, kernel functions, and SVM.

Kernel Methods and Regression

Covers kernel methods, kernel regression, RBF kernel, and SVM for classification.

Feature Expansion: Kernels and KNN