Lecture

Kernel Methods: Representer Theorem and SVM Applications

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture covers the Representer Theorem, Regularized Empirical Risk Minimization in a Reproducing Kernel Hilbert Space (RKHS), Nonlinear Support Vector Machines (SVM), smoothness measurement in RKHS, kernel combinations, scalability issues for kernel methods, and the difference between convolution kernels and Mercer kernels.

Instructor

Yoav Zemel

Official source

Related concepts (18)

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Integral transform

In mathematics, an integral transform maps a function from its original function space into another function space via integration, where some of the properties of the original function might be more easily characterized and manipulated than in the original function space. The transformed function can generally be mapped back to the original function space using the inverse transform. An integral transform is any transform of the following form: The input of this transform is a function , and the output is another function .

Reproducing kernel Hilbert space

In functional analysis (a branch of mathematics), a reproducing kernel Hilbert space (RKHS) is a Hilbert space of functions in which point evaluation is a continuous linear functional. Roughly speaking, this means that if two functions and in the RKHS are close in norm, i.e., is small, then and are also pointwise close, i.e., is small for all . The converse does not need to be true. Informally, this can be shown by looking at the supremum norm: the sequence of functions converges pointwise, but does not converge uniformly i.

Convolution

In mathematics (in particular, functional analysis), convolution is a mathematical operation on two functions (f and g) that produces a third function () that expresses how the shape of one is modified by the other. The term convolution refers to both the result function and to the process of computing it. It is defined as the integral of the product of the two functions after one is reflected about the y-axis and shifted. The choice of which function is reflected and shifted before the integral does not change the integral result (see commutativity).

Positive-definite kernel

In operator theory, a branch of mathematics, a positive-definite kernel is a generalization of a positive-definite function or a positive-definite matrix. It was first introduced by James Mercer in the early 20th century, in the context of solving integral operator equations. Since then, positive-definite functions and their various analogues and generalizations have arisen in diverse parts of mathematics.

Related lectures (52)

Kernel Methods: SVM and Regression

Introduces kernel methods like SVM and regression, covering concepts such as margin, support vector machine, curse of dimensionality, and Gaussian process regression.

Kernel Methods: RKHS and Kernels

Explores RKHS, positive definite kernels, and the Moore-Aronszajn theorem in kernel methods.

Kernel Methods: Machine Learning

Covers Kernel Methods in Machine Learning, focusing on overfitting, model selection, cross-validation, regularization, kernel functions, and SVM.

Feature Expansion and Kernel Methods

Explores feature expansion, kernel methods, SVM, and nonlinear classification in machine learning.

Kernel Methods: Neural Networks