Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Kernel Methods: RKHS and Kernels

In course

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning ...

Description

This lecture covers the Reproducing Kernel Hilbert Space (RKHS) as a Hilbert space with continuous evaluation functionals, the positive definite property of RKHS reproducing kernels, and the Moore-Aronszajn theorem. It also discusses linear, polynomial, and Gaussian RKHS.

Instructor

Yoav Zemel

Official source

Related concepts (32)

Positive-definite kernel

In operator theory, a branch of mathematics, a positive-definite kernel is a generalization of a positive-definite function or a positive-definite matrix. It was first introduced by James Mercer in the early 20th century, in the context of solving integral operator equations. Since then, positive-definite functions and their various analogues and generalizations have arisen in diverse parts of mathematics.

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Reproducing kernel Hilbert space

In functional analysis (a branch of mathematics), a reproducing kernel Hilbert space (RKHS) is a Hilbert space of functions in which point evaluation is a continuous linear functional. Roughly speaking, this means that if two functions and in the RKHS are close in norm, i.e., is small, then and are also pointwise close, i.e., is small for all . The converse does not need to be true. Informally, this can be shown by looking at the supremum norm: the sequence of functions converges pointwise, but does not converge uniformly i.

Hilbert space

In mathematics, Hilbert spaces (named after David Hilbert) allow the methods of linear algebra and calculus to be generalized from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise naturally and frequently in mathematics and physics, typically as function spaces. Formally, a Hilbert space is a vector space equipped with an inner product that induces a distance function for which the space is a complete metric space.

Rigged Hilbert space

In mathematics, a rigged Hilbert space (Gelfand triple, nested Hilbert space, equipped Hilbert space) is a construction designed to link the distribution and square-integrable aspects of functional analysis. Such spaces were introduced to study spectral theory in the broad sense. They bring together the 'bound state' (eigenvector) and 'continuous spectrum', in one place. A function such as is an eigenfunction of the differential operator on the real line R, but isn't square-integrable for the usual Borel measure on R.

Related lectures (48)

Mercer Theorem and Kernels

Explores the Mercer Theorem, Kernels, and their role in machine learning applications.

Kernel Methods: Machine Learning

Covers Kernel Methods in Machine Learning, focusing on overfitting, model selection, cross-validation, regularization, kernel functions, and SVM.

Postulates of Quantum Mechanics

Explores the postulates of Quantum Mechanics, focusing on states, time evolution, and measurement.

Quantization: Topological Operators

Covers the quantization of topological operators and Ising models on square lattices.

Linear Algebra in 3D: Images and Kernels