Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Feature Expansion and Kernel Methods

Description

This lecture covers the concepts of feature expansion and kernel methods in machine learning. It starts with a recap of linear models, margin, maximum margin classifier, and support vector machines. It then delves into topics like overlapping classes, slack variables, different linear models for classification, and the impact of loss functions. The lecture progresses to discuss polynomial curve fitting, nearest neighbor methods, k-nearest neighbors, curse of dimensionality, and nonlinear classification. It concludes with an in-depth exploration of polynomial feature expansion, kernel functions, kernel regression, kernel SVM, and the kernel trick.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Related concepts (32)

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Kernel (statistics)

The term kernel is used in statistical analysis to refer to a window function. The term "kernel" has several distinct meanings in different branches of statistics. In statistics, especially in Bayesian statistics, the kernel of a probability density function (pdf) or probability mass function (pmf) is the form of the pdf or pmf in which any factors that are not functions of any of the variables in the domain are omitted. Note that such factors may well be functions of the parameters of the pdf or pmf.

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Kernel regression

In statistics, kernel regression is a non-parametric technique to estimate the conditional expectation of a random variable. The objective is to find a non-linear relation between a pair of random variables X and Y. In any nonparametric regression, the conditional expectation of a variable relative to a variable may be written: where is an unknown function. Nadaraya and Watson, both in 1964, proposed to estimate as a locally weighted average, using a kernel as a weighting function.

Kernel smoother

A kernel smoother is a statistical technique to estimate a real valued function as the weighted average of neighboring observed data. The weight is defined by the kernel, such that closer points are given higher weights. The estimated function is smooth, and the level of smoothness is set by a single parameter. Kernel smoothing is a type of weighted moving average. Let be a kernel defined by where: is the Euclidean norm is a parameter (kernel radius) D(t) is typically a positive real valued function, whose value is decreasing (or not increasing) for the increasing distance between the X and X0.

Related lectures (55)

Nonparametric Regression

Covers nonparametric regression, scatterplot smoothing, kernel methods, and bias-variance tradeoff.

Introduction to Machine Learning: Linear Models

Introduces linear models for supervised learning, covering overfitting, regularization, and kernels, with applications in machine learning tasks.

Introduction to Machine Learning: Supervised Learning

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Kernel Methods: Machine Learning

Covers Kernel Methods in Machine Learning, focusing on overfitting, model selection, cross-validation, regularization, kernel functions, and SVM.

Linear Models: Recap and Extensions