Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Feature Expansion: Kernels and KNN

In course

DH-406: Machine learning for DH

This course aims to introduce the basic principles of machine learning in the context of the digital humanities. We will cover both supervised and unsupervised learning techniques, and study and implement methods to analyze diverse data types, such as imag ...

Description

This lecture covers the concepts of feature expansion, kernels, and K-nearest neighbors (KNN). It starts with a recap of linear models and multi-output prediction, then delves into non-linearity, probabilistic interpretation, and dealing with multiple classes. The lecture explains decision boundaries, support vector machines (SVM), and handling overlapping classes. It introduces polynomial curve fitting, feature expansion, and polynomial kernels. The instructor discusses kernel functions, Gaussian kernels, and the kernel trick. The lecture also covers kernel regression, the Representer theorem, and kernelization of algorithms like SVM. It concludes with demonstrations of kernel regression and KNN for classification and regression.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Mathieu Salzmann

I am a Senior Researcher at EPFL-CVLab, and, since May 2020, an Artificial Intelligence Engineer at ClearSpace (50%). Previously, I was a Senior Researcher and Research Leader in NICTA's computer vision research group. Prior to this, from Sept. 2010 to Jan 2012, I was a Research Assistant Professor at TTI-Chicago, and, from Feb. 2009 to Aug. 2010, a postdoctoral fellow at ICSI and EECS at UC Berkeley under the supervision of Prof. Trevor Darrell. I obtained my PhD in Jan. 2009 from EPFL under the supervision of Prof. Pascal Fua.

Official source

Related concepts (32)

Kernel (operating system)

The kernel is a computer program at the core of a computer's operating system and generally has complete control over everything in the system. It is the portion of the operating system code that is always resident in memory and facilitates interactions between hardware and software components. A full kernel controls all hardware resources (e.g. I/O, memory, cryptography) via device drivers, arbitrates conflicts between processes concerning such resources, and optimizes the utilization of common resources e.

Hybrid kernel

A hybrid kernel is an operating system kernel architecture that attempts to combine aspects and benefits of microkernel and monolithic kernel architectures used in operating systems. The traditional kernel categories are monolithic kernels and microkernels (with nanokernels and exokernels seen as more extreme versions of microkernels). The "hybrid" category is controversial, due to the similarity of hybrid kernels and ordinary monolithic kernels; the term has been dismissed by Linus Torvalds as simple marketing.

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Monolithic kernel

A monolithic kernel is an operating system architecture where the entire operating system is working in kernel space. The monolithic model differs from other operating system architectures (such as the microkernel architecture) in that it alone defines a high-level virtual interface over computer hardware. A set of primitives or system calls implement all operating system services such as process management, concurrency, and memory management. Device drivers can be added to the kernel as modules.

Comparison of operating system kernels

A kernel is a component of a computer operating system. A comparison of system kernels can provide insight into the design and architectural choices made by the developers of particular operating systems. The following tables compare general and technical information for a number of widely used and currently available operating system kernels. Please see the individual products' articles for further information.

Related lectures (47)

Introduction to Machine Learning: Linear Models

Introduces linear models for supervised learning, covering overfitting, regularization, and kernels, with applications in machine learning tasks.

Kernel Methods: Machine Learning

Covers Kernel Methods in Machine Learning, focusing on overfitting, model selection, cross-validation, regularization, kernel functions, and SVM.

Introduction to Machine Learning: Supervised Learning

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Supervised Learning: Classification and Regression

Covers supervised learning, classification, regression, decision boundaries, overfitting, Perceptron, SVM, and logistic regression.

Kernel Methods and Regression