Lecture

SVM Hyperparameters

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MICRO-455: Machine learning I

Real-world engineering applications must cope with a large dataset of dynamic variables, which cannot be well approximated by classical or deterministic models. This course gives an overview of methods from Machine Learning for the analysis of non-linear, highly noisy and multi dimensional data

Description

This lecture explains the key hyperparameters in Support Vector Machines (SVM), focusing on the C parameter for controlling misclassifications and the kernel width parameter for adjusting the smoothness of the boundary. The instructor illustrates how different values of these hyperparameters impact the classification results, highlighting the trade-off between accuracy and overfitting.

Instructor

Aude Billard

Expertise RoboticsMachine LearningArtificial IntelligenceHuman-Robot InteractionMechatronicsDynamical Systems ControlComputational Neuroscience Aude Billard is full professor and head of the LASA laboratory at the School of Engineering at the Swiss Institute of Technology Lausanne (EPFL), and the Director of the Swiss Innovation Booster on Robotics. Dr. Billard acts as the President of the IEEE Robotics and Automation Society (RAS), after serving in several roles in the administrative and executive committe

Official source

Related concepts (19)

Kernel (statistics)

The term kernel is used in statistical analysis to refer to a window function. The term "kernel" has several distinct meanings in different branches of statistics. In statistics, especially in Bayesian statistics, the kernel of a probability density function (pdf) or probability mass function (pmf) is the form of the pdf or pmf in which any factors that are not functions of any of the variables in the domain are omitted. Note that such factors may well be functions of the parameters of the pdf or pmf.

Overfitting

In mathematical modeling, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably". An overfitted model is a mathematical model that contains more parameters than can be justified by the data. In a mathematical sense, these parameters represent the degree of a polynomial. The essence of overfitting is to have unknowingly extracted some of the residual variation (i.

Kernel smoother

A kernel smoother is a statistical technique to estimate a real valued function as the weighted average of neighboring observed data. The weight is defined by the kernel, such that closer points are given higher weights. The estimated function is smooth, and the level of smoothness is set by a single parameter. Kernel smoothing is a type of weighted moving average. Let be a kernel defined by where: is the Euclidean norm is a parameter (kernel radius) D(t) is typically a positive real valued function, whose value is decreasing (or not increasing) for the increasing distance between the X and X0.

Kernel density estimation

In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on kernels as weights. KDE answers a fundamental data smoothing problem where inferences about the population are made, based on a finite data sample. In some fields such as signal processing and econometrics it is also termed the Parzen–Rosenblatt window method, after Emanuel Parzen and Murray Rosenblatt, who are usually credited with independently creating it in its current form.

Generalization error

For supervised learning applications in machine learning and statistical learning theory, generalization error (also known as the out-of-sample error or the risk) is a measure of how accurately an algorithm is able to predict outcome values for previously unseen data. Because learning algorithms are evaluated on finite samples, the evaluation of a learning algorithm may be sensitive to sampling error. As a result, measurements of prediction error on the current data may not provide much information about predictive ability on new data.

Related lectures (52)

Kernel Methods: Machine Learning

Covers Kernel Methods in Machine Learning, focusing on overfitting, model selection, cross-validation, regularization, kernel functions, and SVM.

Kernel Methods in Machine Learning: Kernel Regression and SVM

Discusses kernel methods in machine learning, focusing on kernel regression and support vector machines, including their formulations and applications.

Kernel Methods

Covers overfitting, model selection, validation methods, kernel functions, and SVM concepts.

Feature Expansion and Kernel Methods

Explores feature expansion, kernel methods, SVM, and nonlinear classification in machine learning.

Kernel Methods: Understanding Overfitting and Model Selection