Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Linear SVM derivation

In course

MICRO-455: Machine learning I

Description

This lecture covers the derivation of Linear Support Vector Machine (SVM) starting from the classifier margin concept, computing the distance to the separating hyperplane, determining constraints, solving the constrained optimization problem, interpreting the solution, and discussing the dual optimization using the Karush-Kuhn-Tucker (KKT) conditions.

Instructor

Aude Billard

Official source

Related concepts (23)

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Convex optimization

Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization problems admit polynomial-time algorithms, whereas mathematical optimization is in general NP-hard.

Mathematical optimization

Mathematical optimization (alternatively spelled optimisation) or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfields: discrete optimization and continuous optimization. Optimization problems arise in all quantitative disciplines from computer science and engineering to operations research and economics, and the development of solution methods has been of interest in mathematics for centuries.

Duality (optimization)

In mathematical optimization theory, duality or the duality principle is the principle that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. If the primal is a minimization problem then the dual is a maximization problem (and vice versa). Any feasible solution to the primal (minimization) problem is at least as large as any feasible solution to the dual (maximization) problem.

Linear classifier

In the field of machine learning, the goal of statistical classification is to use an object's characteristics to identify which class (or group) it belongs to. A linear classifier achieves this by making a classification decision based on the value of a linear combination of the characteristics. An object's characteristics are also known as feature values and are typically presented to the machine in a vector called a feature vector.

Related lectures (30)

Linear Models for Classification

Explores linear models, logistic regression, classification metrics, SVM, and their practical use in data science methods.

Linear Models for Classification

Covers linear models for classification, including SVM, decision boundaries, support vectors, and Lagrange duality.

Max-Margin Classifiers

Explores maximizing margins for better classification using support vector machines and the importance of choosing the right parameter.

Support Vector Machines: Maximizing Margin

Explores Support Vector Machines, maximizing margin for robust classification and the transition to soft SVM for non-linearly separable data.

SVM - Principle: Linear Classifiers