Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Linear Models for Classification

In course

CS-233(a): Introduction to machine learning (BA3)

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed and practically implemented. ...

Description

This lecture covers linear models for classification, starting with a recap on simple parametric models and hyperplanes. It then delves into linear regression, binary classification, adding non-linearity, and probabilistic interpretation. The lecture also discusses logistic regression training, evaluating classifiers, classification metrics, ROC curves, decision boundaries, margin, support vectors, and the maximum margin classifier. It concludes with an overview of SVM, constrained optimization, Lagrange duality, and the practical use of SVMs in data science methods.

Instructor

Mathieu Salzmann

I am a Senior Researcher at EPFL-CVLab, and, since May 2020, an Artificial Intelligence Engineer at ClearSpace (50%). Previously, I was a Senior Researcher and Research Leader in NICTA's computer vision research group. Prior to this, from Sept. 2010 to Jan 2012, I was a Research Assistant Professor at TTI-Chicago, and, from Feb. 2009 to Aug. 2010, a postdoctoral fellow at ICSI and EECS at UC Berkeley under the supervision of Prof. Trevor Darrell. I obtained my PhD in Jan. 2009 from EPFL under the supervision of Prof. Pascal Fua.

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Theoretical computer science

Algorithms and data structures: Combinatorial optimization

Related concepts (32)

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Multiclass classification

In machine learning and statistical classification, multiclass classification or multinomial classification is the problem of classifying instances into one of three or more classes (classifying instances into one of two classes is called binary classification). While many classification algorithms (notably multinomial logistic regression) naturally permit the use of more than two classes, some are by nature binary algorithms; these can, however, be turned into multinomial classifiers by a variety of strategies.

Duality (optimization)

In mathematical optimization theory, duality or the duality principle is the principle that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. If the primal is a minimization problem then the dual is a maximization problem (and vice versa). Any feasible solution to the primal (minimization) problem is at least as large as any feasible solution to the dual (maximization) problem.

Lagrange multiplier

In mathematical optimization, the method of Lagrange multipliers is a strategy for finding the local maxima and minima of a function subject to equation constraints (i.e., subject to the condition that one or more equations have to be satisfied exactly by the chosen values of the variables). It is named after the mathematician Joseph-Louis Lagrange. The basic idea is to convert a constrained problem into a form such that the derivative test of an unconstrained problem can still be applied.

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Related lectures (35)

Linear Models for Classification

Covers linear models for classification, including SVM, decision boundaries, support vectors, and Lagrange duality.

Linear Models for Classification: Part 3

Explores linear models for classification, including binary classification, logistic regression, decision boundaries, and support vector machines.

Linear Models for Classification

Explores linear models for classification, including parametric models, regression, and logistic regression, along with model evaluation metrics and maximum margin classifiers.

Linear Models: Classification

Explores linear models for classification, including logistic regression, decision boundaries, and support vector machines.

Linear Classification Models: From Binary to Multiclass

Explores the extension of linear classifiers to handle multiclass problems and compares their performance on various datasets.

Related concepts (32)

Support vector machine

Multiclass classification

Duality (optimization)

Lagrange multiplier

Linear regression