Lecture

Linear Models for Classification

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-233(a): Introduction to machine learning (BA3)

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed and practically implemented. ...

Description

This lecture covers linear models for classification, starting with a recap on simple parametric models and hyperplanes. It delves into linear regression, binary classification, adding non-linearity, logistic regression, and evaluating classifiers. The instructor explains the concept of margin between classes, the formulation for a maximum margin classifier (SVM), and basic notions of constrained optimization. The lecture also touches on decision boundaries, support vectors, and the SVM formulation. An interlude on constrained optimization and Lagrange duality is included, followed by the derivation of the Lagrange dual problem. The use of slack variables in SVM is discussed, along with the Lagrangian and the optimal values. The lecture concludes with a comparison of linear models for classification.

Instructor

Mathieu Salzmann

I am a Senior Researcher at EPFL-CVLab, and, since May 2020, an Artificial Intelligence Engineer at ClearSpace (50%). Previously, I was a Senior Researcher and Research Leader in NICTA's computer vision research group. Prior to this, from Sept. 2010 to Jan 2012, I was a Research Assistant Professor at TTI-Chicago, and, from Feb. 2009 to Aug. 2010, a postdoctoral fellow at ICSI and EECS at UC Berkeley under the supervision of Prof. Trevor Darrell. I obtained my PhD in Jan. 2009 from EPFL under the supervision of Prof. Pascal Fua.

Official source

Ontological neighbourhood

Statistics

Data analysis: Regression analysis

Information engineering

Machine learning: Topics in machine learning

Theoretical computer science

Algorithms and data structures: Combinatorial optimization

Related concepts (32)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Multiclass classification

In machine learning and statistical classification, multiclass classification or multinomial classification is the problem of classifying instances into one of three or more classes (classifying instances into one of two classes is called binary classification). While many classification algorithms (notably multinomial logistic regression) naturally permit the use of more than two classes, some are by nature binary algorithms; these can, however, be turned into multinomial classifiers by a variety of strategies.

Decision boundary

NOTOC In a statistical-classification problem with two classes, a decision boundary or decision surface is a hypersurface that partitions the underlying vector space into two sets, one for each class. The classifier will classify all the points on one side of the decision boundary as belonging to one class and all those on the other side as belonging to the other class. A decision boundary is the region of a problem space in which the output label of a classifier is ambiguous.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Nonlinear regression

In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations. In nonlinear regression, a statistical model of the form, relates a vector of independent variables, , and its associated observed dependent variables, . The function is nonlinear in the components of the vector of parameters , but otherwise arbitrary.

Related lectures (64)

Linear Models for Classification

Explores linear models, logistic regression, classification metrics, SVM, and their practical use in data science methods.

Linear Models for Classification

Explores linear models for classification, including parametric models, regression, and logistic regression, along with model evaluation metrics and maximum margin classifiers.

Linear Models & k-NN

Covers linear models, logistic regression, decision boundaries, k-NN, and practical applications in authorship attribution and image data analysis.

Max-Margin Classifiers

Explores maximizing margins for better classification using support vector machines and the importance of choosing the right parameter.

Linear Regression and Logistic Regression

Covers linear and logistic regression for regression and classification tasks, focusing on loss functions and model training.

Related concepts (32)

Linear regression

Multiclass classification

Decision boundary

Logistic regression

Nonlinear regression