Lecture

Linear Classification: Signed Distance and Perceptron

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-233(b): Introduction to machine learning (BA4)

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed and practically implemented. ...

Description

This lecture covers the concept of signed distance in N dimensions for linear classification, the problem statement, and the non-centered perceptron algorithm. It also discusses test time evaluation, Python and JAVA implementations, the problem with the perceptron, logistic regression, sigmoid function, cross entropy, and probabilistic interpretation. The transition from binary to multi-class classification, linear discriminant, decision regions, and multi-class linear classification are explained. The lecture concludes with multi-class logistic regression, cross entropy, activation, and the results of multi-class classification.

Instructor

Pascal Fua

Pascal Fua received an engineering degree from Ecole Polytechnique, Paris, in 1984 and the Ph.D. degree in Computer Science from the University of Orsay in 1989. He then worked at SRI International and INRIA Sophia-Antipolis as a Computer Scientist. He joined EPFL in 1996 where he is now a Professor in the School of Computer and Communication Science and heads the Computer Vision Laboratory. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and Augmented Reality. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and machine learning. He has (co)authored over 300 publications in refereed journals and conferences. He is an IEEE Fellow and has been an Associate Editor of IEEE journal Transactions for Pattern Analysis and Machine Intelligence. He often serves as program committee member, area chair, and program chair of major vision conferences and has cofounded three spinoff companies (Pix4D, PlayfulVision, and NeuralConcept).

Official source

Related concepts (32)

Linear discriminant analysis

Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

Linear classifier

In the field of machine learning, the goal of statistical classification is to use an object's characteristics to identify which class (or group) it belongs to. A linear classifier achieves this by making a classification decision based on the value of a linear combination of the characteristics. An object's characteristics are also known as feature values and are typically presented to the machine in a vector called a feature vector.

Logistic function

A logistic function or logistic curve is a common S-shaped curve (sigmoid curve) with the equation where For values of in the domain of real numbers from to , the S-curve shown on the right is obtained, with the graph of approaching as approaches and approaching zero as approaches . The logistic function finds applications in a range of fields, including biology (especially ecology), biomathematics, chemistry, demography, economics, geoscience, mathematical psychology, probability, sociology, political science, linguistics, statistics, and artificial neural networks.

Perceptron

In machine learning, the perceptron (or McCulloch-Pitts neuron) is an algorithm for supervised learning of binary classifiers. A binary classifier is a function which can decide whether or not an input, represented by a vector of numbers, belongs to some specific class. It is a type of linear classifier, i.e. a classification algorithm that makes its predictions based on a linear predictor function combining a set of weights with the feature vector.

Free Java implementations

Free Java implementations are software projects that implement Oracle's Java technologies and are distributed under free software licences, making them free software. Sun released most of its Java source code as free software in May 2007, so it can now almost be considered a free Java implementation. Java implementations include compilers, runtimes, class libraries, etc. Advocates of free and open source software refer to free or open source Java virtual machine software as free runtimes or free Java runtimes.

Related lectures (64)

Linear Models for Classification: Multi-Class Extensions

Covers linear models for multi-class classification, focusing on logistic regression and evaluation metrics.

Linear Models for Classification: Part 3

Explores linear models for classification, including binary classification, logistic regression, decision boundaries, and support vector machines.

Linear Models: Classification Basics

Explores linear models for classification, logistic regression, SVM, k-NN, and curse of dimensionality.

Linear Binary Classification

Covers the extension of the 0-1 loss to real-valued score functions and logistic regression.

Multiclass Classification