Lecture

Linear Discriminant Analysis: Generative Methods

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

EE-566: Adaptation and learning

In this course, students learn to design and master algorithms and core concepts related to inference and learning from data and the foundations of adaptation and learning theories with applications.

Description

This lecture covers Linear Discriminant Analysis (LDA) as a generative method for classification. It describes how LDA approximates the optimal Bayes classifier by estimating prior probabilities and assuming Gaussian distributions. The lecture also discusses discriminative models like Fisher discriminant analysis and logistic regression, highlighting supervised learning procedures.

Instructor

Ali H. Sayed

Ali H. Sayed is Dean of Engineering at EPFL, Switzerland, where he also leads the Adaptive Systems Laboratory. He has also served as Distinguished Professor and Chairman of Electrical Engineering at UCLA. He is recognized as a Highly Cited Researcher and is a member of the US National Academy of Engineering. He is also a member of the World Academy of Sciences and served as President of the IEEE Signal Processing Society during 2018 and 2019. Dr. Sayed is an author/co-author of over 570 scholarly publications and six books. His research involves several areas including adaptation and learning theories, data and network sciences, statistical inference, and multiagent systems. His work has been recognized with several major awards including the 2022 IEEE Fourier Award, the 2020 Norbert Wiener Society Award and the 2015 Education Award from the IEEE Signal Processing Society, the 2014 Papoulis Award from the European Association for Signal Processing, the 2013 Meritorious Service Award and the 2012 Technical Achievement Award from the IEEE Signal Processing Society, the 2005 Terman Award from the American Society for Engineering Education, the 2005 Distinguished Lecturer from the IEEE Signal Processing Society, the 2003 Kuwait Prize, and the 1996 IEEE Donald G. Fink Prize. His publications have been awarded several Best Paper Awards from the IEEE (2002, 2005, 2012, 2014) and EURASIP (2015). He is a Fellow of IEEE, EURASIP, and the American Association for the Advancement of Science (AAAS); the publisher of the journal Science.

Official source

Related concepts (32)

Linear discriminant analysis

Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

Linear classifier

In the field of machine learning, the goal of statistical classification is to use an object's characteristics to identify which class (or group) it belongs to. A linear classifier achieves this by making a classification decision based on the value of a linear combination of the characteristics. An object's characteristics are also known as feature values and are typically presented to the machine in a vector called a feature vector.

Generative model

In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsistent, but three major types can be distinguished, following : A generative model is a statistical model of the joint probability distribution on given observable variable X and target variable Y; A discriminative model is a model of the conditional probability of the target Y, given an observation x; and Classifiers computed without using a probability model are also referred to loosely as "discriminative".

Prior probability

A prior probability distribution of an uncertain quantity, often simply called the prior, is its assumed probability distribution before some evidence is taken into account. For example, the prior could be the probability distribution representing the relative proportions of voters who will vote for a particular politician in a future election. The unknown quantity may be a parameter of the model or a latent variable rather than an observable variable.

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Related lectures (64)

Logistic Regression

Covers logistic regression for linear classification and unsupervised dimensionality reduction techniques.

Gaussian Naive Bayes & K-NN

Covers Gaussian Naive Bayes, K-nearest neighbors, and hyperparameter tuning in machine learning.

Linear Regression: Basics and Gradient Descent

Covers the basics of linear regression, including feature engineering, supervised vs. unsupervised learning, and minimizing the cost function.

Classification Algorithms: Generative and Discriminative Approaches

Explores generative and discriminative classification algorithms, emphasizing their applications and differences in machine learning tasks.

Linear Models for Classification: Part 3