Lecture

Probabilistic Topic Models: Latent Dirichlet Allocation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

COM-308: Internet analytics

Internet analytics is the collection, modeling, and analysis of user data in large-scale online services, such as social networking, e-commerce, search, and advertisement. This class explores a number of the key functions of such online services that have ...

Description

This lecture covers probabilistic topic models, focusing on Latent Dirichlet Allocation (LDA). It explains how LDA models topics as distributions over words, with documents being mixtures of these topics. The instructor discusses the drawbacks of Probabilistic Latent Semantic Indexing (PLSI) and introduces LDA as a solution. The lecture delves into the Dirichlet distribution, which is used to model the distribution of distributions in LDA. It also explores the application of LDA in document clustering and its inference methods, such as MCMC and Variational Bayesian inference. Examples from the TREC-AP corpus are provided to illustrate LDA's effectiveness in soft document clustering. The lecture concludes by discussing LDA variants, extensions, and the general idea of topic models in document analysis.

Instructor

Matthias Grossglauser

Matthias Grossglauser is a Professor of Computer and Communication Sciences at EPFL in Lausanne, Switzerland, where he co-directs the Information and Network Dynamics lab. His current research interests center on machine learning, stochastic networks, and discrete choice models, and on their applications in artificial intelligence, reinforcement learning, network science, computational social sciences, and recommender systems. He is also a member of the Federal Communications Commission (ComCom), the indepe

Official source

Related concepts (26)

Dirichlet-multinomial distribution

In probability theory and statistics, the Dirichlet-multinomial distribution is a family of discrete multivariate probability distributions on a finite support of non-negative integers. It is also called the Dirichlet compound multinomial distribution (DCM) or multivariate Pólya distribution (after George Pólya). It is a compound probability distribution, where a probability vector p is drawn from a Dirichlet distribution with parameter vector , and an observation drawn from a multinomial distribution with probability vector p and number of trials n.

Dirichlet negative multinomial distribution

In probability theory and statistics, the Dirichlet negative multinomial distribution is a multivariate distribution on the non-negative integers. It is a multivariate extension of the beta negative binomial distribution. It is also a generalization of the negative multinomial distribution (NM(k, p)) allowing for heterogeneity or overdispersion to the probability vector. It is used in quantitative marketing research to flexibly model the number of household transactions across multiple brands.

Mixture model

In statistics, a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observation belongs. Formally a mixture model corresponds to the mixture distribution that represents the probability distribution of observations in the overall population.

Dirichlet process

In probability theory, Dirichlet processes (after the distribution associated with Peter Gustav Lejeune Dirichlet) are a family of stochastic processes whose realizations are probability distributions. In other words, a Dirichlet process is a probability distribution whose range is itself a set of probability distributions. It is often used in Bayesian inference to describe the prior knowledge about the distribution of random variables—how likely it is that the random variables are distributed according to one or another particular distribution.

Plate notation

In Bayesian inference, plate notation is a method of representing variables that repeat in a graphical model. Instead of drawing each repeated variable individually, a plate or rectangle is used to group variables into a subgraph that repeat together, and a number is drawn on the plate to represent the number of repetitions of the subgraph in the plate. The assumptions are that the subgraph is duplicated that many times, the variables in the subgraph are indexed by the repetition number, and any links that cross a plate boundary are replicated once for each subgraph repetition.

Related lectures (46)

Topic Models: Understanding Latent Structures

Explores topic models, Gaussian mixture models, Latent Dirichlet Allocation, and variational inference in understanding latent structures within data.

Topic Models

Introduces topic models, covering clustering, GMM, LDA, Dirichlet distribution, and variational inference.

Document Analysis and Topic Modeling

Covers document analysis, topic modeling, and deep generative models, including autoencoders and GANs.

Topic Models: Latent Dirichlet Allocation

Covers topic models, focusing on Latent Dirichlet Allocation, clustering, GMMs, Dirichlet distribution, LDA learning, and applications in digital humanities.

Dirichlet-Multinomial Model