Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Probabilistic Topic Models: Latent Dirichlet Allocation

In course

COM-308: Internet analytics

Internet analytics is the collection, modeling, and analysis of user data in large-scale online services, such as social networking, e-commerce, search, and advertisement. This class explores a number of the key functions of such online services that have ...

Description

This lecture covers probabilistic topic models, focusing on Latent Dirichlet Allocation (LDA). It explains how LDA models topics as distributions over words, with documents being mixtures of these topics. The instructor discusses the drawbacks of Probabilistic Latent Semantic Indexing (PLSI) and introduces LDA as a solution. The lecture delves into the Dirichlet distribution, which is used to model the distribution of distributions in LDA. It also explores the application of LDA in document clustering and its inference methods, such as MCMC and Variational Bayesian inference. Examples from the TREC-AP corpus are provided to illustrate LDA's effectiveness in soft document clustering. The lecture concludes by discussing LDA variants, extensions, and the general idea of topic models in document analysis.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Matthias Grossglauser

Official source

Related concepts (23)

Latent Dirichlet allocation

In natural language processing, Latent Dirichlet Allocation (LDA) is a Bayesian network (and, therefore, a generative statistical model) that explains a set of observations through unobserved groups, and each group explains why some parts of the data are similar. The LDA is an example of a Bayesian topic model. In this, observations (e.g., words) are collected into documents, and each word's presence is attributable to one of the document's topics. Each document will contain a small number of topics.

Topic model

In statistics and natural language processing, a topic model is a type of statistical model for discovering the abstract "topics" that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden semantic structures in a text body. Intuitively, given that a document is about a particular topic, one would expect particular words to appear in the document more or less frequently: "dog" and "bone" will appear more often in documents about dogs, "cat" and "meow" will appear in documents about cats, and "the" and "is" will appear approximately equally in both.

Dirichlet distribution

In probability and statistics, the Dirichlet distribution (after Peter Gustav Lejeune Dirichlet), often denoted , is a family of continuous multivariate probability distributions parameterized by a vector of positive reals. It is a multivariate generalization of the beta distribution, hence its alternative name of multivariate beta distribution (MBD). Dirichlet distributions are commonly used as prior distributions in Bayesian statistics, and in fact, the Dirichlet distribution is the conjugate prior of the categorical distribution and multinomial distribution.

Dirichlet-multinomial distribution

In probability theory and statistics, the Dirichlet-multinomial distribution is a family of discrete multivariate probability distributions on a finite support of non-negative integers. It is also called the Dirichlet compound multinomial distribution (DCM) or multivariate Pólya distribution (after George Pólya). It is a compound probability distribution, where a probability vector p is drawn from a Dirichlet distribution with parameter vector , and an observation drawn from a multinomial distribution with probability vector p and number of trials n.

Probabilistic latent semantic analysis

Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data. In effect, one can derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables, just as in latent semantic analysis, from which PLSA evolved.

Related lectures (32)

Dirichlet-Multinomial Model

Discusses the Dirichlet distribution, Bayesian inference, posterior mean and variance, conjugate priors, and predictive distribution in the Dirichlet-Multinomial model.

Common Distributions: Moment Generating Functions

Explores common probability distributions, special distributions, and entropy concepts.

Review of Probability

Covers the review of probability concepts including Poisson distribution and moment generating functions.

Embedding Models: Concepts and Retrieval

Covers embedding models for document retrieval, latent semantic indexing, SVD, and topic models.

Probability and Statistics