Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Text Models: Word Embeddings and Topic Models

In course

COM-308: Internet analytics

Internet analytics is the collection, modeling, and analysis of user data in large-scale online services, such as social networking, e-commerce, search, and advertisement. This class explores a number of the key functions of such online services that have ...

Description

This lecture covers probabilistic models for text, including word embeddings to capture word semantics, topic models for document sets, and an introduction to graphical models. It explains how word embeddings find relationships among words based on context, and how topic models classify document meanings without a query. The lecture also delves into the skip-gram model of Word2vec, negative sampling, and properties of word vectors. Additionally, it explores Bayesian Networks, their parameterization, and inference methods like Gibbs sampling. The lecture concludes with a discussion on the computational challenges in large models and the key ideas behind Bayesian Networks.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Matthias Grossglauser

Official source

Ontological neighbourhood

Information engineering

Natural language processing: Topics in natural language processing

Related concepts (32)

Word2vec

Word2vec is a technique for natural language processing (NLP) published in 2013. The word2vec algorithm uses a neural network model to learn word associations from a large corpus of text. Once trained, such a model can detect synonymous words or suggest additional words for a partial sentence. As the name implies, word2vec represents each distinct word with a particular list of numbers called a vector.

Word embedding

In natural language processing (NLP), a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that words that are closer in the vector space are expected to be similar in meaning. Word embeddings can be obtained using language modeling and feature learning techniques, where words or phrases from the vocabulary are mapped to vectors of real numbers.

Bayesian network

A Bayesian network (also known as a Bayes network, Bayes net, belief network, or decision network) is a probabilistic graphical model that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG). It is one of several forms of causal notation. Bayesian networks are ideal for taking an event that occurred and predicting the likelihood that any one of several possible known causes was the contributing factor. For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms.

Latent semantic analysis

Latent semantic analysis (LSA) is a technique in natural language processing, in particular distributional semantics, of analyzing relationships between a set of documents and the terms they contain by producing a set of concepts related to the documents and terms. LSA assumes that words that are close in meaning will occur in similar pieces of text (the distributional hypothesis).

Probabilistic latent semantic analysis

Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data. In effect, one can derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables, just as in latent semantic analysis, from which PLSA evolved.

Related lectures (31)

Handling Text: Document Retrieval, Classification, Sentiment Analysis

Explores document retrieval, classification, sentiment analysis, TF-IDF matrices, nearest-neighbor methods, matrix factorization, regularization, LDA, contextualized word vectors, and BERT.

Vector Space Semantics (and Information Retrieval)

Explores the Vector Space model, Bag of Words, tf-idf, cosine similarity, Okapi BM25, and Precision and Recall in Information Retrieval.

Word Embeddings: Models and Applications

Explores word embeddings, models like CBOW and Skipgram, Fasttext, Glove, subword embeddings, and their applications in document search and classification.

Text Processing: Large Digital Text Collections Analysis

Delves into the processing of large digital text collections, exploring hidden regularities, text reuse, and TF-IDF analysis.

Document Retrieval and Classification