Lecture

Word Embeddings: Pytorch Basics and Evaluation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture introduces the basics of Pytorch and word embeddings. It covers loading, manipulating, and evaluating pre-trained word embeddings. The lecture also delves into the evaluation of word embedding models through intrinsic and extrinsic tasks, including analogy-based benchmarking. Additionally, it explores the visualization and algebra of word embeddings, as well as the process of training new word embeddings. Practical exercises involve creating, normalizing, and aggregating word embeddings for downstream tasks.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Word embedding

In natural language processing (NLP), a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that words that are closer in the vector space are expected to be similar in meaning. Word embeddings can be obtained using language modeling and feature learning techniques, where words or phrases from the vocabulary are mapped to vectors of real numbers.

Cognitive evaluation theory

Cognitive evaluation theory (CET) is a theory in psychology that is designed to explain the effects of external consequences on internal motivation. Specifically, CET is a sub-theory of self-determination theory that focuses on competence and autonomy while examining how intrinsic motivation is affected by external forces in a process known as motivational "crowding out.

Overjustification effect

The overjustification effect occurs when an expected external incentive such as money or prizes decreases a person's intrinsic motivation to perform a task. Overjustification is an explanation for the phenomenon known as motivational "crowding out". The overall effect of offering a reward for a previously unrewarded activity is a shift to extrinsic motivation and the undermining of pre-existing intrinsic motivation.

Word2vec

Word2vec is a technique for natural language processing (NLP) published in 2013. The word2vec algorithm uses a neural network model to learn word associations from a large corpus of text. Once trained, such a model can detect synonymous words or suggest additional words for a partial sentence. As the name implies, word2vec represents each distinct word with a particular list of numbers called a vector.

Self-determination theory

Self-determination theory (SDT) is a macro theory of human motivation and personality that concerns people's innate growth tendencies and innate psychological needs. It pertains to the motivation behind people's choices in the absence of external influences and distractions. SDT focuses on the degree to which human behavior is self-motivated and self-determined. In the 1970s, research on SDT evolved from studies comparing intrinsic and extrinsic motives, and from growing understanding of the dominant role that intrinsic motivation played in individual behavior.

Related lectures (36)

Binary Sentiment Classifier Training

Covers the training of a binary sentiment classifier using an RNN.

Document Retrieval and Classification

Covers document retrieval, classification, sentiment analysis, and topic detection using TF-IDF matrices and contextualized word vectors like BERT.

Self-Determination Theory: Rewards and Intrinsic Motivation

Discusses the Self-Determination theory, rewards impact on motivation, and effective feedback.

Interaction and Biases in Learning

Discusses the impact of interaction, biases, and feedback on learning tools and educational settings.

Interest in Learning: Motivation and Goals