Concept

Transformer (machine learning model)

Related lectures (32)

Transformers: Overview and Self-Attention

Provides an overview of Transformers, self-attention, multi-headed attention, and the Transformer decoder and encoder.

Language Models: From Theory to Computation

Explores the mathematics of language models, covering architecture design, pre-training, and fine-tuning, emphasizing the importance of pre-training and fine-tuning for various tasks.

Graph-to-Graph Transformers: Syntax-aware Graph Encoding

Introduces the Syntax-aware Graph-to-Graph Transformer architecture for effective conditioning on syntactic dependency graphs.

Multi-layered Perceptron: History and Training Algorithm

Explores the historical development and training of multi-layered perceptrons, emphasizing the backpropagation algorithm and feature design.

Contextual Representations: ELMo & BERT

Explores the development of contextualized embeddings in NLP, focusing on ELMo and BERT's advancements and impact on NLP tasks.

Transformers: Full Architecture and Self-Attention Mechanism

Explains the full architecture of Transformers and the self-attention mechanism, highlighting the paradigm shift towards using completely pretrained models.

Transformer Architecture: The X Gomega

Delves into the Transformer architecture, self-attention, and training strategies for machine translation and image recognition.

Contextual Representations: ELMO and BERT Overview

Covers contextual representations in NLP, focusing on ELMO and BERT architectures and their applications in various tasks.

Transformer Networks: Self-Attention

Explains Transformer networks and self-attention layers for mapping inputs and multi-head attention.

Decoding from Neural Models

Explores decoding from neural models in modern NLP, covering encoder-decoder models, decoding algorithms, issues with argmax decoding, and the impact of beam size.

Pretraining Sequence-to-Sequence Models: BART + T5

Explores pretraining sequence-to-sequence models with BART and T5, discussing transfer learning, fine-tuning, model architectures, tasks, performance comparison, summarization results, and references.

Transformers: Pretraining and Decoding Techniques

Covers advanced transformer concepts, focusing on pretraining and decoding techniques in NLP.

Pretraining Sequence-to-Sequence Models: BART and T5

Covers the pretraining of sequence-to-sequence models, focusing on BART and T5 architectures.

Chemical Reaction Prediction: Molecular Transformer

Explores chemical reaction prediction using generative models and molecular transformers, emphasizing the importance of molecular language processing and stereochemistry.

Trajectory Forecasting in Autonomous Vehicles

Explores trajectory forecasting in autonomous vehicles, focusing on deep learning models for predicting human trajectories in socially-aware transportation scenarios.

Model Finetuning for NLI

Covers the second assignment for the CS-552: Modern NLP course, focusing on transfer learning and data augmentation.

Chemical Reactions: Transformer Architecture

Explores atom mapping in chemical reactions and the transition to reaction grammar using the transformer architecture.

Neural Networks: Multilayer Perceptron

Explores the history, models, training, convergence, and limitations of neural networks, including the backpropagation algorithm and universal approximation.

Neural Networks: Two Layers Neural Network

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Transformer Architecture: Subquadratic Attention Mechanisms

Covers transformer architecture, focusing on encoder-decoder models and subquadratic attention mechanisms for efficient processing of input sequences.

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.