Lectures related to Pretraining: Transformers & Models

Contextual Representations: ELMO and BERT Overview

Covers contextual representations in NLP, focusing on ELMO and BERT architectures and their applications in various tasks.

Language Models: From Theory to Computation

Explores the mathematics of language models, covering architecture design, pre-training, and fine-tuning, emphasizing the importance of pre-training and fine-tuning for various tasks.

Transformers: Pretraining and Decoding Techniques

Covers advanced transformer concepts, focusing on pretraining and decoding techniques in NLP.

Data Annotation: Collection and Biases in NLP

Addresses data collection, annotation processes, and biases in natural language processing.

Modern NLP: From GPT to ChatGPT

Explores the evolution of modern NLP from GPT-2 to GPT-3, emphasizing in-context learning and the development of ChatGPT.

Contextual Representations: ELMo & BERT

Explores the development of contextualized embeddings in NLP, focusing on ELMo and BERT's advancements and impact on NLP tasks.

Prompting and Alignment

Explores prompting, alignment, and the capabilities of large language models for natural language processing tasks.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Neural Word Embeddings: Learning Representations for Natural Language

Covers neural word embeddings and methods for learning word representations in natural language processing.

Binary Sentiment Classifier Training

Covers the training of a binary sentiment classifier using an RNN.

Language Models: Fixed-context and Recurrent Neural Networks

Discusses language models, focusing on fixed-context neural models and recurrent neural networks.

Decoding from Neural Models

Explores decoding from neural models in modern NLP, covering encoder-decoder models, decoding algorithms, issues with argmax decoding, and the impact of beam size.

Transformers: Overview and Self-Attention

Provides an overview of Transformers, self-attention, multi-headed attention, and the Transformer decoder and encoder.

Transformers: Full Architecture and Self-Attention Mechanism

Explains the full architecture of Transformers and the self-attention mechanism, highlighting the paradigm shift towards using completely pretrained models.

Transformers: Revolutionizing Attention Mechanisms in NLP

Covers the development of transformers and their impact on attention mechanisms in NLP.

Pre-Training: BiLSTM and Transformer Log in to Mediaspace to watch this video

Delves into pre-training BiLSTM and Transformer models for NLP tasks, showcasing their effectiveness and applications.

BERT: Pretraining and Applications Log in to Mediaspace to watch this video

Delves into BERT pretraining for transformers, discussing its applications in NLP tasks.

Biological structure and function emerge from unsupervised learning Log in to Mediaspace to watch this video

Delves into how biological structure and function are decoded through unsupervised learning of protein sequences.

Deep Learning for NLP Log in to Mediaspace to watch this video

Explores deep learning for NLP, covering word embeddings, context representations, learning techniques, and challenges like vanishing gradients and ethical considerations.

Modern NLP and Ethics in NLP Log in to Mediaspace to watch this video

Delves into advancements and challenges in NLP, along with ethical considerations and potential harms.