Lecture

Recurrent Neural Networks: Training and Challenges

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Deep Learning Building Blocks

Covers the fundamental building blocks of deep learning, including tensors, backpropagation, and PyTorch.

Building Physical Neural Networks

Discusses challenges in building physical neural networks, focusing on depth, connections, and trainability.

Multi-layered Perceptron: History and Training Algorithm

Explores the historical development and training of multi-layered perceptrons, emphasizing the backpropagation algorithm and feature design.

Deep Learning for Autonomous Vehicles: Learning

Explores learning in deep learning for autonomous vehicles, covering predictive models, RNN, ImageNet, and transfer learning.

Deep Learning: Recurrent Neural Networks

Explores Recurrent Neural Networks for behavioral data, covering Deep Knowledge Tracing, LSTM, GRU networks, hyperparameter tuning, and time series prediction tasks.

Regularized Cross-Entropy Risk

Explores the regularized cross-entropy risk in neural networks, covering training processes and challenges in deep networks.

Deep Neural Networks: Training and Optimization

Explores deep neural network training, optimization, preventing overfitting, and different network architectures.

Deep Generative Models: Part 2

Explores deep generative models, including mixtures of multinomials, PCA, deep autoencoders, convolutional autoencoders, and GANs.

Neural Networks: Multi-layers

Explains the learning process in multi-layer neural networks, including back-propagation, activation functions, weights update, and error backpropagation.

Neural Networks: Training and Optimization

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.

Neural Networks: Multilayer Perceptrons

Covers Multilayer Perceptrons, artificial neurons, activation functions, matrix notation, flexibility, regularization, regression, and classification tasks.

Transformers: Revolutionizing Attention Mechanisms in NLP

Covers the development of transformers and their impact on attention mechanisms in NLP.

Foundations of Deep Learning: Transformer Architecture Overview

Covers the foundational concepts of deep learning and the Transformer architecture, focusing on neural networks, attention mechanisms, and their applications in sequence modeling tasks.

Deep Learning: Backward Propagation and Vanishing Gradient

Delves into backward propagation in deep learning, addressing the vanishing gradient challenge and the need for effective hidden units.

Tricks of the Trade in Deep Learning: Aims

Covers practical questions and aims in deep learning, including neuron types, network architecture, optimization, and weight initialization.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Splines and Imaging: From Compressed Sensing to Deep Neural Nets

Explores the optimality of splines for imaging and deep neural networks, demonstrating sparsity and global optimality with spline activations.

Transformers: Pretraining and Decoding Techniques

Covers advanced transformer concepts, focusing on pretraining and decoding techniques in NLP.

Non-Conceptual Knowledge Systems

Delves into the impact of deep learning on non-conceptual knowledge systems and the advancements in transformers and generative adversarial networks.

Recurrent Neural Networks: LSTMs & GRUs

Explores LSTM and GRU variants in recurrent neural networks, addressing challenges and advantages.