Lecture

Deep Learning: Backward Propagation and Vanishing Gradient

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Recurrent Neural Networks: Neural Language Models

Explores fixed-context neural language models, RNNs, vanishing gradients, and sequence labeling in NLP.

Splines and Machine Learning

Explores supervised learning as an ill-posed problem and the integration of sparse adaptive splines into neural architectures.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Mathematics of Data: Deep Learning Introduction

Covers a mathematical introduction to deep learning, including challenges, power of linear classifiers, model scaling, and theoretical aspects.

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.

Deep Learning for Question Answering

Explores deep learning for question answering, analyzing neural networks and model robustness to noise.

Neural Networks: Multilayer Learning

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.

Neural Networks: Perceptron

Covers the main concepts of neural networks, including the Perceptron model and training algorithms.

Image Processing II: Neural Networks and CNNs

Explores the evolution of CNNs in image processing, covering classical and deep neural networks, training algorithms, backpropagation, non-linear steps, loss functions, and software frameworks.

Weight Update: Mean Input and Bias Problem

Discusses the mean input shift and bias problem in weight updates for neural networks, highlighting the importance of correct initialization to prevent gradient issues.

Mathematics of Data: From Theory to Computation

Covers key concepts in data mathematics, including automatic differentiation, linear layers, and attention layers.

Bagging: Regularization Method in Deep Learning

Explores bagging as a regularization method in deep learning, training multiple model variants on different data subsets to improve generalization.

Neural Networks: Multi-layers

Explains the learning process in multi-layer neural networks, including back-propagation, activation functions, weights update, and error backpropagation.

Deep Knowledge Tracing

Explores Deep Knowledge Tracing and its application in predicting student learning outcomes using neural networks and loss functions.

Multilayer Perceptron: Training and Optimization

Explores the multilayer perceptron model, training, optimization, data preprocessing, activation functions, backpropagation, and regularization.

Medical Image Reconstruction: From Model-Based to Data-Driven

Explores the evolution of medical image reconstruction techniques, from classical methods to data-driven approaches using deep neural networks.

Deep Learning: Convolutional Networks

Explores convolutional neural networks, backpropagation, and stochastic gradient descent in deep learning.

Deep and Robust Reinforcement Learning Techniques

Discusses advanced reinforcement learning techniques, focusing on deep and robust methods, including actor-critic frameworks and adversarial learning strategies.

Deep Learning Building Blocks

Covers the fundamental building blocks of deep learning, including tensors, backpropagation, and PyTorch.

Provable and Generalizable Robustness in Deep Learning

Explores adversarial examples, defenses, and certifiable robustness in deep learning, including Gaussian smoothing and perceptual attacks.