Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Concept

Vanishing gradient problem

Related lectures (30)

Multilayer Perceptron: Training and Optimization

Explores the multilayer perceptron model, training, optimization, data preprocessing, activation functions, backpropagation, and regularization.

Recurrent Neural Networks: Neural Language Models

Explores fixed-context neural language models, RNNs, vanishing gradients, and sequence labeling in NLP.

Language Modelling and Recurrent Neural Networks

Explores language modelling, RNNs, n-gram models, LSTMs, and bidirectional RNNs.

Deep Learning: Backward Propagation and Vanishing Gradient

Delves into backward propagation in deep learning, addressing the vanishing gradient challenge and the need for effective hidden units.

Recurrent Neural Networks: LSTMs & GRUs

Explores LSTM and GRU variants in recurrent neural networks, addressing challenges and advantages.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Vanishing Gradient Problem: Deep Learning

Discusses the vanishing gradient problem in deep neural networks and its solutions.

Batch Normalization: Why It Works

Explores the aim and process of batch normalization in deep neural networks, emphasizing its importance in stabilizing mean input and solving the vanishing gradient problem.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

NFNets: Removing BatchNorm for High-Performance Image Recognition

Explores NFNets as an alternative to BatchNorm in ResNets, achieving high performance on ImageNet.

Weight Update: Mean Input and Bias Problem

Discusses the mean input shift and bias problem in weight updates for neural networks, highlighting the importance of correct initialization to prevent gradient issues.

Deep Learning Building Blocks

Covers the fundamental building blocks of deep learning, including tensors, backpropagation, and PyTorch.

Linear Systems: Convergence and Methods

Explores linear systems, convergence, and solving methods with a focus on CPU time and memory requirements.

Deep Learning for Autonomous Vehicles: Learning

Explores learning in deep learning for autonomous vehicles, covering predictive models, RNN, ImageNet, and transfer learning.

Learning with Deep Neural Networks

Explores the success and challenges of deep learning, including overfitting, generalization, and the impact on various domains.

Artificial Neural Networks and Deep Learning: Loss landscape and optimization methods

Explores error function landscape, optimization methods, and deep neural networks for classification.

Neural Networks: Multi-layers

Explains the learning process in multi-layer neural networks, including back-propagation, activation functions, weights update, and error backpropagation.

Convolutional Neural Networks: Structure & Regularization

Explores Convolutional Neural Networks, data augmentation, weight decay, and dropout for enhancing model performance.

Feed-forward Networks Log in to Mediaspace to watch this video

Introduces feed-forward networks, covering neural network structure, training, activation functions, and optimization, with applications in forecasting and finance.

Long Short-Term Memory Networks Log in to Mediaspace to watch this video

Introduces Long Short-Term Memory (LSTM) networks as a solution to vanishing and exploding gradients in recurrent neural networks.