Lecture

Feedforward Neural Networks: Activation Functions and Backpropagation

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Related lectures (64)

Neural Networks: Two Layers Neural Network

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Building Physical Neural Networks

Discusses challenges in building physical neural networks, focusing on depth, connections, and trainability.

Deep Learning: Data Representations and Neural Networks

Covers data representations, Bag of Words, histograms, data pre-processing, and neural networks.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Deep Splines: Unifying Framework for Deep Neural Networks

Introduces a functional framework for deep neural networks with adaptive piecewise-linear splines, focusing on biomedical image reconstruction and the challenges of deep splines.

Neural Networks

Explores neural networks, hidden layers, weight adjustments, activation functions, and the universal approximation theorem.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Dynamics of Linear Neural Networks

Explores the learning dynamics of deep neural networks using linear networks for analysis, covering two-layer and multi-layer networks, self-supervised learning, and benefits of decoupled initialization.

Neural Networks: Multi-layers

Explains the learning process in multi-layer neural networks, including back-propagation, activation functions, weights update, and error backpropagation.

Deep Learning: Data Representation and Multilayer Perceptron

Covers data representation, MLP training, activation functions, and gradient-based learning in deep neural networks.

Image Processing II: Neural Networks and CNNs

Explores the evolution of CNNs in image processing, covering classical and deep neural networks, training algorithms, backpropagation, non-linear steps, loss functions, and software frameworks.

Deep Learning for Autonomous Vehicles: Learning

Explores learning in deep learning for autonomous vehicles, covering predictive models, RNN, ImageNet, and transfer learning.

Splines and Imaging: From Compressed Sensing to Deep Neural Nets

Explores the optimality of splines for imaging and deep neural networks, demonstrating sparsity and global optimality with spline activations.

Introduction to Machine Learning: Supervised Learning

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Multi-layered Perceptron: History and Training Algorithm

Explores the historical development and training of multi-layered perceptrons, emphasizing the backpropagation algorithm and feature design.

Statistical Physics of Learning

Offers insights into the statistical physics of learning, exploring the relationship between neural network structure and disordered systems.

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.

MLPs: Multi-Layer Perceptrons

Introduces Multi-Layer Perceptrons (MLPs) and covers logistic regression, reformulation, gradient descent, AdaBoost, and practical applications.

Tricks of the Trade in Deep Learning: Aims

Covers practical questions and aims in deep learning, including neuron types, network architecture, optimization, and weight initialization.

Bio-Inspired Learning: Neural Networks, Genetic Algorithms

Explores bio-inspired learning with neural networks and genetic algorithms, covering structure, training, and practical applications.