Lecture

Convolutional Networks: Overview and Architecture

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practised. ...

Description

This lecture introduces the motivation and idea behind convolutional networks, explaining the concept of fully connected layers, post-activation, and pooling layers. It covers the evolution from LeNet to AlexNet, detailing the architecture, activation functions, and pooling techniques used in modern convolutional networks.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Lenka Zdeborová

Lenka Zdeborová is a Professor of Physics and of Computer Science in École Polytechnique Fédérale de Lausanne where she leads the Statistical Physics of Computation Laboratory. She received a PhD in physics from University Paris-Sud and from Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director's Postdoctoral Fellow. Between 2010 and 2020 she was a researcher at CNRS working in the Institute of Theoretical Physics in CEA Saclay, France. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize, in 2021 the Gibbs lectureship of AMS. She is an editorial board member for Journal of Physics A, Physical Review E, Physical Review X, SIMODS, Machine Learning: Science and Technology, and Information and Inference. Lenka's expertise is in applications of concepts from statistical physics, such as advanced mean field methods, replica method and related message-passing algorithms, to problems in machine learning, signal processing, inference and optimization. She enjoys erasing the boundaries between theoretical physics, mathematics and computer science.

Official source

Ontological neighbourhood

Information engineering

Machine learning: Artificial neural networks

Related concepts (32)

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

Activation function

Activation function of a node in an artificial neural network is a function that calculates the output of the node (based on its inputs and the weights on individual inputs). Nontrivial problems can be solved only using a nonlinear activation function. Modern activation functions include the smooth version of the ReLU, the GELU, which was used in the 2018 BERT model, the logistic (sigmoid) function used in the 2012 speech recognition model developed by Hinton et al, the ReLU used in the 2012 AlexNet computer vision model and in the 2015 ResNet model.

Sodium channel

Sodium channels are integral membrane proteins that form ion channels, conducting sodium ions (Na+) through a cell's membrane. They belong to the superfamily of cation channels. They are classified into 2 types: In excitable cells such as neurons, myocytes, and certain types of glia, sodium channels are responsible for the rising phase of action potentials. These channels go through three different states called resting, active and inactive states.

Potassium channel

Potassium channels are the most widely distributed type of ion channel found in virtually all organisms. They form potassium-selective pores that span cell membranes. Potassium channels are found in most cell types and control a wide variety of cell functions. Potassium channels function to conduct potassium ions down their electrochemical gradient, doing so both rapidly (up to the diffusion rate of K+ ions in bulk water) and selectively (excluding, most notably, sodium despite the sub-angstrom difference in ionic radius).

Feedforward neural network

A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes (if any) and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow.

Related lectures (64)

Deep Learning

Covers the fundamentals of deep learning, including data representations, bag of words, data pre-processing, artificial neural networks, and convolutional neural networks.

Neural Networks: Multilayer Learning

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.

Convolutional Neural Networks: Fundamentals

Covers the basics of Convolutional Neural Networks, including training optimization, layer structure, and potential pitfalls of summary statistics.

Deep Learning: Convolutional Neural Networks

Covers Convolutional Neural Networks, standard architectures, training techniques, and adversarial examples in deep learning.

Neural Networks: Training and Activation