Lecture

Deep Neural Networks and Splines

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MICRO-512: Image processing II

Study of advanced image processing; mathematical imaging. Development of image-processing software and prototyping in Jupyter Notebooks; application to real-world examples in industrial vision and biomedical imaging. ...

Description

This lecture covers the fundamentals of deep neural networks and splines, starting with feedforward deep neural networks and the ReLU activation function. It then delves into continuous-piecewise linear functions in one and multiple dimensions, discussing their algebraic properties and implications for deep ReLU neural networks. The lecture also explores the universal approximation properties of CPWL functions and their implementation via deep ReLU networks. Additionally, it examines the refinement of activation functions, constraining activation functions, the representer theorem for deep neural networks, and the outcome of this theorem. The lecture concludes with a comparison of linear interpolators, the discussion on deep spline networks, their opportunities, challenges, and their connection with existing schemes.

Instructors (4)

Dimitri Nestor Alice Van De Ville

Expertise Brain and spinal cord imaging, signal and image processing, network science, dynamic systems Teaching & PhD PhD Students Alexandre Cionca, Chun Hei Michael Chan, Flavia Petruso, Bassam El Rawas, Nikolaos Molochidis, Christophe Pascal Jérémie Chênes, Fabrice Jean Guibert, Alessio Siviglia, Ekansh Sareen, Beatrice Lugli Past EPFL PhD Students Djano Kandaswamy, Fikret Isik Karahanoglu, Nora Leonardi, Zafer Dogan, Jeffrey Adam Kasten, Naghmeh Ghazaleh, Rotem Roza Kopel, Daniela Maria Zöller, Thomas

Michael Stefan Daniel Liebling

Teaching & PhD Past EPFL PhD Students as codirector Christian Jaques, Olivia Mariani, Adrian Shajkofci, François Thierry M Marelli Courses Fundamentals of machine learning EE-311 This course provides a general overview of machine learning, covering the main algorithms, theoretical formalisms, and experimental protocols.

Michaël Unser

Expertise Image Processing, Medical Imaging, Biological Imaging, Wavelets, Splines, Multiresolution Michael Unser is Full Professor with EPFL’s school of Engineering and the academic director of EPFL's Center for Imaging, Lausanne, Switzerland. His primary areas of investigation are signal processing, biomedical imaging and applied functional analysis. He is internationally recognized for his research contributions to sampling theory, wavelets, the use of splines for image processing, stochastic processes,

Daniel Sage

Expertise PublicationsPublications Coordinates Biomedical Imaging GroupAddressEPFL STI-IMT-LIB, BM 4.135, Station 17, CH-1015 Lausanne, SwitzerlandTel.+41 21 693 51 89Fax.+41 21 693 68 10Emaildaniel.sage@epfl.chWeb pagehttp://bigwww.epfl.ch/sage/OfficeBM 4.135 Teaching & PhD Past EPFL PhD Students as codirector Daniel Andreas Schmitter, Anaïs Badoual, Laurène Donati Courses Bioimage informatics BIO-410 The course provides a comprehensive overview of methods, algorithms, and computer tools us

Official source

Ontological neighbourhood

Information engineering

Machine learning: Artificial neural networks

Related concepts (32)

Spline interpolation

In the mathematical field of numerical analysis, spline interpolation is a form of interpolation where the interpolant is a special type of piecewise polynomial called a spline. That is, instead of fitting a single, high-degree polynomial to all of the values at once, spline interpolation fits low-degree polynomials to small subsets of the values, for example, fitting nine cubic polynomials between each of the pairs of ten points, instead of fitting a single degree-ten polynomial to all of them.

Interpolation

In the mathematical field of numerical analysis, interpolation is a type of estimation, a method of constructing (finding) new data points based on the range of a discrete set of known data points. In engineering and science, one often has a number of data points, obtained by sampling or experimentation, which represent the values of a function for a limited number of values of the independent variable. It is often required to interpolate; that is, estimate the value of that function for an intermediate value of the independent variable.

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Types of artificial neural networks

There are many types of artificial neural networks (ANN). Artificial neural networks are computational models inspired by biological neural networks, and are used to approximate functions that are generally unknown. Particularly, they are inspired by the behaviour of neurons and the electrical signals they convey between input (such as from the eyes or nerve endings in the hand), processing, and output from the brain (such as reacting to light, touch, or heat). The way neurons semantically communicate is an area of ongoing research.

Related lectures (64)

Neural Networks: Two Layers Neural Network

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Splines and Machine Learning

Explores supervised learning as an ill-posed problem and the integration of sparse adaptive splines into neural architectures.

Deep Splines: Unifying Framework for Deep Neural Networks

Introduces a functional framework for deep neural networks with adaptive piecewise-linear splines, focusing on biomedical image reconstruction and the challenges of deep splines.

Deep Neural Networks: Optimization and Approximation

Explores optimization and approximation in deep neural networks, including optimal control and numerical experiments.

Splines and Imaging: From Compressed Sensing to Deep Neural Nets