Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Deep and Convolutional Networks: Generalization and Optimization

In course

EE-311: Fundamentals of machine learning

Ce cours présente une vue générale des techniques d'apprentissage automatique, passant en revue les algorithmes, le formalisme théorique et les protocoles expérimentaux. ...

Description

This lecture covers the generalization of architectures in deep and convolutional networks, including the extension of multi-layered perceptrons. The instructor explains the concept of directed acyclic graphs of operators, forward spread, back-propagation of gradients, and the use of tensor operators for gradient descent optimization. The lecture also delves into the implementation of stochastic gradient descent, mini-batch processing, and the efficiency of parameter optimization in deep learning. Auto-encoders, interpolation in latent space, and adversarial models are discussed, along with visualization techniques to understand network representations. Examples and practical applications are provided to illustrate the concepts.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructors (2)

Michael Stefan Daniel Liebling

François Fleuret

Official source

Ontological neighbourhood

Information engineering

Machine learning: Artificial neural networks

Related concepts (32)

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Residual neural network

A Residual Neural Network (a.k.a. Residual Network, ResNet) is a deep learning model in which the weight layers learn residual functions with reference to the layer inputs. A Residual Network is a network with skip connections that perform identity mappings, merged with the layer outputs by addition. It behaves like a Highway Network whose gates are opened through strongly positive bias weights. This enables deep learning models with tens or hundreds of layers to train easily and approach better accuracy when going deeper.

Online machine learning

In computer science, online machine learning is a method of machine learning in which data becomes available in a sequential order and is used to update the best predictor for future data at each step, as opposed to batch learning techniques which generate the best predictor by learning on the entire training data set at once. Online learning is a common technique used in areas of machine learning where it is computationally infeasible to train over the entire dataset, requiring the need of out-of-core algorithms.

Related lectures (39)

Deep Learning Building Blocks

Covers tensors, loss functions, autograd, and convolutional layers in deep learning.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Deep Learning: Convolutional Neural Networks

Covers Convolutional Neural Networks, standard architectures, training techniques, and adversarial examples in deep learning.

Pavement Distress Detection

Covers the importance of preventive maintenance for pavement distress detection and introduces machine learning concepts for engineers.

Neural Networks: Multilayer Learning