Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Diffusion Models in Generative Modeling

Description

This lecture delves into diffusion models in generative modeling, starting with an overview of generative models and diving into the intuition and mathematics behind diffusion models. The instructor explains how these models estimate probability distributions of data, generate new data points, and evaluate probabilities. The lecture covers the challenges of modeling complex high-dimensional distributions, the use of deep neural networks, and the concept of denoising diffusion models. It also explores the transition to continuous time and the benefits of using stochastic differential equations. The discussion extends to controlling generation, evaluating probabilities, and the potential of diffusion models in various applications.

Official source

Ontological neighbourhood

Information engineering

Machine learning: Artificial neural networks

Related concepts (32)

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Types of artificial neural networks

There are many types of artificial neural networks (ANN). Artificial neural networks are computational models inspired by biological neural networks, and are used to approximate functions that are generally unknown. Particularly, they are inspired by the behaviour of neurons and the electrical signals they convey between input (such as from the eyes or nerve endings in the hand), processing, and output from the brain (such as reacting to light, touch, or heat). The way neurons semantically communicate is an area of ongoing research.

Stochastic differential equation

A stochastic differential equation (SDE) is a differential equation in which one or more of the terms is a stochastic process, resulting in a solution which is also a stochastic process. SDEs have many applications throughout pure mathematics and are used to model various behaviours of stochastic models such as stock prices, random growth models or physical systems that are subjected to thermal fluctuations. SDEs have a random differential that is in the most basic case random white noise calculated as the derivative of a Brownian motion or more generally a semimartingale.

Probability distribution

In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space). For instance, if X is used to denote the outcome of a coin toss ("the experiment"), then the probability distribution of X would take the value 0.5 (1 in 2 or 1/2) for X = heads, and 0.

Related lectures (43)

Language Models: Fixed-context and Recurrent Neural Networks

Discusses language models, focusing on fixed-context neural models and recurrent neural networks.

Score-based Generative Modelling

Explores score-based generative modelling through stochastic differential equations, emphasizing denoising score matching and diffusion probabilistic models.

Natural Language Generation: Decoding Techniques and Training Challenges

Covers decoding methods and training challenges in natural language generation.

Diffusion Models and Robustness

Explores diffusion models, GAN training challenges, SDE-based generative models, and robustness in deep learning.

Generative Neural Networks: Sampling and Training