Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Variational Inference: Lower Bound and ELBO

In course

MGT-448: Statistical inference and machine learning

This course aims to provide graduate students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The course covers topics from machine learning, classical statistics, and data ...

Description

This lecture covers the concept of variational inference, focusing on the lower bound and Evidence Lower Bound (ELBO). It explains the Jensen's inequality, the E-step, and the M-step in the context of Mixure of Gaussians. The lecture also discusses the Markov chain Monte Carlo (MCMC) method for sampling from complex probability distributions.

Instructor

Negar Kiyavash

Official source

Ontological neighbourhood

Statistics

Statistical inference: Mathematical statistics

Related concepts (16)

Probability distribution

In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space). For instance, if X is used to denote the outcome of a coin toss ("the experiment"), then the probability distribution of X would take the value 0.5 (1 in 2 or 1/2) for X = heads, and 0.

Complex normal distribution

In probability theory, the family of complex normal distributions, denoted or , characterizes complex random variables whose real and imaginary parts are jointly normal. The complex normal family has three parameters: location parameter μ, covariance matrix , and the relation matrix . The standard complex normal is the univariate distribution with , , and . An important subclass of complex normal family is called the circularly-symmetric (central) complex normal and corresponds to the case of zero relation matrix and zero mean: and .

Jensen's inequality

In mathematics, Jensen's inequality, named after the Danish mathematician Johan Jensen, relates the value of a convex function of an integral to the integral of the convex function. It was proved by Jensen in 1906, building on an earlier proof of the same inequality for doubly-differentiable functions by Otto Hölder in 1889. Given its generality, the inequality appears in many forms depending on the context, some of which are presented below.

Cumulative distribution function

In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable , or just distribution function of , evaluated at , is the probability that will take a value less than or equal to . Every probability distribution supported on the real numbers, discrete or "mixed" as well as continuous, is uniquely identified by a right-continuous monotone increasing function (a càdlàg function) satisfying and .

Probability theory

Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of axioms. Typically these axioms formalise probability in terms of a probability space, which assigns a measure taking values between 0 and 1, termed the probability measure, to a set of outcomes called the sample space.

Related lectures (53)

Continuous Random Variables

Explores continuous random variables, density functions, joint variables, independence, and conditional densities.

Probability and Statistics

Covers p-quantile, normal approximation, joint distributions, and exponential families in probability and statistics.

Normal Distribution: Properties and Calculations

Covers properties and calculations related to the normal distribution, including probabilities and quantiles.

Linear Combinations: Moment-Generating Functions

Explores moment-generating functions, linear combinations, and normality of random variables.

Introduction to Continuous Random Variables: Probability Distributions