Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Stochastic Simulation: Markov Chains and Transition Matrices

In course

MATH-414: Stochastic simulation

The student who follows this course will get acquainted with computational tools used to analyze systems with uncertainty arising in engineering, physics, chemistry, and economics. Focus will be on sampling methods as Monte Carlo, quasi Monte Carlo, Mark ...

Description

This lecture covers the concept of Markov chains and transition matrices in the context of stochastic simulation. It explains the difficulties in modeling high-dimensional spaces and provides examples from statistical physics. The instructor discusses the energy function, Bayesian statistics, and the Markov property. The lecture delves into the construction of Markov chains, the estimation of energy functions, and the properties of transition matrices. It also explores the invariant vectors of transition matrices and their significance in stochastic processes.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Fabio Nobile

Official source

Related concepts (28)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Markov model

In probability theory, a Markov model is a stochastic model used to model pseudo-randomly changing systems. It is assumed that future states depend only on the current state, not on the events that occurred before it (that is, it assumes the Markov property). Generally, this assumption enables reasoning and computation with the model that would otherwise be intractable. For this reason, in the fields of predictive modelling and probabilistic forecasting, it is desirable for a given model to exhibit the Markov property.

State function

In the thermodynamics of equilibrium, a state function, function of state, or point function for a thermodynamic system is a mathematical function relating several state variables or state quantities (that describe equilibrium states of a system) that depend only on the current equilibrium thermodynamic state of the system (e.g. gas, liquid, solid, crystal, or emulsion), not the path which the system has taken to reach that state. A state function describes equilibrium states of a system, thus also describing the type of system.

Helmholtz free energy

In thermodynamics, the Helmholtz free energy (or Helmholtz energy) is a thermodynamic potential that measures the useful work obtainable from a closed thermodynamic system at a constant temperature (isothermal). The change in the Helmholtz energy during a process is equal to the maximum amount of work that the system can perform in a thermodynamic process in which temperature is held constant. At constant temperature, the Helmholtz free energy is minimized at equilibrium.

Internal energy

The internal energy of a thermodynamic system is the energy contained within it, measured as the quantity of energy necessary to bring the system from its standard internal state to its present internal state of interest, accounting for the gains and losses of energy due to changes in its internal state, including such quantities as magnetization. It excludes the kinetic energy of motion of the system as a whole and the potential energy of position of the system as a whole, with respect to its surroundings and external force fields.

Related lectures (33)

Monte Carlo: Markov Chains

Covers unsupervised learning, dimensionality reduction, SVD, low-rank estimation, PCA, and Monte Carlo Markov Chains.

Markov Chains: Introduction and Properties

Covers the introduction and properties of Markov chains, including transition matrices and stochastic processes.

Hidden Markov Models: Primer

Introduces Hidden Markov Models, explaining the basic problems and algorithms like Forward-Backward, Viterbi, and Baum-Welch, with a focus on Expectation-Maximization.

Curie Weiss Model

Covers the Curie-Weiss model in Statistical Physics, including magnetization probability, free entropy, and the cavity method.

Markov Chains: Simulation and Optimization