Lecture

Stochastic Simulation: Markov Chains and Transition Matrices

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-414: Stochastic simulation

The student who follows this course will get acquainted with computational tools used to analyze systems with uncertainty arising in engineering, physics, chemistry, and economics. Focus will be on sampling methods as Monte Carlo, quasi Monte Carlo, Mark ...

Description

This lecture covers the concept of Markov chains and transition matrices in the context of stochastic simulation. It explains the difficulties in modeling high-dimensional spaces and provides examples from statistical physics. The instructor discusses the energy function, Bayesian statistics, and the Markov property. The lecture delves into the construction of Markov chains, the estimation of energy functions, and the properties of transition matrices. It also explores the invariant vectors of transition matrices and their significance in stochastic processes.

Instructor

Fabio Nobile

Teaching & PhD PhD Students Veronica Montanaro, Matteo Raviola, Francesca Bettinelli, Eliott Van Dieren, Riccardo Saporiti, Giacomo Mossinelli, Noé Stauffer, Fabio Zoccolan Past EPFL PhD Students Diane Sylvie Guignard, Francesco Tesei, Eleonora Musharbash, Matthieu Claude Martin, Davide Pradovera, Sundar Subramaniam Ganesh, Eva Vidlicková, Juan Pablo Madrigal Cianci, Andrea Zanoni, Thomas Simon Spencer Trigo Trindade Past EPFL PhD Students as codirector Michele Pisaroni Courses Advanced analysis II - ve

Official source

Related concepts (30)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

State function

In the thermodynamics of equilibrium, a state function, function of state, or point function for a thermodynamic system is a mathematical function relating several state variables or state quantities (that describe equilibrium states of a system) that depend only on the current equilibrium thermodynamic state of the system (e.g. gas, liquid, solid, crystal, or emulsion), not the path which the system has taken to reach that state. A state function describes equilibrium states of a system, thus also describing the type of system.

Helmholtz free energy

In thermodynamics, the Helmholtz free energy (or Helmholtz energy) is a thermodynamic potential that measures the useful work obtainable from a closed thermodynamic system at a constant temperature (isothermal). The change in the Helmholtz energy during a process is equal to the maximum amount of work that the system can perform in a thermodynamic process in which temperature is held constant. At constant temperature, the Helmholtz free energy is minimized at equilibrium.

Internal energy

The internal energy of a thermodynamic system is the energy contained within it, measured as the quantity of energy necessary to bring the system from its standard internal state to its present internal state of interest, accounting for the gains and losses of energy due to changes in its internal state, including such quantities as magnetization. It excludes the kinetic energy of motion of the system as a whole and the potential energy of position of the system as a whole, with respect to its surroundings and external force fields.

Statistical mechanics

In physics, statistical mechanics is a mathematical framework that applies statistical methods and probability theory to large assemblies of microscopic entities. It does not assume or postulate any natural laws, but explains the macroscopic behavior of nature from the behavior of such ensembles. Sometimes called statistical physics or statistical thermodynamics, its applications include many problems in the fields of physics, biology, chemistry, and neuroscience.

Related lectures (64)

Markov Chains: Theory and Applications

Covers the theory and applications of Markov chains in modeling random phenomena and decision-making under uncertainty.

Monte Carlo: Markov Chains

Covers unsupervised learning, dimensionality reduction, SVD, low-rank estimation, PCA, and Monte Carlo Markov Chains.

Probability & Stochastic Processes

Covers applied probability, stochastic processes, Markov chains, rejection sampling, and Bayesian inference methods.

Stochastic Simulation: Markov Chains and Metropolis Hastings

Introduces Markov chains and Metropolis Hastings algorithm in stochastic simulation.

Markov Chains: Introduction and Properties