Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Markov Chains: Introduction and Properties

Description

This lecture introduces Markov chains, discussing elementary properties such as the transition matrix, communicating classes, and stochastic processes. It covers the definition of a stochastic process, probability vectors, and transition matrices. The lecture also explores the Markov property, equivalence theorems, and consequences of Markov chains.

Instructor

Thomas Mountford

D'origine britannique, né en 1961. Il a reçu la bourse présidentielle des jeunes investisseurs (Presidential Young Investigator Grant) en 1990 et le prix de la Fraternité Sloan (Sloan Fellowship) en 1991. Son travail montre que les valeurs critiques d'une classe large de de systèmes de particules proches sont égales à 1, et que, suite à plusieurs travaux sur la trajectoire du mouvement brownien, incluant une simulation numérique, le nombre d'îlots browniens en 2 dimensions tend vers l'infini quand leurs tailles tendent vers 0. Il reçoit le prix Rosenbaum en 1993 lui ouvrant ainsi les portes du Isaac Newton Institute à Cambridge. Il est également décoré par le prix Rollo Davidson en 1995 et nommé membre honoraire de l'Institute of Mathematical Statistics en 2001. Assistant Profeseur de Mathematique a UCLA 1987- 1991 Associate Professeur de Mathematique a UCLA 1991-1993 Professeur de Mathematique a UCLA 1993-2001 Professeur Departement de mathématiques, EPFL dès 2001

Official source

Related concepts (32)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Markov property

In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time. The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model.

Markov model

In probability theory, a Markov model is a stochastic model used to model pseudo-randomly changing systems. It is assumed that future states depend only on the current state, not on the events that occurred before it (that is, it assumes the Markov property). Generally, this assumption enables reasoning and computation with the model that would otherwise be intractable. For this reason, in the fields of predictive modelling and probabilistic forecasting, it is desirable for a given model to exhibit the Markov property.

Hidden Markov model

A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process — call it — with unobservable ("hidden") states. As part of the definition, HMM requires that there be an observable process whose outcomes are "influenced" by the outcomes of in a known way.

Markov chain Monte Carlo

In statistics, Markov chain Monte Carlo (MCMC) methods comprise a class of algorithms for sampling from a probability distribution. By constructing a Markov chain that has the desired distribution as its equilibrium distribution, one can obtain a sample of the desired distribution by recording states from the chain. The more steps that are included, the more closely the distribution of the sample matches the actual desired distribution. Various algorithms exist for constructing chains, including the Metropolis–Hastings algorithm.

Related lectures (36)

Hidden Markov Models: Primer

Introduces Hidden Markov Models, explaining the basic problems and algorithms like Forward-Backward, Viterbi, and Baum-Welch, with a focus on Expectation-Maximization.

Lindblad equation

Covers the interpretation of the Lindblad equation and its unitary part in quantum gases.

Markov chains

Covers Markov chains, Monte Carlo sampling, isotropy, and the curse of dimensionality.

Continuous Time Markov Chains

Introduces continuous time Markov chains on a finite state space with exponential waiting times and jump probabilities.

Markov Chains and Algorithm Applications