Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Markov Chains: Communicating Classes

Description

This lecture covers the identification of communicating classes in Markov chains, distinguishing between transient and recurrent classes. The instructor explains the concepts using examples of Markov chains with different transition probability matrices. The lecture also delves into the properties of these classes, such as recurrence and transience, providing insights into the behavior of the chains over time. Additionally, the lecture explores the stationary distribution of Markov chains and the significance of doubly stochastic transition probability matrices. Various exercises are presented to reinforce the understanding of the discussed concepts.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Markov model

In probability theory, a Markov model is a stochastic model used to model pseudo-randomly changing systems. It is assumed that future states depend only on the current state, not on the events that occurred before it (that is, it assumes the Markov property). Generally, this assumption enables reasoning and computation with the model that would otherwise be intractable. For this reason, in the fields of predictive modelling and probabilistic forecasting, it is desirable for a given model to exhibit the Markov property.

Markov property

In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time. The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model.

Stochastic matrix

In mathematics, a stochastic matrix is a square matrix used to describe the transitions of a Markov chain. Each of its entries is a nonnegative real number representing a probability. It is also called a probability matrix, transition matrix, substitution matrix, or Markov matrix. The stochastic matrix was first developed by Andrey Markov at the beginning of the 20th century, and has found use throughout a wide variety of scientific fields, including probability theory, statistics, mathematical finance and linear algebra, as well as computer science and population genetics.

Hidden Markov model

A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process — call it — with unobservable ("hidden") states. As part of the definition, HMM requires that there be an observable process whose outcomes are "influenced" by the outcomes of in a known way.

Related lectures (47)

Markov Chains: Introduction and Properties

Covers the introduction and properties of Markov chains, including transition matrices and stochastic processes.

Markov Chains: Ergodicity and Stationary Distribution

Explores ergodicity and stationary distribution in Markov chains, emphasizing convergence properties and unique distributions.

Markov Chains: Simulation and Optimization

Explores Markov chains, Metropolis-Hastings, and simulation for optimization purposes, highlighting the significance of ergodicity in efficient variable simulation.

Stationary Distribution in Markov Chains

Explores the concept of stationary distribution in Markov chains, discussing its properties and implications, as well as the conditions for positive-recurrence.

Ergodic Theorem: Proof and Applications