Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Markov Chains: Basics and Applications

In course

MATH-414: Stochastic simulation

The student who follows this course will get acquainted with computational tools used to analyze systems with uncertainty arising in engineering, physics, chemistry, and economics. Focus will be on sampling methods as Monte Carlo, quasi Monte Carlo, Mark ...

Description

This lecture covers the fundamentals of Markov chains, focusing on discrete time and discrete space Markov chains. It explains the Markov property, transition matrices, generation algorithms, and examples like random walks. It also delves into continuous space Markov chains, transition kernels, and Poisson processes. The lecture concludes with discussions on non-homogeneous Poisson processes and general continuous time Markov chains.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Fabio Nobile

Official source

Related concepts (32)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Poisson point process

In probability, statistics and related fields, a Poisson point process is a type of random mathematical object that consists of points randomly located on a mathematical space with the essential feature that the points occur independently of one another. The Poisson point process is often called simply the Poisson process, but it is also called a Poisson random measure, Poisson random point field or Poisson point field.

Markov property

In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time. The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model.

Homogeneous space

In mathematics, a homogeneous space is, very informally, a space that looks the same everywhere, as you move through it, with movement given by the action of a group. Homogeneous spaces occur in the theories of Lie groups, algebraic groups and topological groups. More precisely, a homogeneous space for a group G is a non-empty manifold or topological space X on which G acts transitively. The elements of G are called the symmetries of X.

Compound Poisson process

A compound Poisson process is a continuous-time stochastic process with jumps. The jumps arrive randomly according to a Poisson process and the size of the jumps is also random, with a specified probability distribution. To be precise, a compound Poisson process, parameterised by a rate and jump size distribution G, is a process given by where, is the counting variable of a Poisson process with rate , and are independent and identically distributed random variables, with distribution function G, which are also independent of When are non-negative integer-valued random variables, then this compound Poisson process is known as a stuttering Poisson process.

Related lectures (50)

Continuous Time Markov Chains

Introduces continuous time Markov chains on a finite state space with exponential waiting times and jump probabilities.

Markov Chains: Introduction and Properties

Covers the introduction and properties of Markov chains, including transition matrices and stochastic processes.

Simulation & Optimization: Poisson Process & Random Numbers

Explores simulation pitfalls, random numbers, discrete & continuous distributions, and Monte-Carlo integration.

Markov Chains: Definition and Examples

Covers the definition and properties of Markov chains, including transition matrix and examples.

Markov Chains: Definitions and Transitions