Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Markov Chains: Theory and Applications

In course

MATH-414: Stochastic simulation

The student who follows this course will get acquainted with computational tools used to analyze systems with uncertainty arising in engineering, physics, chemistry, and economics. Focus will be on sampling methods as Monte Carlo, quasi Monte Carlo, Mark ...

Description

This lecture covers the theory and applications of Markov chains, focusing on topics such as discrete state space, transition matrices, probability mass functions, recurrent states, and periodic chains. The instructor explains the concepts of irreducibility, aperiodicity, positive recurrence, and invariant probability distributions. The lecture also delves into the properties of eigenvalues, eigenvectors, and the limiting behavior of Markov chains.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Fabio Nobile

Official source

Related concepts (32)

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Probability density function

In probability theory, a probability density function (PDF), density function, or density of an absolutely continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can be interpreted as providing a relative likelihood that the value of the random variable would be equal to that sample.

Eigenvalues and eigenvectors

In linear algebra, an eigenvector (ˈaɪgənˌvɛktər) or characteristic vector of a linear transformation is a nonzero vector that changes at most by a constant factor when that linear transformation is applied to it. The corresponding eigenvalue, often represented by , is the multiplying factor. Geometrically, a transformation matrix rotates, stretches, or shears the vectors it acts upon. The eigenvectors for a linear transformation matrix are the set of vectors that are only stretched, with no rotation or shear.

Chat room

The term chat room, or chatroom (and sometimes group chat; abbreviated as GC), is primarily used to describe any form of synchronous conferencing, occasionally even asynchronous conferencing. The term can thus mean any technology, ranging from real-time online chat and online interaction with strangers (e.g., online forums) to fully immersive graphical social environments. The primary use of a chat room is to share information via text with a group of other users.

Web chat

A web chat is a text-based system that allows users to communicate in real-time using easily accessible web interfaces. It is a type of Internet online chat distinguished by its simplicity and accessibility to users who do not wish to take the time to install and learn to use specialized chat software. This trait allows users instantaneous access and only a web browser is required to chat. Users will always get the latest version of a chat service because no software installation or update are required.

Related lectures (41)

Stationary Distribution in Markov Chains

Explores the concept of stationary distribution in Markov chains, discussing its properties and implications, as well as the conditions for positive-recurrence.

Advanced Probabilities: Random Variables & Expected Values

Explores advanced probabilities, random variables, and expected values, with practical examples and quizzes to reinforce learning.

Random Variables and Expected Value

Introduces random variables, probability distributions, and expected values through practical examples.

Continuous Random Variables

Covers continuous random variables, probability density functions, and distributions, with practical examples.

Fundamental Limits of Gradient-Based Learning