Lecture

Markov Chains: Theory and Applications

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-414: Stochastic simulation

The student who follows this course will get acquainted with computational tools used to analyze systems with uncertainty arising in engineering, physics, chemistry, and economics. Focus will be on sampling methods as Monte Carlo, quasi Monte Carlo, Mark ...

Description

This lecture covers the theory and applications of Markov chains, focusing on topics such as discrete state space, transition matrices, probability mass functions, recurrent states, and periodic chains. The instructor explains the concepts of irreducibility, aperiodicity, positive recurrence, and invariant probability distributions. The lecture also delves into the properties of eigenvalues, eigenvectors, and the limiting behavior of Markov chains.

Instructor

Fabio Nobile

Teaching & PhD PhD Students Veronica Montanaro, Matteo Raviola, Francesca Bettinelli, Eliott Van Dieren, Riccardo Saporiti, Giacomo Mossinelli, Noé Stauffer, Fabio Zoccolan Past EPFL PhD Students Diane Sylvie Guignard, Francesco Tesei, Eleonora Musharbash, Matthieu Claude Martin, Davide Pradovera, Sundar Subramaniam Ganesh, Eva Vidlicková, Juan Pablo Madrigal Cianci, Andrea Zanoni, Thomas Simon Spencer Trigo Trindade Past EPFL PhD Students as codirector Michele Pisaroni Courses Advanced analysis II - ve

Official source

Related concepts (32)

Probability density function

In probability theory, a probability density function (PDF), density function, or density of an absolutely continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can be interpreted as providing a relative likelihood that the value of the random variable would be equal to that sample.

Eigenvalues and eigenvectors

In linear algebra, an eigenvector (ˈaɪgənˌvɛktər) or characteristic vector of a linear transformation is a nonzero vector that changes at most by a constant factor when that linear transformation is applied to it. The corresponding eigenvalue, often represented by , is the multiplying factor. Geometrically, a transformation matrix rotates, stretches, or shears the vectors it acts upon. The eigenvectors for a linear transformation matrix are the set of vectors that are only stretched, with no rotation or shear.

Search engine

A search engine is a software system that finds web pages that match a web search. They search the World Wide Web in a systematic way for particular information specified in a textual web search query. The search results are generally presented in a line of results, often referred to as search engine results pages (SERPs). The information may be a mix of hyperlinks to web pages, images, videos, infographics, articles, and other types of files. Some search engines also mine data available in databases or open directories.

Google Search

Google Search (also known simply as Google or Google.com) is a search engine provided and operated by Google. Handling more than 3.5 billion searches per day, it has a 92% share of the global search engine market. It is the most-visited website in the world. Additionally, it is the most searched and used search engine in the entire world. The order of search results returned by Google is based, in part, on a priority rank system called "PageRank".

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Related lectures (64)

Geometric Ergodicity: Convergence Diagnostics

Covers the concept of geometric ergodicity in the context of convergence diagnostics for Markov chains.

Markov Chain Monte Carlo

Covers the Markov Chain Monte Carlo method and the Metropolis-Hastings algorithm for generating samples from a target probability distribution.

Estimating Relaxation Time: Variance and Chains

Covers the estimation of relaxation time in chains and the importance of sample sizes.

Advanced Probabilities: Random Variables & Expected Values

Explores advanced probabilities, random variables, and expected values, with practical examples and quizzes to reinforce learning.

Stationary Distribution in Markov Chains