Lecture

Hitting Probabilities: Markov Chains

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-332: Markov chains

The course follows the text of Norris and the polycopie (which will be distributed chapter by chapter). ...

Description

This lecture discusses hitting probabilities in Markov chains, focusing on a state space with disjoint subsets A and B. The instructor explains the concept of hitting probabilities, formal definitions, and properties of the function h(i). The lecture covers theorems, proofs, and expected time to hit calculations, emphasizing the minimal function satisfying specific conditions. The presentation concludes with uniqueness considerations and examples to illustrate the concepts.

Instructor

Thomas Mountford

Teaching & PhD Past EPFL PhD Students Damien Gauthier, Samir Brahim Belhaouari, Driss Baraka, Daniel Rodrigues Valesin, Samuel Schöpfer, Kamran Kalbasi, Jacques Saliba, Hao Xue, Keming Chen Past EPFL PhD Students as codirector Guillaume Charles Woessner Courses Analysis I MATH-101(d) Study the fundamental concepts of analysis, and the differential and integral calculus of real-valued functions of one real variable. Markov chains MATH-332 The course follows the text of Norris

Official source

Related concepts (20)

Gödel's incompleteness theorems

Gödel's incompleteness theorems are two theorems of mathematical logic that are concerned with the limits of in formal axiomatic theories. These results, published by Kurt Gödel in 1931, are important both in mathematical logic and in the philosophy of mathematics. The theorems are widely, but not universally, interpreted as showing that Hilbert's program to find a complete and consistent set of axioms for all mathematics is impossible. The first incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an effective procedure (i.

Abelian and Tauberian theorems

In mathematics, Abelian and Tauberian theorems are theorems giving conditions for two methods of summing divergent series to give the same result, named after Niels Henrik Abel and Alfred Tauber. The original examples are Abel's theorem showing that if a series converges to some limit then its Abel sum is the same limit, and Tauber's theorem showing that if the Abel sum of a series exists and the coefficients are sufficiently small (o(1/n)) then the series converges to the Abel sum.

Markov model

In probability theory, a Markov model is a stochastic model used to model pseudo-randomly changing systems. It is assumed that future states depend only on the current state, not on the events that occurred before it (that is, it assumes the Markov property). Generally, this assumption enables reasoning and computation with the model that would otherwise be intractable. For this reason, in the fields of predictive modelling and probabilistic forecasting, it is desirable for a given model to exhibit the Markov property.

Proof by contradiction

In logic, proof by contradiction is a form of proof that establishes the truth or the validity of a proposition, by showing that assuming the proposition to be false leads to a contradiction. Although it is quite freely used in mathematical proofs, not every school of mathematical thought accepts this kind of nonconstructive proof as universally valid. More broadly, proof by contradiction is any form of argument that establishes a statement by arriving at a contradiction, even when the initial assumption is not the negation of the statement to be proved.

Hidden Markov model

A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process — call it — with unobservable ("hidden") states. As part of the definition, HMM requires that there be an observable process whose outcomes are "influenced" by the outcomes of in a known way.

Related lectures (64)

Proofs: Logic, Mathematics & Algorithms

Explores proof concepts, techniques, and applications in logic, mathematics, and algorithms.

Lower Bound on Total Variation Distance

Explores the lower bound on total variation distance in Markov chains and its implications on mixing time.

Modular Arithmetic: Inverses and Equations

Explores modular arithmetic, emphasizing inverses and equations in Z/mZ, with practical examples and exercises.

Continuous Time Markov Chains

Covers the basic theory for continuous time Markov chains and discusses communication, hitting probabilities, recurrence, and transience.

Birth and Death Chains