Lecture

Hitting Times: Examples

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MATH-332: Markov chains

The course follows the text of Norris and the polycopie (which will be distributed chapter by chapter). ...

Description

This lecture covers examples related to hitting times in Markov chains, starting with a simple state space of four points and then moving on to more complex scenarios involving infinite state spaces and snakes and ladders. The instructor demonstrates how to calculate the probabilities of hitting specific states and the expected number of steps before reaching a certain state.

Instructor

Thomas Mountford

Teaching & PhD Past EPFL PhD Students Damien Gauthier, Samir Brahim Belhaouari, Driss Baraka, Daniel Rodrigues Valesin, Samuel Schöpfer, Kamran Kalbasi, Jacques Saliba, Hao Xue, Keming Chen Past EPFL PhD Students as codirector Guillaume Charles Woessner Courses Analysis I MATH-101(d) Study the fundamental concepts of analysis, and the differential and integral calculus of real-valued functions of one real variable. Markov chains MATH-332 The course follows the text of Norris

Official source

Related concepts (22)

Probability

Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speaking, 0 indicates impossibility of the event and 1 indicates certainty. The higher the probability of an event, the more likely it is that the event will occur. A simple example is the tossing of a fair (unbiased) coin.

Probability theory

Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of axioms. Typically these axioms formalise probability in terms of a probability space, which assigns a measure taking values between 0 and 1, termed the probability measure, to a set of outcomes called the sample space.

Markov chain

A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC).

Bayesian probability

Bayesian probability (ˈbeɪziən or ˈbeɪʒən ) is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification of a personal belief. The Bayesian interpretation of probability can be seen as an extension of propositional logic that enables reasoning with hypotheses; that is, with propositions whose truth or falsity is unknown.

Probability axioms

The Kolmogorov axioms are the foundations of probability theory introduced by Russian mathematician Andrey Kolmogorov in 1933. These axioms remain central and have direct contributions to mathematics, the physical sciences, and real-world probability cases. An alternative approach to formalising probability, favoured by some Bayesians, is given by Cox's theorem. The assumptions as to setting up the axioms can be summarised as follows: Let be a measure space with being the probability of some event , and .

Related lectures (64)

Expected Number of Visits in State

Covers the criterion for recurrence in infinite chains based on the expected number of visits in a state.

Probability & Stochastic Processes

Covers applied probability, stochastic processes, Markov chains, rejection sampling, and Bayesian inference methods.

Probability and Statistics

Introduces key concepts in probability and statistics, illustrating their application through various examples and emphasizing the importance of mathematical language in understanding the universe.

Markov Chains: Theory and Applications

Covers the theory and applications of Markov chains in modeling random phenomena and decision-making under uncertainty.

Conditional Probability: Prediction Decomposition