Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Applied Probability and Stochastic Processes: Tutorial 02

In course

MGT-484: Applied probability & stochastic processes

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc. ...

Description

This lecture covers the main concepts of communicating classes, hitting time, hitting probabilities, birth and death chains, and the Gambler's Ruin Model. It explains how to find communicating classes, calculate hitting times and probabilities, and analyze different models. Examples and exercises are provided to illustrate the theoretical concepts.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Negar Kiyavash

Official source

Related concepts (29)

Gambler's ruin

In statistics, 'gambler's ruin' is the fact that a gambler playing a game with negative expected value will eventually go broke, regardless of their betting system. The concept was initially stated: A persistent gambler who raises his or her bet to a fixed fraction of the gambler's bankroll after a win, but does not reduce it after a loss, will eventually and inevitably go broke, even if each bet has a positive expected value.

Stochastic process

In probability theory and related fields, a stochastic (stəˈkæstɪk) or random process is a mathematical object usually defined as a sequence of random variables, where the index of the sequence has the interpretation of time. Stochastic processes are widely used as mathematical models of systems and phenomena that appear to vary in a random manner. Examples include the growth of a bacterial population, an electrical current fluctuating due to thermal noise, or the movement of a gas molecule.

Random walk

In mathematics, a random walk is a random process that describes a path that consists of a succession of random steps on some mathematical space. An elementary example of a random walk is the random walk on the integer number line which starts at 0, and at each step moves +1 or −1 with equal probability. Other examples include the path traced by a molecule as it travels in a liquid or a gas (see Brownian motion), the search path of a foraging animal, or the price of a fluctuating stock and the financial status of a gambler.

Stochastic calculus

Stochastic calculus is a branch of mathematics that operates on stochastic processes. It allows a consistent theory of integration to be defined for integrals of stochastic processes with respect to stochastic processes. This field was created and started by the Japanese mathematician Kiyosi Itô during World War II. The best-known stochastic process to which stochastic calculus is applied is the Wiener process (named in honor of Norbert Wiener), which is used for modeling Brownian motion as described by Louis Bachelier in 1900 and by Albert Einstein in 1905 and other physical diffusion processes in space of particles subject to random forces.

Infinitesimal generator (stochastic processes)

In mathematics — specifically, in stochastic analysis — the infinitesimal generator of a Feller process (i.e. a continuous-time Markov process satisfying certain regularity conditions) is a Fourier multiplier operator that encodes a great deal of information about the process. The generator is used in evolution equations such as the Kolmogorov backward equation, which describes the evolution of statistics of the process; its L2 Hermitian adjoint is used in evolution equations such as the Fokker–Planck equation, also known as Kolmogorov forward equation, which describes the evolution of the probability density functions of the process.

Related lectures (31)

Random Walks: Return Probabilities in Lattice Dimensions

Covers random walks on a lattice, focusing on return probabilities and their dependence on dimensionality.

Probability and Statistics

Delves into probability, statistics, paradoxes, and random variables, showcasing their real-world applications and properties.

Fokker-Planck & KPZ Equations

Explores the Fokker-Planck and KPZ equations in stochastic processes and random growth models.

Random Walks: Return Probabilities in Multiple Dimensions

Covers the analysis of random walks and their return probabilities in multiple dimensions.