Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Dynamic Programming Recap

In course

MGT-484: Applied probability & stochastic processes

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc. ...

Description

This lecture covers a recap of dynamic programming, focusing on elements of modeling such as horizon, state & action space, terminal cost, and transition dynamics. The instructor explains the optimal betting strategy in a gambling problem using dynamic programming, showing that the gambler should stake 1/3 of their money each round. The correctness of the strategy is verified for different scenarios, including when the probability of winning is 1. The expected logarithm of the terminal capital is calculated under the optimal betting strategy.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Negar Kiyavash

Official source

Related concepts (31)

Event horizon

In astrophysics, an event horizon is a boundary beyond which events cannot affect an observer. Wolfgang Rindler coined the term in the 1950s. In 1784, John Michell proposed that gravity can be strong enough in the vicinity of massive compact objects that even light cannot escape. At that time, the Newtonian theory of gravitation and the so-called corpuscular theory of light were dominant. In these theories, if the escape velocity of the gravitational influence of a massive object exceeds the speed of light, then light originating inside or from it can escape temporarily but will return.

Cosmological horizon

A cosmological horizon is a measure of the distance from which one could possibly retrieve information. This observable constraint is due to various properties of general relativity, the expanding universe, and the physics of Big Bang cosmology. Cosmological horizons set the size and scale of the observable universe. This article explains a number of these horizons. Particle horizon The particle horizon (also called the cosmological horizon, the comoving horizon, or the cosmic light horizon) is the maximum distance from which light from particles could have traveled to the observer in the age of the universe.

Particle horizon

The particle horizon (also called the cosmological horizon, the comoving horizon (in Dodelson's text), or the cosmic light horizon) is the maximum distance from which light from particles could have traveled to the observer in the age of the universe. Much like the concept of a terrestrial horizon, it represents the boundary between the observable and the unobservable regions of the universe, so its distance at the present epoch defines the size of the observable universe.

Optimal control

Optimal control theory is a branch of mathematical optimization that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a spacecraft with controls corresponding to rocket thrusters, and the objective might be to reach the moon with minimum fuel expenditure.

Dynamic programming

Dynamic programming is both a mathematical optimization method and an algorithmic paradigm. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics. In both contexts it refers to simplifying a complicated problem by breaking it down into simpler sub-problems in a recursive manner. While some decision problems cannot be taken apart this way, decisions that span several points in time do often break apart recursively.

Related lectures (37)

Policy Iteration and Linear Programming in MDPs

Discusses policy iteration and linear programming methods for solving Markov Decision Processes.

Holography in Classical Gravity

Explores hints of holography in classical black hole thermodynamics.

Markov Decision Processes: Foundations of Reinforcement Learning

Covers Markov Decision Processes, their structure, and their role in reinforcement learning.

Linear Quadratic (LQ) Optimal Control: Proof of Theorem

Covers the proof of the recursive formula for the optimal gains in LQ control over a finite horizon.

Inflation and Cosmology