Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Dynamic Programming: Optimal Control

In course

MGT-484: Applied probability & stochastic processes

This course focuses on dynamic models of random phenomena, and in particular, the most popular classes of such models: Markov chains and Markov decision processes. We will also study applications in queuing theory, finance, project management, etc. ...

Description

This lecture covers the application of Dynamic Programming to solve optimal control problems, focusing on the recursive solution and the stability of the uncontrolled system. The lecture also discusses the concept of a stationary policy and the convergence to a unique fixed point. The recursive solution leads to closed-form solutions for the optimal control policy, emphasizing the importance of the Bellman equation and the principle of optimality.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Negar Kiyavash

Official source

Related concepts (32)

Optimal control

Optimal control theory is a branch of mathematical optimization that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a spacecraft with controls corresponding to rocket thrusters, and the objective might be to reach the moon with minimum fuel expenditure.

Policy

Policy is a deliberate system of guidelines to guide decisions and achieve rational outcomes. A policy is a statement of intent and is implemented as a procedure or protocol. Policies are generally adopted by a governance body within an organization. Policies can assist in both subjective and objective decision making. Policies used in subjective decision-making usually assist senior management with decisions that must be based on the relative merits of a number of factors, and as a result, are often hard to test objectively, e.

Bellman equation

A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes.

Public policy

Public policy is an institutionalized proposal or a decided set of elements like laws, regulations, guidelines, and actions to solve or address relevant and real-world problems, guided by a conception and often implemented by programs. The implementation of public policy is known as public administration. Public policy can be considered to be the sum of a government's direct and indirect activities and has been conceptualized in a variety of ways. They are created and/or enacted on behalf of the public typically by a government.

Policy studies

Policy studies is a subdiscipline of political science that includes the analysis of the process of policymaking (the policy process) and the contents of policy (policy analysis). Policy analysis includes substantive area research (such as health or education policy), program evaluation and impact studies, and policy design. It "involves systematically studying the nature, causes, and effects of alternative public policies, with particular emphasis on determining the policies that will achieve given goals.

Related lectures (33)

Linear Quadratic (LQ) Optimal Control: Proof of Theorem

Covers the proof of the recursive formula for the optimal gains in LQ control over a finite horizon.

Infinite-Horizon LQ Control: Solution & Example

Explores Infinite-Horizon Linear Quadratic (LQ) optimal control, emphasizing solution methods and practical examples.

Markov Decision Processes: Foundations of Reinforcement Learning

Covers Markov Decision Processes, their structure, and their role in reinforcement learning.

Stability: Poles, Zeros, and Control

Covers stability, poles, zeros, and control in dynamic systems, emphasizing the importance of observability.

Value Iteration Acceleration: PID and Operator Splitting