Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Dynamic Programming: Optimal Control

Description

This lecture covers the application of Dynamic Programming (DP) in solving optimal control problems. It discusses the optimality of DP, the machine replacement problem, time-homogeneous Markov chains, control objectives, and policies. The lecture also delves into the linear quadratic control problem, stability of uncontrolled systems, and recursive solutions for optimal policies.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Optimal control

Optimal control theory is a branch of mathematical optimization that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a spacecraft with controls corresponding to rocket thrusters, and the objective might be to reach the moon with minimum fuel expenditure.

Bellman equation

A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes.

Dynamic programming

Dynamic programming is both a mathematical optimization method and an algorithmic paradigm. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics. In both contexts it refers to simplifying a complicated problem by breaking it down into simpler sub-problems in a recursive manner. While some decision problems cannot be taken apart this way, decisions that span several points in time do often break apart recursively.

Recursion (computer science)

In computer science, recursion is a method of solving a computational problem where the solution depends on solutions to smaller instances of the same problem. Recursion solves such recursive problems by using functions that call themselves from within their own code. The approach can be applied to many types of problems, and recursion is one of the central ideas of computer science. The power of recursion evidently lies in the possibility of defining an infinite set of objects by a finite statement.

Linear–quadratic regulator

The theory of optimal control is concerned with operating a dynamic system at minimum cost. The case where the system dynamics are described by a set of linear differential equations and the cost is described by a quadratic function is called the LQ problem. One of the main results in the theory is that the solution is provided by the linear–quadratic regulator (LQR), a feedback controller whose equations are given below. LQR controllers possess inherent robustness with guaranteed gain and phase margin, and they also are part of the solution to the LQG (linear–quadratic–Gaussian) problem.

Related lectures (49)

Linear Quadratic (LQ) Optimal Control: Proof of Theorem

Covers the proof of the recursive formula for the optimal gains in LQ control over a finite horizon.

Linear Quadratic Optimal Control: Analysis and Solution

Explores Linear Quadratic optimal control, analyzing cost, and presenting the solution to the FH-LQ problem.

Infinite-Horizon LQ Control: Solution & Example

Explores Infinite-Horizon Linear Quadratic (LQ) optimal control, emphasizing solution methods and practical examples.

Advanced Machine Learning: Discrete Reinforcement Learning

Introduces the basics of Reinforcement Learning, covering discrete states, actions, policies, value functions, MDPs, and optimal policies.

Markov Decision Processes: Foundations of Reinforcement Learning