Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Séance de cours

Équation de Bellman : Cohérence de la valeur et actions optimales

Dans cours

CS-456: Deep reinforcement learning

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approaches, like PPO and DQN. The course will introduce student ...

Description

Cette séance de cours présente l'équation de Bellman, qui relie les valeurs Q des paires état-action aux récompenses futures. Il couvre l'importance du facteur d'actualisation, le concept de récompense escomptée totale escomptée, et la cohérence de la valeur des États voisins. L'instructeur explique comment l'équation de Bellman est utilisée pour déterminer les actions optimales et les implications des différentes politiques sur la formulation de l'équation.

Enseignant·e·s (2)

Caglar Gulcehre

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Source officielle

Concepts associés (7)

Bellman equation

A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes.

Actualisation

L'actualisation est l'application de taux, dits taux d'actualisation, à des flux financiers non directement comparables et portant sur des durées différentes, afin de les comparer ou combiner de diverses façons. Elle apporte de la méthode dans le choix des investissements et peut intégrer l'évolution de la valeur de l'argent. Les méthodes d'actualisation doivent prendre en considération deux facteurs humains déterminant la valeur temps de l'argent : la préférence pour la jouissance immédiate et l'aversion au risque.

Discounted cash flow

The discounted cash flow (DCF) analysis, in finance, is a method used to value a security, project, company, or asset, that incorporates the time value of money. Discounted cash flow analysis is widely used in investment finance, real estate development, corporate financial management, and patent valuation. Used in industry as early as the 1700s or 1800s, it was widely discussed in financial economics in the 1960s, and U.S. courts began employing the concept in the 1980s and 1990s.

Valuation using discounted cash flows

Valuation using discounted cash flows (DCF valuation) is a method of estimating the current value of a company based on projected future cash flows adjusted for the time value of money. The cash flows are made up of those within the “explicit” forecast period, together with a continuing or terminal value that represents the cash flow stream after the forecast period. In several contexts, DCF valuation is referred to as the "income approach".

Optimal stopping

In mathematics, the theory of optimal stopping or early stopping is concerned with the problem of choosing a time to take a particular action, in order to maximise an expected reward or minimise an expected cost. Optimal stopping problems can be found in areas of statistics, economics, and mathematical finance (related to the pricing of American options). A key example of an optimal stopping problem is the secretary problem. Optimal stopping problems can often be written in the form of a Bellman equation, and are therefore often solved using dynamic programming.

Afficher plus

Séances de cours associées (23)

Iteration des politiques et programmation linéaire dans les MDP

Discute de l'itération des politiques et des méthodes de programmation linéaire pour résoudre les processus décisionnels de Markov.

Processus de décision de Markov: Techniques de programmation dynamique

Discute des processus décisionnels de Markov et des techniques de programmation dynamique pour résoudre des politiques optimales dans divers scénarios.

Économie immobilière MOOC: Economie du Sol et de l'Immobilier I

Couvre les calculs intertemporels, l'arbitrage du propriétaire foncier et l'évaluation de la rentabilité en économie immobilière.

Analyse des investissements immobiliers

Explore l'analyse de l'investissement immobilier, en comparant les coûts de location avec la propriété et en évaluant les aspects financiers pour des décisions d'achat optimales.

Apprentissage des différences temporelles

Couvertures Renforcement Théorie de l'apprentissage, Q-Learning, et horizons multi-étapes.

Afficher plus

Concepts associés (7)

Bellman equation

Actualisation

Discounted cash flow

Valuation using discounted cash flows

Optimal stopping

Afficher plus