Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Continuous space: action space

Description

This lecture introduces the concept of continuous state and action spaces in reinforcement learning, focusing on methods to transfer techniques from discrete to continuous spaces. Topics covered include parametric approximation, SARSA algorithm, eligibility traces, and results from the mountain car task.

Official source

Related concepts (27)

Reinforcement learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.

Continuous function

In mathematics, a continuous function is a function such that a continuous variation (that is a change without jump) of the argument induces a continuous variation of the value of the function. This means that there are no abrupt changes in value, known as discontinuities. More precisely, a function is continuous if arbitrarily small changes in its value can be assured by restricting to sufficiently small changes of its argument. A discontinuous function is a function that is .

Q-learning

Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.

Continuous linear operator

In functional analysis and related areas of mathematics, a continuous linear operator or continuous linear mapping is a continuous linear transformation between topological vector spaces. An operator between two normed spaces is a bounded linear operator if and only if it is a continuous linear operator. Continuous function (topology) and Discontinuous linear map Bounded operator Suppose that is a linear operator between two topological vector spaces (TVSs). The following are equivalent: is continuous.

Dual space

In mathematics, any vector space has a corresponding dual vector space (or just dual space for short) consisting of all linear forms on together with the vector space structure of pointwise addition and scalar multiplication by constants. The dual space as defined above is defined for all vector spaces, and to avoid ambiguity may also be called the . When defined for a topological vector space, there is a subspace of the dual space, corresponding to continuous linear functionals, called the continuous dual space.

Related lectures (32)

Advanced Machine Learning: Discrete Reinforcement Learning

Introduces the basics of Reinforcement Learning, covering discrete states, actions, policies, value functions, MDPs, and optimal policies.

Policy Gradient and Actor-Critic Methods: Eligibility Traces Explained

Discusses policy gradient and actor-critic methods, focusing on eligibility traces and their application in reinforcement learning tasks.

Approximation Landau: Ising Model

Explores the Landau approximation applied to the Ising model in statistical physics.

Bounded Operators: Theory and Applications

Covers bounded operators between normed vector spaces, emphasizing the importance of continuity and exploring applications like the Fourier transform.

Reinforcement Learning: Q-Learning