Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Reinforcement Learning: Q-Learning

In course

EE-411: Fundamentals of inference and learning

This is an introductory course in the theory of statistics, inference, and machine learning, with an emphasis on theoretical understanding & practical exercises. The course will combine, and alternate, between mathematical theoretical foundations and prac ...

Description

This lecture covers the concept of Q-Learning, which involves finding the optimal policy by iteratively updating a Q-table based on rewards. It explains how to represent the Q-table, define the cost function, and learn the optimal Q-values using gradient descent. The lecture also delves into Deep Q-Learning, where a neural network approximates the Q-values, and explores the challenges of dealing with large state spaces in games like Atari. Additionally, it discusses the REINFORCE algorithm for policy gradient methods and Monte-Carlo Tree Search for decision-making. The presentation concludes with a glimpse into AlphaGo Zero, a milestone in reinforcement learning. Various concepts such as Bellman equation, value networks, and policy networks are elucidated.

Instructor

Florent Gérard Krzakala

Official source

Related concepts (32)

Reinforcement learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.

Q-learning

Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.

Deep reinforcement learning

Deep reinforcement learning (deep RL) is a subfield of machine learning that combines reinforcement learning (RL) and deep learning. RL considers the problem of a computational agent learning to make decisions by trial and error. Deep RL incorporates deep learning into the solution, allowing agents to make decisions from unstructured input data without manual engineering of the state space. Deep RL algorithms are able to take in very large inputs (e.g.

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Bellman equation

A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes.

Related lectures (37)

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Deep Learning: Fundamentals

Covers the fundamentals of deep learning, from artificial neurons to modern networks.

Non-Conceptual Knowledge Systems

Delves into the impact of deep learning on non-conceptual knowledge systems and the advancements in transformers and generative adversarial networks.

Reinforcement Learning Fundamentals

Delves into the fundamentals of reinforcement learning, discussing states, actions, rewards, policies, and neural network applications.

Neural Networks: Multilayer Learning