Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Deep Learning Agents: Reinforcement Learning

In course

CS-430: Intelligent agents

Software agents are widely used to control physical, economic and financial processes. The course presents practical methods for implementing software agents and multi-agent systems, supported by programming exercises, and the theoretical underpinnings inc ...

Description

This lecture by the instructor covers the topic of Deep Learning Agents in Reinforcement Learning. It delves into concepts such as Deep Reinforcement Learning, Off-Policy Learning, Multiagent Reinforcement Learning, and the challenges faced in training agents. The lecture discusses the use of neural networks to approximate Q-tables, values, and policies, enabling reinforcement learning without a model. It explores the Actor-Critic architecture, multi-task learning, and Proximal Policy Optimization. The importance of generalization for learning in unseen states and actions is highlighted, along with the complexities of multiagent settings and the potential weaknesses in training simulations.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Instructor

Boi Faltings

Professor Faltings joined EPFL in 1987 as professor of Artificial Intelligence. He holds a PhD degree from the University of Illinois at Urbana-Champaign, and a diploma from the ETHZ. His research has spanned different areas of intelligent systems linked to model-based reasoning. In particular, he has contributed to qualitative spatial reasoning, case-based reasoning (especially for design problems), constraint satisfaction for design and logistics problems, multi-agent systems, and intelligent user interfaces. His current work is oriented towards multi-agent systems and social computing, using concepts of game theory, constraint optimization and machine learning. In 1999, Professor Faltings co-founded Iconomic Systems, a company that developed a new agent-based paradigm for travel e-commerce. He has since co-founded 5 other startup companies and advised several others. Prof. Faltings has published more than 150 refereed papers on his work, and participates regularly in program committees of all major conferences in the field. He has served as associate editor of of the major journals, including the Journal of Artificial Intelligence Research (JAIR) and the Artificial Intelligence Journal. From 1996 to 1998, he served as head of the computer science department.

Official source

Related concepts (32)

Reinforcement learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.

Q-learning

Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.

Deep reinforcement learning

Deep reinforcement learning (deep RL) is a subfield of machine learning that combines reinforcement learning (RL) and deep learning. RL considers the problem of a computational agent learning to make decisions by trial and error. Deep RL incorporates deep learning into the solution, allowing agents to make decisions from unstructured input data without manual engineering of the state space. Deep RL algorithms are able to take in very large inputs (e.g.

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Related lectures (29)

Reinforcement Learning: Policy Gradient and Actor-Critic Methods

Provides an overview of reinforcement learning, focusing on policy gradient and actor-critic methods for deep artificial neural networks.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Statistical Physics in Machine Learning: Understanding Deep Learning

Explores the application of statistical physics in understanding deep learning with a focus on neural networks and machine learning challenges.

Policy Gradient Methods: Direct Action Learning in Reinforcement Learning

Covers policy gradient methods, focusing on direct action learning and optimizing rewards in reinforcement learning.

Reinforcement Learning: BackUp Diagrams