Lecture

Model-based versus Model-free Reinforcement Learning

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-456: Deep reinforcement learning

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approaches, like PPO and DQN. The course will introduce student ...

Description

This lecture explores the difference between model-based and model-free reinforcement learning, focusing on how the agent adapts when the goal changes, the definition of each approach, and the advantages of model-based RL, such as the ability to readapt to reward changes and plan future actions in the 'mind'. The lecture also discusses the implementation of Chess and Go as model-based systems without the need to learn the model.

Instructors (2)

Caglar Gulcehre

Teaching & PhD PhD Students Tim Ruben Davidson, Skander Moalla, Anja Surina, Xiuying Wei, Liangze Jiang, Mikhail Terekhov, Amin Mansouri, Justin Samuel Deschenaux, Yitao Xu Courses Training Large Language Models EE-628 This PhD-level course dives deep into the training of Large Language Models (LLMs), focusing on the complementary roles of datasets, pre-training and post training methodologies in shaping model performance and scalability.

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Official source

Related concepts (12)

Agent-based model

An agent-based model (ABM) is a computational model for simulating the actions and interactions of autonomous agents (both individual or collective entities such as organizations or groups) in order to understand the behavior of a system and what governs its outcomes. It combines elements of game theory, complex systems, emergence, computational sociology, multi-agent systems, and evolutionary programming. Monte Carlo methods are used to understand the stochasticity of these models.

Intelligent agent

In artificial intelligence, an intelligent agent (IA) is an agent acting in an intelligent manner; It perceives its environment, takes actions autonomously in order to achieve goals, and may improve its performance with learning or acquiring knowledge. An intelligent agent may be simple or complex: A thermostat or other control system is considered an example of an intelligent agent, as is a human being, as is any system that meets the definition, such as a firm, a state, or a biome.

Software agent

In computer science, a software agent or software AI is a computer program that acts for a user or other program in a relationship of agency, which derives from the Latin agere (to do): an agreement to act on one's behalf. Such "action on behalf of" implies the authority to decide which, if any, action is appropriate. Some agents are colloquially known as bots, from robot. They may be embodied, as when execution is paired with a robot body, or as software such as a chatbot executing on a phone (e.g.

Urban planning

Urban planning, also known as town planning, city planning, regional planning, or rural planning, is a technical and political process that is focused on the development and design of land use and the built environment, including air, water, and the infrastructure passing into and out of urban areas, such as transportation, communications, and distribution networks and their accessibility.

Spatial planning

Spatial planning mediates between the respective claims on space of the state, market, and community. In so doing, three different mechanisms of involving stakeholders, integrating sectoral policies and promoting development projects mark the three schools of transformative strategy formulation, innovation action and performance in spatial planning Spatial planning systems refer to the methods and approaches used by the public and private sector to influence the distribution of people and activities in spaces of various scales.

Related lectures (31)

Intelligent Agents: Making Decisions and Planning

Covers intelligent agents, decision-making, planning, machine learning, and game theory.

Multiagent Systems: Planning and Coordination

Explores the use of multi-agent systems for planning, coordination, and task assignment among heterogeneous agents.

Safe Learning for Autonomous Systems

Explores challenges in control, safety, and coordination for autonomous systems like autonomous cars, focusing on safe learning and Nash equilibria.

Reinforcement Learning: Q-Learning

Covers Q-Learning in reinforcement learning, exploring action values, policies, and the societal impact of algorithms.

Introduction to Reinforcement Learning: Key Concepts and Applications