Lecture

Learning Agents: Exploration-Exploitation Tradeoff

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-430: Intelligent agents

Software agents are widely used to control physical, economic and financial processes. The course presents practical methods for implementing software agents and multi-agent systems, supported by programming exercises, and the theoretical underpinnings inc ...

Description

This lecture covers the implementation of reactive agents that learn from observations, focusing on the exploration-exploitation tradeoff in learning unknown effects of actions. It discusses scenarios where an adversary can influence the world and techniques to develop robust strategies. Topics include multi-armed bandits, Q-learning, contextual bandits, and strategies like epsilon-greedy, Thompson sampling, and regret matching. The lecture also explores the challenges of learning with state transitions and the use of deep Q-learning and experience replay to address them.

Instructor

Boi Faltings

Professor Faltings joined EPFL in 1987 as professor of Artificial Intelligence. He holds a PhD degree from the University of Illinois at Urbana-Champaign, and a diploma from the ETHZ. His research has spanned different areas of intelligent systems linked to model-based reasoning. In particular, he has contributed to qualitative spatial reasoning, case-based reasoning (especially for design problems), constraint satisfaction for design and logistics problems, multi-agent systems, and intelligent user interfaces. His current work is oriented towards multi-agent systems and social computing, using concepts of game theory, constraint optimization and machine learning. In 1999, Professor Faltings co-founded Iconomic Systems, a company that developed a new agent-based paradigm for travel e-commerce. He has since co-founded 5 other startup companies and advised several others. Prof. Faltings has published more than 150 refereed papers on his work, and participates regularly in program committees of all major conferences in the field. He has served as associate editor of of the major journals, including the Journal of Artificial Intelligence Research (JAIR) and the Artificial Intelligence Journal. From 1996 to 1998, he served as head of the computer science department.

Official source

Related concepts (32)

Q-learning

Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Deep reinforcement learning

Deep reinforcement learning (deep RL) is a subfield of machine learning that combines reinforcement learning (RL) and deep learning. RL considers the problem of a computational agent learning to make decisions by trial and error. Deep RL incorporates deep learning into the solution, allowing agents to make decisions from unstructured input data without manual engineering of the state space. Deep RL algorithms are able to take in very large inputs (e.g.

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Reinforcement learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.

Related lectures (64)

Acquiring Data for Learning: Modern Approaches and Challenges

Explores modern approaches and challenges in acquiring data for learning optimal controllers through demonstrations and data-driven methods.

Neuroscience and AI: Bridging the Gap

Explores the gap between AI and human intelligence through neuroscience-inspired models and algorithms.

Reinforcement Learning: Policy Gradient and Actor-Critic Methods

Provides an overview of reinforcement learning, focusing on policy gradient and actor-critic methods for deep artificial neural networks.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Statistical Physics in Machine Learning: Understanding Deep Learning