Lecture

Long-Horizon Robot Exploration Strategies

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture presents strategies for long-horizon robot exploration in continuous action spaces, focusing on multi-object search. The approach combines long-horizon thinking with short-horizon control, extending multi-object search to continuous action spaces. The lecture details the architecture, where a predictive head guides the direction to the next closest object, and the policy can follow or deviate from long-horizon goals. Results from training and testing in different environments are discussed, showcasing the effectiveness of the proposed approach.

Official source

Related concepts (5)

Reinforcement learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.

Q-learning

Q-learning is a model-free reinforcement learning algorithm to learn the value of an action in a particular state. It does not require a model of the environment (hence "model-free"), and it can handle problems with stochastic transitions and rewards without requiring adaptations. For any finite Markov decision process (FMDP), Q-learning finds an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state.

Temporal difference learning

Temporal difference (TD) learning refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function. These methods sample from the environment, like Monte Carlo methods, and perform updates based on current estimates, like dynamic programming methods. While Monte Carlo methods only adjust their estimates once the final outcome is known, TD methods adjust predictions to match later, more accurate, predictions about the future before the final outcome is known.

Multi-agent system

A multi-agent system (MAS or "self-organized system") is a computerized system composed of multiple interacting intelligent agents. Multi-agent systems can solve problems that are difficult or impossible for an individual agent or a monolithic system to solve. Intelligence may include methodic, functional, procedural approaches, algorithmic search or reinforcement learning. Despite considerable overlap, a multi-agent system is not always the same as an agent-based model (ABM).

Map

A map is a symbolic depiction emphasizing relationships between elements of some space, such as objects, regions, or themes. Many maps are static, fixed to paper or some other durable medium, while others are dynamic or interactive. Although most commonly used to depict geography, maps may represent any space, real or fictional, without regard to context or scale, such as in brain mapping, DNA mapping, or computer network topology mapping.

Related lectures (4)

Deep Learning Agents: Reinforcement Learning

Explores Deep Learning Agents in Reinforcement Learning, emphasizing neural network approximations and challenges in training multiagent systems.

Learning to Find a Goal

Delves into a biologically inspired version of Reinforcement Learning, focusing on maze navigation and the implementation of spiking neurons.

Autonomous Vehicles: Trajectory Prediction and Social Behavior

Explores challenges in deep learning for autonomous vehicles, emphasizing social behavior modeling and feasible trajectory prediction.

Safe Learning and Control Log in to Mediaspace to watch this video

Explores safe learning, control, multi-agent coordination, and Nash equilibrium convergence in intelligent systems.

Related concepts (5)

Reinforcement learning

Q-learning

Temporal difference learning

Multi-agent system

Map