Lecture

Spatial Memory in RL Agents

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture explores the use of spatial and pattern memory in reinforcement learning agents through maze navigation tasks. The approach involves designing mazes of increasing complexity to test spatial orientation and path choosing abilities. The RL agent is based on a Dueling Network architecture with Double Q-learning for state updates. Results show that increasing visual landmarks improve agent performance in spatial orientation, while path choosing remains inconsistent. Limitations include the lack of environment map building and the need for better reward functions. Possible extensions involve refining starting points and orientations.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (26)

Spatial memory

In cognitive psychology and neuroscience, spatial memory is a form of memory responsible for the recording and recovery of information needed to plan a course to a location and to recall the location of an object or the occurrence of an event. Spatial memory is necessary for orientation in space. Spatial memory can also be divided into egocentric and allocentric spatial memory. A person's spatial memory is required to navigate around a familiar city. A rat's spatial memory is needed to learn the location of food at the end of a maze.

Memory

Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered, it would be impossible for language, relationships, or personal identity to develop. Memory loss is usually described as forgetfulness or amnesia. Memory is often understood as an informational processing system with explicit and implicit functioning that is made up of a sensory processor, short-term (or working) memory, and long-term memory.

Long-term memory

Long-term memory (LTM) is the stage of the Atkinson–Shiffrin memory model in which informative knowledge is held indefinitely. It is defined in contrast to short-term and working memory, which persist for only about 18 to 30 seconds. LTM is commonly labelled as "explicit memory" (declarative), as well as "episodic memory," "semantic memory," "autobiographical memory," and "implicit memory" (procedural memory). The idea of separate memories for short- and long-term storage originated in the 19th century.

Visual memory

Visual memory describes the relationship between perceptual processing and the encoding, storage and retrieval of the resulting neural representations. Visual memory occurs over a broad time range spanning from eye movements to years in order to visually navigate to a previously visited location. Visual memory is a form of memory which preserves some characteristics of our senses pertaining to visual experience. We are able to place in memory visual information which resembles objects, places, animals or people in a mental image.

False memory syndrome

In psychology, false memory syndrome (FMS) was a controversial proposed condition in which a person's identity and relationships are affected by what are believed to be false memories of psychological trauma, recollections which are strongly believed but factually contested by the accused. Peter J. Freyd originated the term partly to explain what he said was a false accusation of sexual abuse made against him by his daughter Jennifer Freyd and his False Memory Syndrome Foundation (FMSF) subsequently popularized the concept.

Related lectures (64)

Neuroscience Inspired Artificial Intelligence

Explores the historical development of deep learning, reinforcement learning, attention mechanisms, and memory systems in AI inspired by neuroscience.

Neuroscience and ML

Explores the intersection between neuroscience and machine learning, discussing deep learning, reinforcement learning, memory systems, and the future of bridging machine and human-level intelligence.

Attention and Cognitive Load

Explores attention, arousal, cognitive load, and learning strategies.

Neuroprosthetics: Hallucinations and Self Distortions

Explores hallucinations, self distortions, cognitive decline, and altered sensorimotor processing in neuroprosthetics.

Integrating Digital Educational Content