Lecture

Learning to Find a Goal

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-456: Deep reinforcement learning

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approaches, like PPO and DQN. The course will introduce student ...

Description

This lecture explores a biologically inspired version of the classic task of Reinforcement Learning, where the instructor discusses the hippocampus, place cells in the rat brain, the Morris Water Maze, modeling a maze task, maze navigation with Actor-Critic, and the implementation of spiking neurons. The lecture concludes with a summary linking TD learning in an actor-critic framework to brain functions and showcasing the possibility of learning in a few trials with well-adapted sensory representation.

Instructors (2)

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Caglar Gulcehre

Teaching & PhD PhD Students Tim Ruben Davidson, Skander Moalla, Anja Surina, Xiuying Wei, Liangze Jiang, Mikhail Terekhov, Amin Mansouri, Justin Samuel Deschenaux, Yitao Xu Courses Training Large Language Models EE-628 This PhD-level course dives deep into the training of Large Language Models (LLMs), focusing on the complementary roles of datasets, pre-training and post training methodologies in shaping model performance and scalability.

Official source

Ontological neighbourhood

Biology

Cell biology: Neurotransmission

Related concepts (32)

Medium spiny neuron

Medium spiny neurons (MSNs), also known as spiny projection neurons (SPNs), are a special type of GABAergic inhibitory cell representing 95% of neurons within the human striatum, a basal ganglia structure. Medium spiny neurons have two primary phenotypes (characteristic types): D1-type MSNs of the direct pathway and D2-type MSNs of the indirect pathway. Most striatal MSNs contain only D1-type or D2-type dopamine receptors, but a subpopulation of MSNs exhibit both phenotypes.

Striatum

The striatum, or corpus striatum (also called the striate nucleus), is a nucleus (a cluster of neurons) in the subcortical basal ganglia of the forebrain. The striatum is a critical component of the motor and reward systems; receives glutamatergic and dopaminergic inputs from different sources; and serves as the primary input to the rest of the basal ganglia. Functionally, the striatum coordinates multiple aspects of cognition, including both motor and action planning, decision-making, motivation, reinforcement, and reward perception.

Ventral tegmental area

The ventral tegmental area (VTA) (tegmentum is Latin for covering), also known as the ventral tegmental area of Tsai, or simply ventral tegmentum, is a group of neurons located close to the midline on the floor of the midbrain. The VTA is the origin of the dopaminergic cell bodies of the mesocorticolimbic dopamine system and other dopamine pathways; it is widely implicated in the drug and natural reward circuitry of the brain.

Morris water navigation task

The Morris water navigation task, also known as the Morris water maze (not to be confused with water maze), is a behavioral procedure mostly used with rodents. It is widely used in behavioral neuroscience to study spatial learning and memory. It enables learning, memory, and spatial working to be studied with great accuracy, and can also be used to assess damage to particular cortical regions of the brain.

Basal ganglia

The basal ganglia (BG), or basal nuclei, are a group of subcortical nuclei, of varied origin, in the brains of vertebrates. In humans, and some primates, there are some differences, mainly in the division of the globus pallidus into an external and internal region, and in the division of the striatum. The basal ganglia are situated at the base of the forebrain and top of the midbrain. Basal ganglia are strongly interconnected with the cerebral cortex, thalamus, and brainstem, as well as several other brain areas.

Related lectures (64)

Parkinson's Disease: Clinical Insights and Treatment Approaches

Covers Parkinson's disease, its symptoms, diagnosis challenges, and current treatment options, emphasizing the need for early biomarkers and disease-modifying therapies.

Reward-based Learning: Cellular Mechanisms MOOC: Cellular Mechanisms of Brain Function

Explores reward-based learning in neuronal networks and the role of dopamine in synaptic plasticity.

Neuroscience Inspired Artificial Intelligence

Explores the historical development of deep learning, reinforcement learning, attention mechanisms, and memory systems in AI inspired by neuroscience.

Neuromorphic Computing: Concepts and Hardware Implementations

Covers neuromorphic computing, challenges in ternary and binary computing, hardware simulations of the brain, and new materials for artificial brain cells.

Spike-Timing Models of plasticity MOOC: Neuronal Dynamics 2- Computational Neuroscience: Neuronal Dynamics of Cognition