Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Learning to Find a Goal

In course

CS-456: Deep reinforcement learning

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approaches, like PPO and DQN. The course will introduce student ...

Description

This lecture explores a biologically inspired version of the classic task of Reinforcement Learning, where the instructor discusses the hippocampus, place cells in the rat brain, the Morris Water Maze, modeling a maze task, maze navigation with Actor-Critic, and the implementation of spiking neurons. The lecture concludes with a summary linking TD learning in an actor-critic framework to brain functions and showcasing the possibility of learning in a few trials with well-adapted sensory representation.

Instructors (2)

Caglar Gulcehre

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Official source

Ontological neighbourhood

Biology

Cell biology: Neurotransmission

Related concepts (32)

Medium spiny neuron

Medium spiny neurons (MSNs), also known as spiny projection neurons (SPNs), are a special type of GABAergic inhibitory cell representing 95% of neurons within the human striatum, a basal ganglia structure. Medium spiny neurons have two primary phenotypes (characteristic types): D1-type MSNs of the direct pathway and D2-type MSNs of the indirect pathway. Most striatal MSNs contain only D1-type or D2-type dopamine receptors, but a subpopulation of MSNs exhibit both phenotypes.

Reinforcement learning

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected.

Spatial memory

In cognitive psychology and neuroscience, spatial memory is a form of memory responsible for the recording and recovery of information needed to plan a course to a location and to recall the location of an object or the occurrence of an event. Spatial memory is necessary for orientation in space. Spatial memory can also be divided into egocentric and allocentric spatial memory. A person's spatial memory is required to navigate around a familiar city. A rat's spatial memory is needed to learn the location of food at the end of a maze.

Striatum

The striatum, or corpus striatum (also called the striate nucleus), is a nucleus (a cluster of neurons) in the subcortical basal ganglia of the forebrain. The striatum is a critical component of the motor and reward systems; receives glutamatergic and dopaminergic inputs from different sources; and serves as the primary input to the rest of the basal ganglia. Functionally, the striatum coordinates multiple aspects of cognition, including both motor and action planning, decision-making, motivation, reinforcement, and reward perception.

Ventral tegmental area

The ventral tegmental area (VTA) (tegmentum is Latin for covering), also known as the ventral tegmental area of Tsai, or simply ventral tegmentum, is a group of neurons located close to the midline on the floor of the midbrain. The VTA is the origin of the dopaminergic cell bodies of the mesocorticolimbic dopamine system and other dopamine pathways; it is widely implicated in the drug and natural reward circuitry of the brain.

Related lectures (37)

Reward-based Learning: Cellular Mechanisms MOOC: Cellular Mechanisms of Brain Function

Explores reward-based learning in neuronal networks and the role of dopamine in synaptic plasticity.

GABAergic Projections: Cellular Mechanisms of Brain Function MOOC: Cellular Mechanisms of Brain Function

Explores GABAergic projection neurons in the brain and their role in brain function, including dopamine's impact on Parkinson's disease.

Parkinson's Disease: Basal Ganglia Circuitry

Explores the basal ganglia circuitry in Parkinson's disease, covering structure, function, dopamine role, pathology, and treatments.

Surprise, Curiosity and Reward: An Evolutionary Perspective

Explores the evolutionary perspective on surprise, curiosity, and reward, focusing on the role of primary and secondary reward signals.

Attractor Networks and Spiking Neurons MOOC: Neuronal Dynamics 2- Computational Neuroscience: Neuronal Dynamics of Cognition