Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Three-factor rules: DeepRL1.5A

In course

CS-456: Deep reinforcement learning

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approaches, like PPO and DQN. The course will introduce student ...

Description

This lecture covers the concept of three-factor rules in policy gradient algorithms, explaining how eligibility traces are updated based on joint neuron activity and proportional to reward. It also discusses the implementation of these rules in both biological and hardware systems.

Instructors (2)

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Caglar Gulcehre

Official source

Related concepts (19)

Artificial neuron

An artificial neuron is a mathematical function conceived as a model of biological neurons, a neural network. Artificial neurons are elementary units in an artificial neural network. The artificial neuron receives one or more inputs (representing excitatory postsynaptic potentials and inhibitory postsynaptic potentials at neural dendrites) and sums them to produce an output (or , representing a neuron's action potential which is transmitted along its axon).

Chemical synapse

Chemical synapses are biological junctions through which neurons' signals can be sent to each other and to non-neuronal cells such as those in muscles or glands. Chemical synapses allow neurons to form circuits within the central nervous system. They are crucial to the biological computations that underlie perception and thought. They allow the nervous system to connect to and control other systems of the body. At a chemical synapse, one neuron releases neurotransmitter molecules into a small space (the synaptic cleft) that is adjacent to another neuron.

Postsynaptic potential

Postsynaptic potentials are changes in the membrane potential of the postsynaptic terminal of a chemical synapse. Postsynaptic potentials are graded potentials, and should not be confused with action potentials although their function is to initiate or inhibit action potentials. They are caused by the presynaptic neuron releasing neurotransmitters from the terminal bouton at the end of an axon into the synaptic cleft. The neurotransmitters bind to receptors on the postsynaptic terminal, which may be a neuron or a muscle cell in the case of a neuromuscular junction.

Excitatory synapse

An excitatory synapse is a synapse in which an action potential in a presynaptic neuron increases the probability of an action potential occurring in a postsynaptic cell. Neurons form networks through which nerve impulses travels, each neuron often making numerous connections with other cells of neurons. These electrical signals may be excitatory or inhibitory, and, if the total of excitatory influences exceeds that of the inhibitory influences, the neuron will generate a new action potential at its axon hillock, thus transmitting the information to yet another cell.

Excitatory postsynaptic potential

In neuroscience, an excitatory postsynaptic potential (EPSP) is a postsynaptic potential that makes the postsynaptic neuron more likely to fire an action potential. This temporary depolarization of postsynaptic membrane potential, caused by the flow of positively charged ions into the postsynaptic cell, is a result of opening ligand-gated ion channels. These are the opposite of inhibitory postsynaptic potentials (IPSPs), which usually result from the flow of negative ions into the cell or positive ions out of the cell.

Related lectures (39)

Understanding Synaptic Transmission

Explores synaptic transmission, neurotransmitters, and neural plasticity principles.

Modeling synaptic transmission: Stochastic Dynamics MOOC: Simulation Neurocience

Explores stochastic synaptic transmission modeling, parameters, inhibitory effects, and synaptic reliability.

Modeling the synaptic potential MOOC: Simulation Neurocience

Explores the modeling of synaptic potentials and the factors influencing their amplitude.

Untitled

Synaptic transmission: Structure and Mechanisms MOOC: Simulation Neurocience