Publications by Wulfram Gerstner

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Whether we prepare a coffee or navigate to a shop: in many tasks we make multiple decisions before reaching a goal. Learning such state-action sequences from sparse reward raises the problem of credit-assignment: which actions out of a long sequence should be reinforced? One solution provided by reinforcement learning (RL) theory is the eligibility trace (ET); a decaying memory of the state-action history. Here we investigate behaviorally and neurally whether humans utilize an ET when learning a multi-step

Fitting network models to neural activity is an important tool in neuroscience. A popular approach is to model a brain area with a probabilistic recurrent spiking network whose parameters maximize the likelihood of the recorded activity. Although this is widely used, we show that the resulting model does not produce realistic neural activity. To correct for this, we suggest to augment the log-likelihood with terms that measure the dissimilarity between simulated and recorded activity. This dissimilarity is

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic pla ...

Synaptic changes induced by neural activity need to be consolidated to maintain memory over a timescale of hours. In experiments, synaptic consolidation can be induced by repeating a stimulation protocol several times and the effectiveness of consolidation ...

Notions of surprise and novelty have been used in various experimental and theoretical studies across multiple brain areas and species. However, 'surprise' and 'novelty' refer to different quantities in different studies, which raises concerns about whethe ...

Neurons in the central nervous system, and in the cortex in particular, are subject to a barrage of pulses from their presynaptic populations. These synaptic pulses are mediated by conductance changes and therefore lead to increases or decreases of the neu ...

We study analytically a model of long-term synaptic plasticity where synaptic changes are triggered by presynaptic spikes, postsynaptic spikes, and the time differences between pre- and postsynaptic spikes. We show that plasticity can lead to an intrinsic stabilization of the mean firing rate of the postsynaptic neuron. Subtractive normalization of the synaptic weights (summed over all presynaptic inputs converging on a postsynaptic neuron) follows if, in addition, the mean input rates and the mean input co

The way in which single neurons transform input into output spike trains has fundamental consequences for network coding. Theories and modeling studies based on standard Integrate-and-Fire models implicitly assume that, in response to increasingly strong i ...

Changes of synaptic connections between neurons are thought to be the physiological basis of learning. These changes can be gated by neuromodulators that encode the presence of reward. We study a family of reward-modulated synaptic learning rules for spiking neurons on a learning task in continuous space inspired by the Morris Water maze. The synaptic update rule modifies the release probability of synaptic transmission and depends on the timing of presynaptic spike arrival, postsynaptic action potentials,

Individual behavioral performance during learning is known to be affected by modulatory factors, such as stress and motivation, and by genetic predispositions that influence sensitivity to these factors. Despite numerous studies, no integrative framework i ...

A path-integral approach is developed for the analysis of spike-triggered-average quantities in neurons with voltage-gated subthreshold currents. Using a linearization procedure to reduce the models to the generalized integrate-and-fire form, analytical ex ...

An analytical model is proposed that can predict the shape of the poststimulus time histogram (PSTH) response to a current pulse of a neuron subjected to uncorrelated background input. The model is based on an explicit description of noise in the form of an escape rate and corresponding hazard function. Two forms of the model are presented. The full model is nonlinear and can be integrated numerically while a linearized version, while the linearized version can be solved analytically. In the linearized vers

Advancements in memristive devices have given rise to a new generation of specialized hardware for bio-inspired computing. However, most of these implementations draw only partial inspiration from the architecture and functionalities of the mammalian brain. Moreover, the use of memristive hardware is typically restricted to specific elements within the learning algorithm, leaving computationally expensive operations to be executed in software. Here, we demonstrate reinforcement learning through an actor-cri

Finite-sized populations of spiking elements are fundamental to brain function but also are used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasirenewal description of neurons ...

Modeling the temporal unfolding of musical events and its interpretation in terms of hierarchical relations is a common theme in music theory, cognition, and composition. To faithfully encode such relations, we need an elegant way to represent both the semantics of prolongation, where a single event is elaborated into multiple events, and process, where the connection from one event to another is elaborated into multiple connections. In existing works, trees are used to capture the former and graphs for the

Dimensionality reduction methods are widely used in neuroscience to investigate two complementary aspects of neural activity: the distribution of single-neuron functional properties and the low-dimensional collective dynamics of population activity. However, how do these two aspects of neural activity relate to the structure of the underlying neural circuit? In this work, we connect circuit structure, single-neuron functional properties, and emerging low-dimensional dynamics in spiking recurrent network mod

For simulations of large spiking neuron networks, an accurate, simple and versatile single-neuron modeling framework is required. Here we explore the versatility of a simple two-equation model: the adaptive exponential integrate-and-fire neuron. We show th ...

We analyse a model for the development of orientation-selective receptive fields of simple cells in a locally connected network of cortical neurons. The Hebbian learning rule that underlies the development is described by a linear differential equation. The structure of the emerging cortical map can be predicted by deriving the eigenfunctions corresponding to the leading eigenvalues of the associated matrix. We show that the receptive fields have the typical form of a wavelet. Mathematically, receptive fiel

Mensi S, Naud R, Pozzorini C, Avermann M, Petersen CCH, Gerstner W. Parameter extraction and classification of three cortical neuron types reveals two distinct adaptation mechanisms. J Neurophysiol 107: 1756-1775, 2012. First published December 7, 2011; do ...

Spike Timing Dependent Plasticity (STDP) is a temporally asymmetric form of Hebbian learning induced by tight temporal correlations between the spikes of pre- and postsynaptic neurons. As with other forms of synaptic plasticity, it is widely believed that ...

Wulfram Gerstner

Chat with Graph Search

Evidence for eligibility traces in human learning

Fitting summary statistics of neural data with a differentiable spiking network simulator

Reinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons

Optimal Stimulation Protocol in a Bistable Synaptic Consolidation Model

Surprise and novelty in the brain

Statistics of subthreshold neuronal voltage fluctuations due to conductance-based synaptic shot noise

Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning

Enhanced Sensitivity to Rapid Input Fluctuations by Nonlinear Threshold Dynamics in Neocortical Pyramidal Neurons

Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail

Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning

Spike-triggered averages for passive and resonant neurons receiving filtered excitatory and inhibitory synaptic drive

Noise and the PSTH response to current transients: I. General theory and application to the integrate-and-fire neuron

Actor-Critic Networks with Analogue Memristors Mimicking Reward-Based Learning

Fluctuations and information filtering in coupled populations of spiking neurons with adaptation

Music as Flow: A Formal Representation of Hierarchical Processes in Music

Linking neural manifolds to circuit structure in recurrent networks

Firing patterns in the adaptive exponential integrate-and-fire model

Analysis of a correlation-based model for the development of orientation-selective receptive fields in the visual cortex

Parameter extraction and classification of three cortical neuron types reveals two distinct adaptation mechanisms

Spike-timing dependent plasticity