Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007)

Add this entry.

Bookmark this on Yahoo Bookmark
Bookmark this on Delicious
Bookmark this on Google Bookmarks
Bookmark this on BuzzURL
Share on StumbleUpon
Share on GREE
Bookmark this on Livedoor Clip
Add Browser Bookmark

vDesktop Size:

Contributor:NIJC Administrator (nijcadmin)  |  Created Date: Mar 16, 2011 16:27:37  |  Visits:4729  |  Runs:115
Free Keywords: STDP, Dopamine, Reward, Conditioning, Network model, Spiking neuron
ID 119
Title Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007)
Free Keywords STDP, Dopamine, Reward, Conditioning, Network model, Spiking neuron
Description This is the readme a model associated with the publication:

Izhikevich E.M. (2007) Solving the Distal Reward Problem through
Linkage of STDP and Dopamine Signaling. Cerebral Cortex,


In Pavlovian and instrumental conditioning, reward typically comes
seconds after reward-triggering actions, creating an explanatory
conundrum known as "distal reward problem": How does the brain know
what firing patterns of what neurons are responsible for the reward if
1) the patterns are no longer there when the reward arrives and 2) all
neurons and synapses are active during the waiting period to the
reward? Here, we show how the conundrum is resolved by a model network
of cortical spiking neurons with spike-timing-dependent plasticity
(STDP) modulated by dopamine (DA). Although STDP is triggered by
nearly coincident firing patterns on a millisecond timescale, slow
kinetics of subsequent synaptic plasticity is sensitive to changes in
the extracellular DA concentration during the critical period of a few
seconds. Random firings during the waiting period to the reward do not
affect STDP and hence make the network insensitive to the ongoing
activity-the key feature that distinguishes our approach from previous
theoretical studies, which implicitly assume that the network be quiet
during the waiting period or that the patterns be preserved until the
reward arrives. This study emphasizes the importance of precise firing
patterns in brain dynamics and suggests how a global diffusive
reinforcement signal in the form of extracellular DA can selectively
influence the right synapses at the right time.

Usage: The model replicates figure 1 in the paper.
Simple start matlab, cd to the daspnet directory and run the matlab
program by typing daspnet in the command window. The program then
begins drawing especially figure 1d, synaptic strength vs time as the
1000 neuron network is run.

n1 - the presynaptic neuron. syn is the synapse to be reinforced.
Plot: top - spike raster. Bottom left - synaptic strength (blue), the
eligibility trace (green), and the rewards (red x). Bottom right - the
distribution of synaptic weights with the chosen synapse marked by red dot.
Last Modified Date Mar 22, 2013 18:45:47
Created Date Mar 16, 2011 16:27:37
Contributor NIJC Administrator (nijcadmin)
Item Type Online Simulation
Change Log(History)
Mar 22, 2013 Modified; Contents URL.
Feb 12, 2012 Modified; Description, Free Keywords, Simulator Name.
Mar 24, 2011 Modified; Index.
Mar 23, 2011 Modified; Preview, Simulator.
Model Contents URL
Model Site Name ModelDB
Simulator Name MATLAB
Simulator Version
/ Public / Title / K-N

Online Users
8 user(s) are online (6 user(s) are browsing Database)

Members: 0
Guests: 8

Related Items