Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007)


Bookmark this on Yahoo Bookmark
Bookmark this on Delicious
Bookmark this on Google Bookmarks
Bookmark this on BuzzURL
Share on StumbleUpon
Share on GREE
Bookmark this on Livedoor Clip
Add Browser Bookmark

vDesktop Size:

登録者:NIJC管理者 (nijcadmin)  |  作成日: Mar 16, 2011 16:27:37  |  来訪者数:4755  |  実行回数:118
フリーキーワード: STDP, Dopamine, Reward, Conditioning, Network model, Spiking neuron
ID 119
タイトル Linking STDP and Dopamine action to solve the distal reward problem (Izhikevich 2007)
フリーキーワード STDP, Dopamine, Reward, Conditioning, Network model, Spiking neuron
概要 This is the readme a model associated with the publication:

Izhikevich E.M. (2007) Solving the Distal Reward Problem through
Linkage of STDP and Dopamine Signaling. Cerebral Cortex,


In Pavlovian and instrumental conditioning, reward typically comes
seconds after reward-triggering actions, creating an explanatory
conundrum known as "distal reward problem": How does the brain know
what firing patterns of what neurons are responsible for the reward if
1) the patterns are no longer there when the reward arrives and 2) all
neurons and synapses are active during the waiting period to the
reward? Here, we show how the conundrum is resolved by a model network
of cortical spiking neurons with spike-timing-dependent plasticity
(STDP) modulated by dopamine (DA). Although STDP is triggered by
nearly coincident firing patterns on a millisecond timescale, slow
kinetics of subsequent synaptic plasticity is sensitive to changes in
the extracellular DA concentration during the critical period of a few
seconds. Random firings during the waiting period to the reward do not
affect STDP and hence make the network insensitive to the ongoing
activity-the key feature that distinguishes our approach from previous
theoretical studies, which implicitly assume that the network be quiet
during the waiting period or that the patterns be preserved until the
reward arrives. This study emphasizes the importance of precise firing
patterns in brain dynamics and suggests how a global diffusive
reinforcement signal in the form of extracellular DA can selectively
influence the right synapses at the right time.

Usage: The model replicates figure 1 in the paper.
Simple start matlab, cd to the daspnet directory and run the matlab
program by typing daspnet in the command window. The program then
begins drawing especially figure 1d, synaptic strength vs time as the
1000 neuron network is run.

n1 - the presynaptic neuron. syn is the synapse to be reinforced.
Plot: top - spike raster. Bottom left - synaptic strength (blue), the
eligibility trace (green), and the rewards (red x). Bottom right - the
distribution of synaptic weights with the chosen synapse marked by red dot.
最終更新日 Mar 22, 2013 18:45:47
作成日 Mar 16, 2011 16:27:37
登録者 NIJC管理者 (nijcadmin)
アイテムタイプ Online Simulation
Mar 22, 2013 Modified; Contents URL.
Feb 12, 2012 Modified; Description, Free Keywords, Simulator Name.
Mar 24, 2011 Modified; Index.
Mar 23, 2011 Modified; Preview, Simulator.
登録PF・DB ModelDB
シミュレータ MATLAB
/ Public / Title / K-N

2 人のユーザが現在オンラインです。 (2 人のユーザが データベース を参照しています。)

登録ユーザ: 0
ゲスト: 2