On the Existence of Fixed Points for Approximate Value Iteration and Temporal-Difference Learning

2000 ◽  
Vol 105 (3) ◽  
pp. 589-608 ◽  
Author(s):  
D. P. De Farias ◽  
B. Van Roy
Keyword(s):  
Fixed Points ◽  
Temporal Difference ◽  
Value Iteration ◽  
Temporal Difference Learning ◽  
Approximate Value Iteration

Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning

2001 ◽  
Vol 13 (10) ◽  
pp. 2221-2237 ◽  
Author(s):  
Rajesh P. N. Rao ◽  
Terrence J. Sejnowski
Keyword(s):  
Cortical Neuron ◽  
Action Potentials ◽  
Learning Rule ◽  
Spike Timing ◽  
Biophysical Model ◽  
Excitatory Synapses ◽  
Temporal Difference ◽  
Temporal Difference Learning ◽  
Neocortical Neurons ◽  
Hebbian Plasticity

A spike-timing-dependent Hebbian mechanism governs the plasticity of recurrent excitatory synapses in the neocortex: synapses that are activated a few milliseconds before a postsynaptic spike are potentiated, while those that are activated a few milliseconds after are depressed. We show that such a mechanism can implement a form of temporal difference learning for prediction of input sequences. Using a biophysical model of a cortical neuron, we show that a temporal difference rule used in conjunction with dendritic backpropagating action potentials reproduces the temporally asymmetric window of Hebbian plasticity observed physiologically. Furthermore, the size and shape of the window vary with the distance of the synapse from the soma. Using a simple example, we show how a spike-timing-based temporal difference learning rule can allow a network of neocortical neurons to predict an input a few milliseconds before the input's expected arrival.

Sign in / Sign up

Export Citation Format

Share Document