scholarly journals Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning

2018 ◽  
Vol 43 (1) ◽  
pp. 130-151 ◽  
Author(s):  
Prasenjit Karmakar ◽  
Shalabh Bhatnagar
Keyword(s):  
Stochastic Approximation ◽  
Time Scale ◽  
Temporal Difference ◽  
Temporal Difference Learning

Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning

2001 ◽  
Vol 13 (10) ◽  
pp. 2221-2237 ◽  
Author(s):  
Rajesh P. N. Rao ◽  
Terrence J. Sejnowski
Keyword(s):  
Cortical Neuron ◽  
Action Potentials ◽  
Learning Rule ◽  
Spike Timing ◽  
Biophysical Model ◽  
Excitatory Synapses ◽  
Temporal Difference ◽  
Temporal Difference Learning ◽  
Neocortical Neurons ◽  
Hebbian Plasticity

A spike-timing-dependent Hebbian mechanism governs the plasticity of recurrent excitatory synapses in the neocortex: synapses that are activated a few milliseconds before a postsynaptic spike are potentiated, while those that are activated a few milliseconds after are depressed. We show that such a mechanism can implement a form of temporal difference learning for prediction of input sequences. Using a biophysical model of a cortical neuron, we show that a temporal difference rule used in conjunction with dendritic backpropagating action potentials reproduces the temporally asymmetric window of Hebbian plasticity observed physiologically. Furthermore, the size and shape of the window vary with the distance of the synapse from the soma. Using a simple example, we show how a spike-timing-based temporal difference learning rule can allow a network of neocortical neurons to predict an input a few milliseconds before the input's expected arrival.

scholarly journals The serial blocking effect: a testbed for the neural mechanisms of temporal-difference learning

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Ashraf Mahmud ◽  
Petio Petrov ◽  
Guillem R. Esber ◽  
Mihaela D. Iordanova
Keyword(s):  
Blocking Effect ◽  
Neural Mechanisms ◽  
Temporal Difference ◽  
Temporal Difference Learning
Sign in / Sign up

Export Citation Format

Share Document