The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution

2020 ◽  
Author(s):  
Jonathan E. Rubin ◽  
Catalina Vich ◽  
Matthew Clapp ◽  
Kendra Noneman ◽  
Timothy Verstynen
Keyword(s):  
Basal Ganglia ◽  
Assignment Problem ◽  
Credit Assignment

scholarly journals Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia

2006 ◽  
Vol 18 (2) ◽  
pp. 283-328 ◽  
Author(s):  
Randall C. O'Reilly ◽  
Michael J. Frank
Keyword(s):  
Working Memory ◽  
Prefrontal Cortex ◽  
Basal Ganglia ◽  
Computational Model ◽  
Computational Models ◽  
Memory Task ◽  
Learning Mechanisms ◽  
Subcortical Structures ◽  
Credit Assignment ◽  
Gating Mechanism

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.

Sign in / Sign up

Export Citation Format

Share Document