scholarly journals Humans are primarily model-based learners in the two-stage task

2019 ◽  
Author(s):  
Carolina Feher da Silva ◽  
Todd A. Hare
Keyword(s):  
Simple Model ◽  
Habit Formation ◽  
Learning Processes ◽  
Reward Learning ◽  
Learning Models ◽  
Two Stage ◽  
Model Based ◽  
Model Free ◽  
Task Instructions ◽  
Versus Model

AbstractDistinct model-free and model-based learning processes are thought to drive both typical and dysfunctional behaviours. Data from two-stage decision tasks have seemingly shown that human behaviour is driven by both processes operating in parallel. However, in this study, we show that more detailed task instructions lead participants to make primarily model-based choices that have little, if any, simple model-free influence. We also demonstrate that behaviour in the two-stage task may falsely appear to be driven by a combination of simple model-free and model-based learning if purely model-based agents form inaccurate models of the task because of misconceptions. Furthermore, we report evidence that many participants do misconceive the task in important ways. Overall, we argue that humans formulate a wide variety of learning models. Consequently, the simple dichotomy of model-free versus model-based learning is inadequate to explain behaviour in the two-stage task and connections between reward learning, habit formation, and compulsivity.

Experimental validation of robot-assisted cardiovascular catheterization: model-based versus model-free control

2018 ◽  
Vol 13 (6) ◽  
pp. 797-804
Author(s):  
Xiaomei Wang ◽  
Kit-Hang Lee ◽  
Denny K. C. Fu ◽  
Ziyang Dong ◽  
Kui Wang ◽  
...  
Keyword(s):  
Experimental Validation ◽  
Robot Assisted ◽  
Model Based ◽  
Model Free ◽  
Versus Model ◽  
Model Free Control

scholarly journals Parallel model-based and model-free reinforcement learning for card sorting performance

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Alexander Steinke ◽  
Florian Lange ◽  
Bruno Kopp
Keyword(s):  
Reinforcement Learning ◽  
Cognitive Flexibility ◽  
Theoretical Perspective ◽  
Behavioral Flexibility ◽  
Wisconsin Card Sorting Test ◽  
Card Sorting ◽  
Learning Models ◽  
Model Based ◽  
Model Free ◽  
Reinforcement Learning Models

Abstract The Wisconsin Card Sorting Test (WCST) is considered a gold standard for the assessment of cognitive flexibility. On the WCST, repeating a sorting category following negative feedback is typically treated as indicating reduced cognitive flexibility. Therefore such responses are referred to as ‘perseveration’ errors. Recent research suggests that the propensity for perseveration errors is modulated by response demands: They occur less frequently when their commitment repeats the previously executed response. Here, we propose parallel reinforcement-learning models of card sorting performance, which assume that card sorting performance can be conceptualized as resulting from model-free reinforcement learning at the level of responses that occurs in parallel with model-based reinforcement learning at the categorical level. We compared parallel reinforcement-learning models with purely model-based reinforcement learning, and with the state-of-the-art attentional-updating model. We analyzed data from 375 participants who completed a computerized WCST. Parallel reinforcement-learning models showed best predictive accuracies for the majority of participants. Only parallel reinforcement-learning models accounted for the modulation of perseveration propensity by response demands. In conclusion, parallel reinforcement-learning models provide a new theoretical perspective on card sorting and it offers a suitable framework for discerning individual differences in latent processes that subserve behavioral flexibility.

Acute Stress Effects on Model-Based versus Model-Free Reinforcement Learning

2013 ◽  
Author(s):  
A. Ross Otto ◽  
Candace M. Raio ◽  
Elizabeth A. Phelps ◽  
Nathaniel Daw
Keyword(s):  
Reinforcement Learning ◽  
Acute Stress ◽  
Stress Effects ◽  
Model Based ◽  
Model Free ◽  
Versus Model

scholarly journals A note on the analysis of two-stage task results: how changes in task structure affect what model-free and model-based strategies predict about the effects of reward and transition on the stay probability

2017 ◽  
Author(s):  
Carolina Feher da Silva ◽  
Todd A. Hare
Keyword(s):  
Scientific Progress ◽  
Task Structure ◽  
Decision Task ◽  
Two Stage ◽  
Model Based ◽  
Model Free ◽  
Task Dependence ◽  
Design Choice ◽  
Important Means ◽  
The Impact

AbstractMany studies that aim to detect model-free and model-based influences on behavior employ two-stage behavioral tasks of the type pioneered by Daw and colleagues in 2011. Such studies commonly modify existing two-stage decision paradigms in order to better address a given hypothesis, which is an important means of scientific progress. It is, however, critical to fully appreciate the impact of any modified or novel experimental design features on the expected results. Here, we use two concrete examples to demonstrate that relatively small changes in the two-stage task design can substantially change the pattern of actions taken by model-free and model-based agents as a function of the reward outcomes and transitions on previous trials. In the first, we show that, under specific conditions, purely model-free agents will produce the reward by transition interactions typically thought to characterize model-based behavior on a two-stage task. The second example shows that model-based agents’ behavior is driven by a main effect of transition-type in addition to the canonical reward by transition interaction whenever the reward probabilities of the final states do not sum to one. Together, these examples emphasize the task-dependence of model-free and model-based behavior and highlight the benefits of using computer simulations to determine what pattern of results to expect from both model-free and model-based agents performing a given two-stage decision task in order to design choice paradigms and analysis strategies best suited to the current question.

Sign in / Sign up

Export Citation Format

Share Document