scholarly journals Addiction beyond pharmacological effects: the role of environment complexity and bounded rationality

2017 ◽  
Author(s):  
Dimitri Ognibene ◽  
Vincenzo G. Fiore ◽  
Xiaosi Gu
Keyword(s):  
Bounded Rationality ◽  
Cognitive Resources ◽  
Environmental Complexity ◽  
Model Based ◽  
Model Free ◽  
Situation Recognition ◽  
Underlying Causes ◽  
Reinforcement Learning Model ◽  
Substances Of Abuse

AbstractSeveral decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a duallearning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g. caused by stress) in the bounded rationality, i.e. the limited amount of resources available for the model-based component, in turn increasing the agent’s chances to develop or maintain addictive behaviours. Our study demonstrates that, independent of drug consumption, addictive behaviours can arise in the interaction between the environmental complexity and the biologically finite resources available to explore and represent it.

scholarly journals Testing the Possibility of Model-based Pavlovian Control of Attention to Threat

2019 ◽  
Vol 31 (1) ◽  
pp. 36-48 ◽  
Author(s):  
Deborah Talmi ◽  
Martina Slapkova ◽  
Matthias J. Wieser
Keyword(s):  
Attention Capture ◽  
Conditioned Stimuli ◽  
Learning Stage ◽  
Internal Representations ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Role Of Information ◽  
Instrumental Control

Signals for reward or punishment attract attention preferentially, a principle termed value-modulated attention capture (VMAC). The mechanisms that govern the allocation of attention can be described with a terminology that is more often applied to the control of overt behaviors, namely, the distinction between instrumental and Pavlovian control, and between model-free and model-based control. Although instrumental control of VMAC can be either model-free or model-based, it is not known whether Pavlovian control of VMAC can be model-based. To decide whether this is possible, we measured steady-state visual evoked potentials (SSVEPs) while 20 healthy adults took part in a novel task. During the learning stage, participants underwent aversive threat conditioning with two conditioned stimuli (CSs): one that predicted pain (CS+) and one that predicted safety (CS−). Instructions given before the test stage allowed participants to infer whether novel, ambiguous CSs (new_CS+/new_CS−) were threatening or safe. Correct inference required combining stored internal representations and new propositional information, the hallmark of model-based control. SSVEP amplitudes quantified the amount of attention allocated to novel CSs on their very first presentation, before they were ever reinforced. We found that SSVEPs were higher for new_CS+ than new_CS−. This result is potentially indicative of model-based Pavlovian control of VMAC, but additional controls are necessary to verify this conclusively. This result underlines the potential transformative role of information and inference in emotion regulation.

scholarly journals Model-based Pavlovian control of attention to threat

2018 ◽  
Author(s):  
D Talmi ◽  
M Slapkova ◽  
MJ Wieser
Keyword(s):  
Task Design ◽  
Attention Capture ◽  
Learning Stage ◽  
Internal Representations ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Role Of Information ◽  
Instrumental Control

AbstractSignals for reward or punishment attract attention preferentially, a principle termed ‘value-modulated attention capture’ (VMAC). The mechanisms that govern the allocation of attention resources can be productively described with a terminology that is more often applied to the control of overt behaviours, namely, the distinction between instrumental and Pavlovian control, and between model-free and model-based control. While instrumental control of VMAC can be either model-free or model-based, it is not known whether Pavlovian control of VMAC can be model-based. To decide whether this is possible we measured Steady-State Visual Evoked Potentials (SSVEPs) while 20 healthy adults took part in a novel task. During the learning stage participants underwent aversive threat conditioning with two CSs, one that predicted pain (CS+) and one that predicted safety (CS-). Instructions given prior to the test stage in the task allowed participants to infer whether novel, ambiguous CSs (new CS+/ new CS-) were threatening or safe. Correct inference required combining stored internal representations and new propositional information, the hallmark of model-based control. SSVEP amplitudes quantified the amount of attention allocated to novel CSs on their very first presentation, before they were ever reinforced. We found that SSVEPs were higher for new CS+ than new CS-. Because task design precluded model-free or instrumental control this result demonstrates a model-based Pavlovian control of VMAC. It confirms, in the domain of internal resource allocation, the model-based Pavlovian control of incentive behaviour and underlines the potential transformative role of information as an emotion regulation technique.

Investigating the Role of Model-Based and Model-Free Imaging Biomarkers as Early Predictors of Neoadjuvant Breast Cancer Therapy Outcome

2019 ◽  
Vol 23 (5) ◽  
pp. 1834-1843 ◽  
Author(s):  
Eleftherios Kontopodis ◽  
Maria Venianaki ◽  
Georgios C. Manikis ◽  
Katerina Nikiforaki ◽  
Ovidio Salvetti ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Cancer Therapy ◽  
Therapy Outcome ◽  
Imaging Biomarkers ◽  
Breast Cancer Therapy ◽  
Model Based ◽  
Model Free ◽  
Early Predictors

scholarly journals Task complexity interacts with state-space uncertainty in the arbitration process between model-based and model-free reinforcement-learning at both behavioral and neural levels

2018 ◽  
Author(s):  
Dongjae Kim ◽  
Geon Yeong Park ◽  
John P. O’Doherty ◽  
Sang Wan Lee
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
State Space ◽  
Task Complexity ◽  
Model Based ◽  
Model Free ◽  
Arbitration Process ◽  
Two Systems ◽  
The Brain

SUMMARYA major open question concerns how the brain governs the allocation of control between two distinct strategies for learning from reinforcement: model-based and model-free reinforcement learning. While there is evidence to suggest that the reliability of the predictions of the two systems is a key variable responsible for the arbitration process, another key variable has remained relatively unexplored: the role of task complexity. By using a combination of novel task design, computational modeling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process between model-based and model-free RL. We found evidence to suggest that task complexity plays a role in influencing the arbitration process alongside state-space uncertainty. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex bilaterally. These findings provide insight into how the inferior prefrontal cortex negotiates the trade-off between model-based and model-free RL in the presence of uncertainty and complexity, and more generally, illustrates how the brain resolves uncertainty and complexity in dynamically changing environments.SUMMARY OF FINDINGS- Elucidated the role of state-space uncertainty and complexity in model-based and model-free RL.- Found behavioral and neural evidence for complexity-sensitive prefrontal arbitration.- High task complexity induces explorative model-based RL.

scholarly journals Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Dongjae Kim ◽  
Geon Yeong Park ◽  
John P. O′Doherty ◽  
Sang Wan Lee
Keyword(s):  
State Space ◽  
Task Complexity ◽  
Behavioral Control ◽  
Computational Modelling ◽  
Model Based ◽  
Model Free ◽  
Neural Representations ◽  
Arbitration Process ◽  
Fmri Analysis

AbstractIt has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex.

scholarly journals Valence bias in metacontrol of decision making in adolescents and young adults

2020 ◽  
Author(s):  
Florian Bolenz ◽  
Ben Eppinger
Keyword(s):  
Decision Making ◽  
Young Adults ◽  
Young Adulthood ◽  
Framing Effects ◽  
Developmental Differences ◽  
Adolescents And Young Adults ◽  
Cognitive Resources ◽  
Model Based ◽  
Model Free ◽  
Outcome Valence

Metacontrol refers to the human ability to dynamically adapt decision-making strategies to changes in internal and external demands. In this study, we investigated the development of metacontrol from adolescence into young adulthood as well as developmental differences in the sensitivity of metacontrol to framing effects. Adolescents and young adults were assessed with a decision-making task that dissociates model-free and model-based decision-making strategies. In this task, we manipulated outcome magnitude and outcome valence, i.e. the framing of outcomes. With increasing age, we found a greater adaptation of model-based decision making to outcome magnitudes. Model-based decision making was more pronounced for loss compared to gain frames but this framing effect did not differ with age. Our findings suggest that metacontrol continues to develop into young adulthood. While losses generally increase the motivation to invest cognitive resources into an effortful decision-making strategy, the development of metacontrol is not sensitive to framing effects.

scholarly journals Model-free RL or action sequences?

2019 ◽  
Author(s):  
Adam Morris ◽  
Fiery Andrews Cushman
Keyword(s):  
Decision Making ◽  
Computational Models ◽  
Empirical Support ◽  
Central Position ◽  
Model Based ◽  
Model Free ◽  
Action Sequences ◽  
Alternate Model ◽  
Selection Of

The alignment of habits with model-free reinforcement learning (MF RL) is a success story for computational models of decision making, and MF RL has been applied to explain phasic dopamine responses, working memory gating, drug addiction, moral intuitions, and more. Yet, the role of MF RL has recently been challenged by an alternate model---model-based selection of chained action sequences---that produces similar behavioral and neural patterns. Here, we present two experiments that dissociate MF RL from this prominent alternative, and present unconfounded empirical support for the role of MF RL in human decision making. Our results also demonstrate that people are simultaneously using model-based selection of action sequences, thus demonstrating two distinct mechanisms of habitual control in a common experimental paradigm. These findings clarify the nature of habits and help solidify MF RL's central position in models of human behavior.

Representation, abstraction, and simple-minded sophisticates

2020 ◽  
Vol 43 ◽  
Author(s):  
Peter Dayan
Keyword(s):  
Decision Theory ◽  
Predictive Coding ◽  
Bayesian Decision Theory ◽  
Bayesian Decision ◽  
Model Based ◽  
Model Free

Abstract Bayesian decision theory provides a simple formal elucidation of some of the ways that representation and representational abstraction are involved with, and exploit, both prediction and its rather distant cousin, predictive coding. Both model-free and model-based methods are involved.

Sign in / Sign up

Export Citation Format

Share Document