scholarly journals Precision weighting of cortical unsigned prediction errors is mediated by dopamine and benefits learning

2018 ◽  
Author(s):  
J. Haarsma ◽  
P.C. Fletcher ◽  
H. Ziauddeen ◽  
T.J. Spencer ◽  
K.M.J. Diederen ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Predictive Coding ◽  
Learning Task ◽  
Anterior Cingulate ◽  
Dopamine Antagonist ◽  
Prediction Errors ◽  
Data Set ◽  
Health And Illness ◽  
Hierarchical Bayesian Inference

AbstractThe predictive coding framework construes the brain as performing a specific form of hierarchical Bayesian inference. In this framework the precision of cortical unsigned prediction error (surprise) signals is proposed to play a key role in learning and decision-making, and to be controlled by dopamine. To test this hypothesis, we re-analysed an existing data-set from healthy individuals who received a dopamine agonist, antagonist or placebo and who performed an associative learning task under different levels of outcome precision. Computational reinforcement-learning modelling of behaviour provided support for precision-weighting of unsigned prediction errors. Functional MRI revealed coding of unsigned prediction errors relative to their precision in bilateral superior frontal gyri and dorsal anterior cingulate. Cortical precision-weighting was (i) perturbed by the dopamine antagonist sulpiride, and (ii) associated with task performance. These findings have important implications for understanding the role of dopamine in reinforcement learning and predictive coding in health and illness.

scholarly journals Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning

2021 ◽  
Author(s):  
Daniel Martins ◽  
Patricia Lockwood ◽  
Jo Cutler ◽  
Rosalyn J. Moran ◽  
Yannis Paloyelis
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Anterior Cingulate ◽  
Best Interests ◽  
Prediction Errors ◽  
Subgenual Anterior Cingulate Cortex ◽  
How People Learn ◽  
Dose Dependent ◽  
Different Doses

Humans often act in the best interests of others. However, how we learn which actions result in good outcomes for other people and the neurochemical systems that support this "prosocial learning" remain poorly understood. Using computational models of reinforcement learning, functional magnetic resonance imaging and dynamic causal modelling, we examined how different doses of intranasal oxytocin, a neuropeptide linked to social cognition, impact how people learn to benefit others (prosocial learning) and whether this influence could be dissociated from how we learn to benefit ourselves (self-oriented learning). We show that a low dose of oxytocin prevented decreases in prosocial performance over time, despite no impact on self-oriented learning. Critically, oxytocin produced dose-dependent changes in the encoding of prediction errors (PE) in the midbrain-subgenual anterior cingulate cortex (sgACC) pathway specifically during prosocial learning. Our findings reveal a new role of oxytocin in prosocial learning by modulating computations of PEs in the midbrain-sgACC pathway.

scholarly journals Skipping a Beat: Heartbeat-Evoked Potentials Reflect Predictions during Interoceptive-Exteroceptive Integration

2020 ◽  
Vol 1 (1) ◽  
Author(s):  
Leah Banellis ◽  
Damian Cruse
Keyword(s):  
Predictive Coding ◽  
Auditory Stimuli ◽  
Prediction Errors ◽  
Self Awareness ◽  
Early Effect ◽  
Temporal Cues ◽  
Heartbeat Evoked Potential ◽  
External Signals ◽  
Potential Tool

Abstract Several theories propose that emotions and self-awareness arise from the integration of internal and external signals and their respective precision-weighted expectations. Supporting these mechanisms, research indicates that the brain uses temporal cues from cardiac signals to predict auditory stimuli and that these predictions and their prediction errors can be observed in the scalp heartbeat-evoked potential (HEP). We investigated the effect of precision modulations on these cross-modal predictive mechanisms, via attention and interoceptive ability. We presented auditory sequences at short (perceived synchronous) or long (perceived asynchronous) cardio-audio delays, with half of the trials including an omission. Participants attended to the cardio-audio synchronicity of the tones (internal attention) or the auditory stimuli alone (external attention). Comparing HEPs during omissions allowed for the observation of pure predictive signals, without contaminating auditory input. We observed an early effect of cardio-audio delay, reflecting a difference in heartbeat-driven expectations. We also observed a larger positivity to the omissions of sounds perceived as synchronous than to the omissions of sounds perceived as asynchronous when attending internally only, consistent with the role of attentional precision for enhancing predictions. These results provide support for attentionally modulated cross-modal predictive coding and suggest a potential tool for investigating its role in emotion and self-awareness.

The role of arousal in predictive coding

2016 ◽  
Vol 39 ◽  
Author(s):  
Fernando Ferreira-Santos
Keyword(s):  
Predictive Models ◽  
Predictive Coding ◽  
Prediction Errors

AbstractWithin a predictive coding approach, the arousal/norepinephrine effects described by the GANE (glutamate amplifies noradrenergic effects) model seem to modulate the precision attributed to prediction errors, favoring the selective updating of predictive models with larger prediction errors. However, to explain how arousal effects are triggered, it is likely that different kinds of prediction errors (including interoceptive/affective) need to be considered.

scholarly journals Precision-weighting of superior frontal cortex unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis

2019 ◽  
Author(s):  
J. Haarsma ◽  
P.C. Fletcher ◽  
J.D. Griffin ◽  
H.J. Taverne ◽  
H. Ziauddeen ◽  
...  
Keyword(s):  
Prediction Error ◽  
Computational Modelling ◽  
Anterior Cingulate ◽  
Prediction Errors ◽  
Healthy Individuals ◽  
Cortical Function ◽  
Dopaminergic Modulation ◽  
Hierarchical Bayesian Inference ◽  
The Brain ◽  
Superior Frontal Cortex

AbstractRecent theories of cortical function construe the brain as performing hierarchical Bayesian inference. According to these theories, the precision of cortical unsigned prediction error (i.e., surprise) signals plays a key role in learning and decision-making, to be controlled by dopamine, and to contribute to the pathogenesis of psychosis. To test these hypotheses, we studied learning with variable outcome-precision in healthy individuals after dopaminergic modulation and in patients with early psychosis. Behavioural computational modelling indicated that precision-weighting of unsigned prediction errors benefits learning in health, and is impaired in psychosis. FMRI revealed coding of unsigned prediction errors relative to their precision in bilateral superior frontal gyri and dorsal anterior cingulate, which was perturbed by dopaminergic modulation, impaired in psychosis, and associated with task performance and schizotypy. We conclude that precision-weighting of cortical prediction error signals is a key mechanism through which dopamine modulates inference and contributes to the pathogenesis of psychosis.

scholarly journals Linking the neural basis of hierarchical prediction with statistical learning: The paradox of attention

2021 ◽  
Author(s):  
Julie M. Schneider ◽  
Yi-Lun Weng ◽  
Anqi Hu ◽  
Zhenghan Qi
Keyword(s):  
Statistical Learning ◽  
Learning Task ◽  
Prediction Errors ◽  
Neural Basis ◽  
Attentional Shift ◽  
Cortical Responses ◽  
Distributional Information ◽  
Segmentation Task ◽  
Dependent Learning

Statistical learning, the process of tracking distributional information and discovering embedded patterns, is traditionally regarded as a form of implicit learning. However, recent studies proposed that both implicit (attention-independent) and explicit (attention-dependent) learning systems are involved in statistical learning. To understand the role of attention in statistical learning, the current study investigates the cortical processing of prediction errors in speech based on either local or global distributional information. We then ask how these cortical responses relate to statistical learning behavior in a word segmentation task. We found ERP evidence of pre-attentive processing of both the local (mismatching negativity) and global distributional information (late discriminative negativity). However, as speech elements became less frequent and more surprising, some participants showed an involuntary attentional shift, reflected in a P3a response. Individuals who displayed attentive neural tracking of distributional information showed faster learning in a speech statistical learning task. These results provide important neural evidence elucidating the facilitatory role of attention in statistical learning.

scholarly journals Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures

2018 ◽  
Author(s):  
Samuel D. McDougle ◽  
Peter A. Butcher ◽  
Darius Parvin ◽  
Fasial Mushtaq ◽  
Yael Niv ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Learning Task ◽  
Prediction Errors ◽  
Action Execution ◽  
Model Driven ◽  
Assignment Of Credit ◽  
Level Information ◽  
High Level

AbstractDecisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should not only be sensitive to whether the choice itself was suboptimal, but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this scenario, we used a modified version of a classic reinforcement learning task in which feedback indicated if negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful but the reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction error in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.

scholarly journals Pupil responses as indicators of value-based decision-making

2018 ◽  
Author(s):  
Joanne C. Van Slooten ◽  
Sara Jahfari ◽  
Tomas Knapen ◽  
Jan Theeuwes
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Computational Model ◽  
Cognitive Processes ◽  
Learning Task ◽  
Brain Regions ◽  
Computational Approach ◽  
Prediction Errors ◽  
Reward Prediction ◽  
Exciting Possibility

AbstractPupil responses have been used to track cognitive processes during decision-making. Studies have shown that in these cases the pupil reflects the joint activation of many cortical and subcortical brain regions, also those traditionally implicated in value-based learning. However, how the pupil tracks value-based decisions and reinforcement learning is unknown. We combined a reinforcement learning task with a computational model to study pupil responses during value-based decisions, and decision evaluations. We found that the pupil closely tracks reinforcement learning both across trials and participants. Prior to choice, the pupil dilated as a function of trial-by-trial fluctuations in value beliefs. After feedback, early dilation scaled with value uncertainty, whereas later constriction scaled with reward prediction errors. Our computational approach systematically implicates the pupil in value-based decisions, and the subsequent processing of violated value beliefs, ttese dissociable influences provide an exciting possibility to non-invasively study ongoing reinforcement learning in the pupil.

scholarly journals Increased ventromedial prefrontal cortex activity in adolescence benefits prosocial reinforcement learning

2021 ◽  
Author(s):  
Bianca Westhoff ◽  
Neeltje E. Blankenstein ◽  
Elisabeth Schreuders ◽  
Eveline A. Crone ◽  
Anna C. K. van Duijvenvoorde
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Functional Neuroimaging ◽  
Learning Task ◽  
Developmental Trajectory ◽  
Ventromedial Prefrontal Cortex ◽  
Prediction Errors ◽  
Age Related ◽  
Probabilistic Reinforcement ◽  
Neuroimaging Study

AbstractLearning which of our behaviors benefit others contributes to social bonding and being liked by others. An important period for the development of (pro)social behavior is adolescence, in which peers become more salient and relationships intensify. It is, however, unknown how learning to benefit others develops across adolescence and what the underlying cognitive and neural mechanisms are. In this functional neuroimaging study, we assessed learning for self and others (i.e., prosocial learning) and the concurring neural tracking of prediction errors across adolescence (ages 9-21, N=74). Participants performed a two-choice probabilistic reinforcement learning task in which outcomes resulted in monetary consequences for themselves, an unknown other, or no one. Participants from all ages were able to learn for themselves and others, but learning for others showed a more protracted developmental trajectory. Prediction errors for self were observed in the ventral striatum and showed no age-related differences. However, prediction error coding for others was specifically observed in the ventromedial prefrontal cortex and showed age-related increases. These results reveal insights into the computational mechanisms of learning for others across adolescence, and highlight that learning for self and others show different age-related patterns.

scholarly journals Acute stress impairs reward learning in men

2020 ◽  
Author(s):  
Joana Carvalheiro ◽  
Vasco A. Conceição ◽  
Ana Mesquita ◽  
Ana Seara-Cardoso
Keyword(s):  
Reinforcement Learning ◽  
Acute Stress ◽  
Learning Task ◽  
Reward Learning ◽  
Prediction Errors ◽  
How People Learn ◽  
And Control ◽  
The Impact ◽  
Reinforcement Learning Model ◽  
Better Than

AbstractAcute stress is ubiquitous in everyday life, but the extent to which acute stress affects how people learn from the outcomes of their choices is still poorly understood. Here, we investigate how acute stress impacts reward and punishment learning in men using a reinforcement-learning task. Sixty-two male participants performed the task whilst under stress and control conditions. We observed that acute stress impaired participants’ choice performance towards monetary gains, but not losses. To unravel the mechanism(s) underlying such impairment, we fitted a reinforcement-learning model to participants’ trial-by-trial choices. Computational modeling indicated that under acute stress participants learned more slowly from positive prediction errors — when the outcomes were better than expected — consistent with stress-induced dopamine disruptions. Such mechanistic understanding of how acute stress impairs reward learning is particularly important given the pervasiveness of stress in our daily life and the impact that stress can have on our wellbeing and mental health.

Sign in / Sign up

Export Citation Format

Share Document