scholarly journals Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

2019 ◽  
Author(s):  
Emilie Werlen ◽  
Soon-Lim Shin ◽  
Francois Gastambide ◽  
Jennifer Francois ◽  
Mark D Tricklebank ◽  
...  
Keyword(s):  
Nucleus Accumbens ◽  
Orbitofrontal Cortex ◽  
Prediction Error ◽  
Learning Task ◽  
Adaptive Behaviour ◽  
Diagnostic Feature ◽  
Prediction Errors ◽  
Guided Learning ◽  
The Difference ◽  
Precise Relationship

AbstractIn an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events, and in the reward domain this has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC) while freely-moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats’ ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies using a proxy signal for BOLD in a freely behaving rodent.

scholarly journals Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

2019 ◽  
Vol 45 (5) ◽  
pp. 793-803 ◽  
Author(s):  
Emilie Werlen ◽  
Soon-Lim Shin ◽  
Francois Gastambide ◽  
Jennifer Francois ◽  
Mark D. Tricklebank ◽  
...  
Keyword(s):  
Nucleus Accumbens ◽  
Orbitofrontal Cortex ◽  
Prediction Error ◽  
Learning Task ◽  
Adaptive Behaviour ◽  
Diagnostic Feature ◽  
Prediction Errors ◽  
Guided Learning ◽  
The Difference ◽  
Precise Relationship

Abstract In an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events and in the reward domain that has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC), while freely moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats’ ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies by using a proxy signal for BOLD in a freely behaving rodent.

scholarly journals Momentary subjective well-being depends on learning and not reward

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Bastien Blain ◽  
Robb B Rutledge
Keyword(s):  
Prediction Error ◽  
Computational Modelling ◽  
Well Being ◽  
Learning Task ◽  
Adaptive Behaviour ◽  
Subjective Well Being ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
The Difference ◽  
Momentary Happiness

Subjective well-being or happiness is often associated with wealth. Recent studies suggest that momentary happiness is associated with reward prediction error, the difference between experienced and predicted reward, a key component of adaptive behaviour. We tested subjects in a reinforcement learning task in which reward size and probability were uncorrelated, allowing us to dissociate between the contributions of reward and learning to happiness. Using computational modelling, we found convergent evidence across stable and volatile learning tasks that happiness, like behaviour, is sensitive to learning-relevant variables (i.e. probability prediction error). Unlike behaviour, happiness is not sensitive to learning-irrelevant variables (i.e. reward prediction error). Increasing volatility reduces how many past trials influence behaviour but not happiness. Finally, depressive symptoms reduce happiness more in volatile than stable environments. Our results suggest that how we learn about our world may be more important for how we feel than the rewards we actually receive.

scholarly journals Reward prediction errors induce risk-seeking

2020 ◽  
Author(s):  
Moritz Moeller ◽  
Jan Grohn ◽  
Sanjay Manohar ◽  
Rafal Bogacz
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Risk Preferences ◽  
Behavioral Effect ◽  
Learning Task ◽  
Expected Value ◽  
Learning Theories ◽  
Prediction Errors ◽  
Risk Seeking ◽  
The Difference

AbstractReinforcement learning theories propose that humans choose based on the estimated values of available options, and that they learn from rewards by reducing the difference between the experienced and expected value. In the brain, such prediction errors are broadcasted by dopamine. However, choices are not only influenced by expected value, but also by risk. Like reinforcement learning, risk preferences are modulated by dopamine: enhanced dopamine levels induce risk-seeking. Learning and risk preferences have so far been studied independently, even though it is commonly assumed that they are (partly) regulated by the same neurotransmitter. Here, we use a novel learning task to look for prediction-error induced risk-seeking in human behavior and pupil responses. We find that prediction errors are positively correlated with risk-preferences in imminent choices. Physiologically, this effect is indexed by pupil dilation: only participants whose pupil response indicates that they experienced the prediction error also show the behavioral effect.

scholarly journals Behavioural and computational evidence for memory consolidation biased by reward-prediction errors

2019 ◽  
Author(s):  
Emma L. Roscow ◽  
Matthew W. Jones ◽  
Nathan F. Lepora
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Learning Task ◽  
Male Rats ◽  
Adaptive Behaviour ◽  
Prediction Errors ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Per Se ◽  
Reinforcement Learning Model

AbstractNeural activity encoding recent experiences is replayed during sleep and rest to promote consolidation of the corresponding memories. However, precisely which features of experience influence replay prioritisation to optimise adaptive behaviour remains unclear. Here, we trained adult male rats on a novel maze-based rein-forcement learning task designed to dissociate reward outcomes from reward-prediction errors. Four variations of a reinforcement learning model were fitted to the rats’ behaviour over multiple days. Behaviour was best predicted by a model incorporating replay biased by reward-prediction error, compared to the same model with no replay; random replay or reward-biased replay produced poorer predictions of behaviour. This insight disentangles the influences of salience on replay, suggesting that reinforcement learning is tuned by post-learning replay biased by reward-prediction error, not by reward per se. This work therefore provides a behavioural and theoretical toolkit with which to measure and interpret replay in striatal, hippocampal and neocortical circuits.

scholarly journals Frontal Theta Oscillatory Activity Is a Common Mechanism for the Computation of Unexpected Outcomes and Learning Rate

2014 ◽  
Vol 26 (3) ◽  
pp. 447-458 ◽  
Author(s):  
Ernest Mas-Herrero ◽  
Josep Marco-Pallarés
Keyword(s):  
Prediction Error ◽  
Learning Task ◽  
Learning Rate ◽  
Oscillatory Activity ◽  
Common Mechanism ◽  
Prediction Errors ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Medial Pfc ◽  
The Impact

In decision-making processes, the relevance of the information yielded by outcomes varies across time and situations. It increases when previous predictions are not accurate and in contexts with high environmental uncertainty. Previous fMRI studies have shown an important role of medial pFC in coding both reward prediction errors and the impact of this information to guide future decisions. However, it is unclear whether these two processes are dissociated in time or occur simultaneously, suggesting that a common mechanism is engaged. In the present work, we studied the modulation of two electrophysiological responses associated to outcome processing—the feedback-related negativity ERP and frontocentral theta oscillatory activity—with the reward prediction error and the learning rate. Twenty-six participants performed two learning tasks differing in the degree of predictability of the outcomes: a reversal learning task and a probabilistic learning task with multiple blocks of novel cue–outcome associations. We implemented a reinforcement learning model to obtain the single-trial reward prediction error and the learning rate for each participant and task. Our results indicated that midfrontal theta activity and feedback-related negativity increased linearly with the unsigned prediction error. In addition, variations of frontal theta oscillatory activity predicted the learning rate across tasks and participants. These results support the existence of a common brain mechanism for the computation of unsigned prediction error and learning rate.

scholarly journals Substrates of Human Prediction Error for Incentives, Perception, Cognition, and Action

2021 ◽  
Author(s):  
Philip R. Corlett ◽  
Jessica A Mollick ◽  
Hedy Kober
Keyword(s):  
Artificial Intelligence ◽  
Computational Neuroscience ◽  
Orbitofrontal Cortex ◽  
Prediction Error ◽  
Functional Neuroimaging ◽  
Meta Analysis ◽  
Basic Research ◽  
Preclinical Studies ◽  
Prediction Errors ◽  
Neural Firing

Prediction errors (PEs) are a keystone for computational neuroscience. Their association with midbrain neural firing has been confirmed across species and has inspired the construction of artificial intelligence that can outperform humans. However, there is still much to learn. Here, we leverage the wealth of human PE data acquired in the functional neuroimaging setting in service of a deeper understanding, using meta-analysis. Across 263 PE studies that have focused on reward, punishment, action, cognition, and perception, we found consistent region-PE associations that were posited theoretically or evinced in preclinical studies, but not yet established in humans, including midbrain PE signals during perceptual and Pavlovian tasks. Further, we found evidence for PEs over successor representations in orbitofrontal cortex, and for default mode network PE signals. By combining functional imaging meta-analysis with theory and basic research, we provide new insights into learning in machines, humans, and other animals.

scholarly journals L-DOPA modulates activity in the vmPFC, nucleus accumbens, and VTA during threat extinction learning in humans

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Roland Esser ◽  
Christoph W Korn ◽  
Florian Ganzer ◽  
Jan Haaker
Keyword(s):  
Prefrontal Cortex ◽  
Nucleus Accumbens ◽  
Prediction Error ◽  
Computational Modelling ◽  
Ventromedial Prefrontal Cortex ◽  
Adaptive Behaviour ◽  
Extinction Learning ◽  
Behavioural Treatment ◽  
Related Pathway ◽  
Time Point

Learning to be safe is central for adaptive behaviour when threats are no longer present. Detecting the absence of an expected threat is key for threat extinction learning and an essential process for the behavioural treatment of anxiety-related disorders. One possible mechanism underlying extinction learning is a dopaminergic mismatch signal that encodes the absence of an expected threat. Here we show that such a dopamine-related pathway underlies extinction learning in humans. Dopaminergic enhancement via administration of L-DOPA (vs. Placebo) was associated with reduced retention of differential psychophysiological threat responses at later test, which was mediated by activity in the ventromedial prefrontal cortex that was specific to extinction learning. L-DOPA administration enhanced signals at the time-point of an expected, but omitted threat in extinction learning within the nucleus accumbens, which were functionally coupled with the ventral tegmental area and the amygdala. Computational modelling of threat expectancies further revealed prediction error encoding in nucleus accumbens that was reduced when L-DOPA was administered. Our results thereby provide evidence that extinction learning is influenced by L-DOPA and provide a mechanistic perspective to augment extinction learning by dopaminergic enhancement in humans.

scholarly journals Mood dynamics are associated with learning and not choice

2019 ◽  
Author(s):  
Bastien Blain ◽  
Robb Rutledge
Keyword(s):  
Affective State ◽  
Computational Modelling ◽  
Risky Choice ◽  
Adaptive Behaviour ◽  
Prediction Errors ◽  
Affective States ◽  
Environmental Statistics ◽  
Probability Prediction ◽  
The Difference ◽  
Choice Tasks

Updating predictions about which stimuli are associated with reward is an important aspect of adaptive behaviour believed to relate to prediction errors, the difference between experienced and predicted outcomes. Behavioural sensitivity to prediction errors flexibly adapts to environmental statistics. Prediction errors also influence affective states during risky choice tasks that do not require learning, but the relationship between emotions and adaptive behaviour is unknown. Here, using computational modelling we found that mood dynamics, like behaviour, are sensitive to learning-relevant model variables (i.e., probability prediction error). Unlike behaviour, mood dynamics are not sensitive to model variables that influence choice (i.e., expected value), and increasing volatility does not reduce how many trials influence affective state. Finally, depressive symptoms reduce overall mood more in volatile than stable environments. Our findings suggest that mood dynamics are selective for variables relevant to adaptive behaviour and suggest a greater role for mood in learning than choice.

scholarly journals Dopamine, Prediction Error and Beyond

2020 ◽  
pp. 107385842090759
Author(s):  
Kelly M. J. Diederen ◽  
Paul C. Fletcher
Keyword(s):  
Prediction Error ◽  
Large Body ◽  
Theoretical Models ◽  
Optimal Choice ◽  
Reward Processing ◽  
Prediction Errors ◽  
The Difference ◽  
Past Experiences ◽  
Mental Pathology

A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference.

scholarly journals Musical reward prediction errors engage the nucleus accumbens and motivate learning

2019 ◽  
Vol 116 (8) ◽  
pp. 3310-3315 ◽  
Author(s):  
Benjamin P. Gold ◽  
Ernest Mas-Herrero ◽  
Yashar Zeighami ◽  
Mitchel Benovoy ◽  
Alain Dagher ◽  
...  
Keyword(s):  
Nucleus Accumbens ◽  
Prediction Errors ◽  
Subjective Response ◽  
Related Activity ◽  
Reward Prediction ◽  
Learning Rates ◽  
Explained Variance ◽  
The Difference ◽  
The Individual ◽  
Motivate Learning

Enjoying music reliably ranks among life’s greatest pleasures. Like many hedonic experiences, it engages several reward-related brain areas, with activity in the nucleus accumbens (NAc) most consistently reflecting the listener’s subjective response. Converging evidence suggests that this activity arises from musical “reward prediction errors” (RPEs) that signal the difference between expected and perceived musical events, but this hypothesis has not been directly tested. In the present fMRI experiment, we assessed whether music could elicit formally modeled RPEs in the NAc by applying a well-established decision-making protocol designed and validated for studying RPEs. In the scanner, participants chose between arbitrary cues that probabilistically led to dissonant or consonant music, and learned to make choices associated with the consonance, which they preferred. We modeled regressors of trial-by-trial RPEs, finding that NAc activity tracked musically elicited RPEs, to an extent that explained variance in the individual learning rates. These results demonstrate that music can act as a reward, driving learning and eliciting RPEs in the NAc, a hub of reward- and music enjoyment-related activity.

Sign in / Sign up

Export Citation Format

Share Document