scholarly journals Reward prediction error modulates saccade vigor

2019 ◽  
Author(s):  
Ehsan Sedaghat-Nejad ◽  
David J. Herzfeld ◽  
Reza Shadmehr
Keyword(s):  
Prediction Error ◽  
Dopamine Release ◽  
Human Subjects ◽  
Reaction Times ◽  
Movement Onset ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Movement Error ◽  
Orderly Fashion ◽  
Learning From Error

AbstractMovements toward rewarding stimuli exhibit greater vigor, i.e., increased velocity and reduced reaction-times. This invigoration may be due to release of dopamine before movement onset. Dopamine release is strongly modulated by reward prediction error (RPE). Here, we generated an RPE event in the milliseconds before movement onset and tested whether there was a causal relationship between RPE and vigor. Human subjects made saccades toward an image. During the execution of their primary saccade, we probabilistically changed the position and content of the image. This led to a secondary saccade following completion of the primary saccade. We focused on properties of this secondary saccade. On some trials, the content of the secondary image was more valuable than the first image, resulting in a +RPE event that preceded the secondary saccade. On other trials, this content was less valuable, resulting in a -RPE event. We found that reaction-time and velocity of the secondary saccade were affected in an orderly fashion by the magnitude and direction of the preceding RPE event: the most vigorous saccades followed the largest +RPE, whereas the least vigorous saccades followed the largest -RPE. Presence of the secondary saccade indicated that the primary saccade had experienced a movement error, inducing trial-to-trial adaptation: the subsequent primary saccade was changed in the direction of the movement error in the previous trial. However, motor learning from error was not affected by the RPE event. Therefore, reward prediction error, and not reward per se, modulated vigor of saccades.Author summaryDoes dopamine release before onset of a movement modulate vigor of the ensuing movement? To test this hypothesis, we relied on the fact that RPE is a strong modulator of dopamine. Our innovation was a task in which an RPE event occurred precisely before onset of a movement. We probabilistically produced a combination of large or small, negative or positive RPE events before onset of a saccade, and observed that the vigor of the saccade that followed carried a robust signature of the preceding RPE event: high vigor saccades followed +RPE events, while low vigor saccades followed -RPE events. This suggests that control of vigor is partly through release of dopamine in the moments before onset of the movement.

Neural Coding of Reward-Prediction Error Signals During Classical Conditioning With Attractive Faces

2007 ◽  
Vol 97 (4) ◽  
pp. 3036-3045 ◽  
Author(s):  
Signe Bray ◽  
John O'Doherty
Keyword(s):  
Neural Coding ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Human Subjects ◽  
Visual Stimuli ◽  
Dopamine Neurons ◽  
Error Signal ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Attractive Female

Attractive faces can be considered to be a form of visual reward. Previous imaging studies have reported activity in reward structures including orbitofrontal cortex and nucleus accumbens during presentation of attractive faces. Given that these stimuli appear to act as rewards, we set out to explore whether it was possible to establish conditioning in human subjects by pairing presentation of arbitrary affectively neutral stimuli with subsequent presentation of attractive and unattractive faces. Furthermore, we scanned human subjects with functional magnetic resonance imaging (fMRI) while they underwent this conditioning procedure to determine whether a reward-prediction error signal is engaged during learning with attractive faces as is known to be the case for learning with other types of reward such as juice and money. Subjects showed changes in behavioral ratings to the conditioned stimuli (CS) when comparing post- to preconditioning evaluations, notably for those CSs paired with attractive female faces. We used a simple Rescorla-Wagner learning model to generate a reward-prediction error signal and entered this into a regression analysis with the fMRI data. We found significant prediction error-related activity in the ventral striatum during conditioning with attractive compared with unattractive faces. These findings suggest that an arbitrary stimulus can acquire conditioned value by being paired with pleasant visual stimuli just as with other types of reward such as money or juice. This learning process elicits a reward-prediction error signal in a main target structure of dopamine neurons: the ventral striatum. The findings we describe here may provide insights into the neural mechanisms tapped into by advertisers seeking to influence behavioral preferences by repeatedly exposing consumers to simple associations between products and rewarding visual stimuli such as pretty faces.

scholarly journals A transient dopamine signal encodes subjective value and causally influences demand in an economic context

2017 ◽  
Vol 114 (52) ◽  
pp. E11303-E11312 ◽  
Author(s):  
Scott A. Schelp ◽  
Katherine J. Pultorak ◽  
Dylan R. Rakowski ◽  
Devan M. Gomez ◽  
Gregory Krzystyniak ◽  
...  
Keyword(s):  
Prediction Error ◽  
Dopamine Release ◽  
Dopamine Neurons ◽  
Price Sensitivity ◽  
Mesolimbic Dopamine ◽  
Fast Scan Cyclic Voltammetry ◽  
Reward Prediction Error ◽  
Dopamine Concentration ◽  
Reward Prediction ◽  
Subjective Value

The mesolimbic dopamine system is strongly implicated in motivational processes. Currently accepted theories suggest that transient mesolimbic dopamine release events energize reward seeking and encode reward value. During the pursuit of reward, critical associations are formed between the reward and cues that predict its availability. Conditioned by these experiences, dopamine neurons begin to fire upon the earliest presentation of a cue, and again at the receipt of reward. The resulting dopamine concentration scales proportionally to the value of the reward. In this study, we used a behavioral economics approach to quantify how transient dopamine release events scale with price and causally alter price sensitivity. We presented sucrose to rats across a range of prices and modeled the resulting demand curves to estimate price sensitivity. Using fast-scan cyclic voltammetry, we determined that the concentration of accumbal dopamine time-locked to cue presentation decreased with price. These data confirm and extend the notion that dopamine release events originating in the ventral tegmental area encode subjective value. Using optogenetics to augment dopamine concentration, we found that enhancing dopamine release at cue made demand more sensitive to price and decreased dopamine concentration at reward delivery. From these observations, we infer that value is decreased because of a negative reward prediction error (i.e., the animal receives less than expected). Conversely, enhancing dopamine at reward made demand less sensitive to price. We attribute this finding to a positive reward prediction error, whereby the animal perceives they received a better value than anticipated.

scholarly journals A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning

2020 ◽  
Author(s):  
Pramod Kaushik ◽  
Jérémie Naudé ◽  
Surampudi Bapi Raju ◽  
Frédéric Alexandre
Keyword(s):  
Classical Conditioning ◽  
Computational Model ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Dopamine Neurons ◽  
System Level ◽  
Reward Prediction Error ◽  
Twin Peaks ◽  
Reward Prediction ◽  
Error Computation

AbstractClassical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

T92. Time Scales of Encoding the Reward Prediction Error in Youth: Representation of Past Events

2018 ◽  
Vol 83 (9) ◽  
pp. S164
Author(s):  
Hanna Keren ◽  
Nathan Fox ◽  
Ellen Leibenluft ◽  
Daniel S. Pine ◽  
Argyris Stringaris
Keyword(s):  
Time Scales ◽  
Prediction Error ◽  
Reward Prediction Error ◽  
Reward Prediction

scholarly journals O5.1. STRIATAL DOPAMINE AND REDUCED REWARD PREDICTION ERROR SIGNALING IN UNMEDICATED SCHIZOPHRENIA PATIENTS

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S11-S11
Author(s):  
Teresa Katthagen ◽  
Jakob Kaminski ◽  
Andreas Heinz ◽  
Ralph Buchert ◽  
Florian Schlagenhauf
Keyword(s):  
Prediction Error ◽  
Negative Symptoms ◽  
Ventral Striatum ◽  
Striatal Dopamine ◽  
Dopamine Synthesis ◽  
Positive Symptoms ◽  
Healthy Controls ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Significant Difference

Abstract Background Increased striatal dopamine synthesis capacity (DSC) has consistently been reported in patients with schizophrenia (Sz). However, the functional mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error (RPE) signaling during reinforcement learning. Methods In this study, we investigated striatal DSC and RPEs and their association in unmedicated Sz and healthy controls. 23 healthy controls (HC) and 20 unmedicated Sz took part in an FDOPA-PET scan measuring DSC and underwent fMRI scanning, where they performed a reversal learning paradigm. We compared groups regarding DSC und neural RPE signals and probed the respective correlation (23 HC and 16 Sz for both measures). Results There was no significant difference between HC and Sz in DSC. Taking into account comorbid alcohol abuse revealed that only patients without such abuse showed elevated DSC in the associative and sensorimotor striatum, while those with abuse did not differ from HC. Patients performed worse during learning, accompanied by a reduced RPE signal in the ventral striatum. In HC, the DSC in the limbic striatum correlated with higher RPE signaling, while there was no significant association in patients. DSC in the associative striatum correlated with higher positive symptoms, and blunted RPE signaling was associated with negative symptoms. Discussion Our results suggest that dopamine modulation of RPE is impaired in schizophrenia. Furthermore, we observed a dissociation with elevated DSC in the associative and sensorimotor striatum contributing to positive symptoms and blunted RPE in the ventral striatum to negative symptoms.

Sign in / Sign up

Export Citation Format

Share Document