AbstractAttributing outcomes to your own actions or to external causes is essential for appropriately learning which actions lead to reward and which actions do not. Our previous work showed that this type of credit assignment is best explained by a Bayesian reinforcement learning model which posits that beliefs about the causal structure of the environment modulate reward prediction errors (RPEs) during action value updating. In this study, we investigated the neural circuits underlying reinforcement learning that are influenced by causal beliefs using functional magnetic resonance imaging (fMRI) while human participants (N = 31; 13 males, 18 females) completed a behavioral task that manipulated beliefs about causal structure. We found evidence that RPEs modulated by causal beliefs are represented in posterior putamen, while standard (unmodulated) RPEs are represented in ventral striatum. Further analyses revealed that beliefs about causal structure are represented in anterior insula and inferior frontal gyrus. Finally, structural equation modeling revealed effective connectivity from anterior insula to posterior putamen. Together, these results are consistent with a neural architecture in which causal beliefs in anterior insula are integrated with prediction error signals in posterior putamen to update action values.Significance StatementLearning which actions lead to reward – a process known as reinforcement learning – is essential for survival. Inferring the causes of observed outcomes – a process known as causal inference – is crucial for appropriately assigning credit to one’s own actions and restricting learning to effective action-outcome contingencies. Previous studies have linked reinforcement learning to the striatum and causal inference to prefrontal regions, yet how these neural processes interact to guide adaptive behavior remains poorly understood. Here, we found evidence that causal beliefs represented in the prefrontal cortex modulate action value updating in posterior striatum, separately from the unmodulated action value update in ventral striatum posited by standard reinforcement learning models.
The influence of early audiovisual experience on multisensory integration and causal inference (Commentary on Smyre et al., 2021)