scholarly journals A computational psychiatry approach identifies how alpha-2A noradrenergic agonist Guanfacine affects feature-based reinforcement learning in the macaque

2017 ◽  
Vol 7 (1) ◽  
Author(s):  
S. A. Hassani ◽  
M. Oemisch ◽  
M. Balcarras ◽  
S. Westendorff ◽  
S. Ardid ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Computational Psychiatry ◽  
Feature Based

scholarly journals Concept learning of ecological and artificial stimuli in rhesus macaques

2015 ◽  
Author(s):  
Drew Altschul ◽  
Greg Jensen ◽  
Herbert S Terrace
Keyword(s):  
Computer Vision ◽  
Reinforcement Learning ◽  
Concept Learning ◽  
Rhesus Macaques ◽  
Form Class ◽  
Specific Order ◽  
Feature Based ◽  
Vision Algorithm ◽  
Physical Features ◽  
Better Than

The study of concepts in animals is complicated by the possibility that performance reflects reinforcement learning of discriminative cues, which might be used to categorize of stimuli. To minimize that possibility, we trained seven rhesus macaques to respond, in a specific order, to four simultaneously presented exemplars of different perceptual concepts. These exemplars were drawn at random from large banks of images; in some conditions, the stimuli changed on every trial. Subjects nevertheless identified and ordered these stimuli correctly. Three subjects learned to correctly order ecologically relevant concepts; four subjects, to order close-up sections of paintings by four artists with distinctive styles. All subjects classified stimuli significantly better than that predicted by chance, and outperformed a feature-based computer vision algorithm, even when the exemplars were changed on every trial. Furthermore, six subjects (three using ecological stimuli and three using paintings) transferred these concepts to novel stimuli. Our results suggest that monkeys possess a flexible ability to form class-based perceptual concepts that cannot be explained as the mere discrimination of physical features.

scholarly journals Attentional Selection Can Be Predicted by Reinforcement Learning of Task-relevant Stimulus Features Weighted by Value-independent Stickiness

2016 ◽  
Vol 28 (2) ◽  
pp. 333-349 ◽  
Author(s):  
Matthew Balcarras ◽  
Salva Ardid ◽  
Daniel Kaping ◽  
Stefan Everling ◽  
Thilo Womelsdorf
Keyword(s):  
Reinforcement Learning ◽  
Choice Behavior ◽  
Selection Process ◽  
Attentional Selection ◽  
Stimulus Selection ◽  
Action Plans ◽  
Relevant Stimulus ◽  
Learning Mechanisms ◽  
Feature Based ◽  
Stimulus Features

Attention includes processes that evaluate stimuli relevance, select the most relevant stimulus against less relevant stimuli, and bias choice behavior toward the selected information. It is not clear how these processes interact. Here, we captured these processes in a reinforcement learning framework applied to a feature-based attention task that required macaques to learn and update the value of stimulus features while ignoring nonrelevant sensory features, locations, and action plans. We found that value-based reinforcement learning mechanisms could account for feature-based attentional selection and choice behavior but required a value-independent stickiness selection process to explain selection errors while at asymptotic behavior. By comparing different reinforcement learning schemes, we found that trial-by-trial selections were best predicted by a model that only represents expected values for the task-relevant feature dimension, with nonrelevant stimulus features and action plans having only a marginal influence on covert selections. These findings show that attentional control subprocesses can be described by (1) the reinforcement learning of feature values within a restricted feature space that excludes irrelevant feature dimensions, (2) a stochastic selection process on feature-specific value representations, and (3) value-independent stickiness toward previous feature selections akin to perseveration in the motor domain. We speculate that these three mechanisms are implemented by distinct but interacting brain circuits and that the proposed formal account of feature-based stimulus selection will be important to understand how attentional subprocesses are implemented in primate brain networks.

scholarly journals Kinematic Synthesis Using Reinforcement Learning

2018 ◽  
Author(s):  
Kaz Vermeer ◽  
Reinier Kuppens ◽  
Justus Herder
Keyword(s):  
Reinforcement Learning ◽  
Path Generation ◽  
Two Dimensional ◽  
Kinematic Synthesis ◽  
Prior Art ◽  
Straight Line ◽  
Feature Based ◽  
Kinematic Mechanisms ◽  
Straight Line Mechanism ◽  
Kinematic Simulations

The presented research demonstrates the synthesis of two-dimensional kinematic mechanisms using feature-based reinforcement learning. As a running example the classic challenge of designing a straight-line mechanism is adopted: a mechanism capable of tracing a straight line as part of its trajectory. This paper presents a basic framework, consisting of elements such as mechanism representations, kinematic simulations and learning algorithms, as well as some of the resulting mechanisms and a comparison to prior art. Series of successful mechanisms have been synthesized for path generation of a straight line and figure-eight.

A Valuation Framework for Emotions Applied to Depression and Recurrence

2016 ◽  
Author(s):  
Quentin J. M. Huys
Keyword(s):  
Emotion Regulation ◽  
Reinforcement Learning ◽  
Integrative Approach ◽  
Computational Framework ◽  
Normative Framework ◽  
Model Based ◽  
Cognitive Components ◽  
Model Free ◽  
Stress Sensitization ◽  
Computational Psychiatry

The burden of depression is substantially aggravated by relapses and recurrences, and these become more inevitable with every episode of depression. This chapter describes how computational psychiatry can provide a normative framework for emotions and an integrative approach to core cognitive components of depression and relapse. Central to this is the notion that emotions effectively imply a valuation; thus they are amenable to description and dissection by reinforcement-learning methods. It is argued that cognitive accounts of emotion can be viewed in terms of model-based valuation, and that automatic emotional responses relate to model-free valuation and the innate recruitment of fixed behavioral patterns. This model-based view captures phenomena such as helplessness, hopelessness, attributions, and stress sensitization. Considering it in more atomic algorithmic detail opens up the possibility of viewing rumination and emotion regulation in this same normative framework. The problem of treatment selection for relapse and recurrence prevention is outlined and suggestions made on how the computational framework of emotions might help improve this. The chapter closes with a brief overview of what we can hope to gain from computational psychiatry.

scholarly journals Hippocampal Pattern Separation Supports Reinforcement Learning

2018 ◽  
Author(s):  
Ian Ballard ◽  
Anthony D. Wagner ◽  
Samuel M. McClure
Keyword(s):  
Reinforcement Learning ◽  
Functional Mri ◽  
The Other ◽  
Pattern Separation ◽  
Poison Ivy ◽  
Feature Based ◽  
Individual Features ◽  
Object Features

1ABSTRACTAnimals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three-leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multifeatural conjunctive representations. We tested if the hippocampus formed separable conjunctive representations that enabled learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data showed the hippocampus formed conjunctive representations that were dissociable from feature components and that these representations influenced striatal PEs. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.

Computational Psychiatry

2017 ◽  
Vol 65 (1) ◽  
pp. 21-26 ◽  
Author(s):  
Quentin J. M. Huys
Keyword(s):  
Computational Psychiatry ◽  
Klinische Anwendungen

Zusammenfassung. „Computational Psychiatry“ ist eine neue Forschungsrichtung, die Fortschritte aus den theoretischen und experimentellen Neurowissenschaften in klinische Anwendungen für die Psychiatrie umzusetzen will. Der mögliche Nutzen mathematischer Modelle für psychiatrische Anwendungen ergibt sich vor allem aus der Komplexität psychiatrischer Phänomene, deren Beherrschung neue analytische Herangehensweisen erfordert. Konkret können mithilfe solcher Modelle erstens innerpsychische und ansonsten nicht direkt messbare Prozesse erfasst werden. Ein Beispiel hierfür sind Lernprozesse. Zweitens können Phänomene auf verschiedenen Ebenen quantitativ miteinander in Verbindung gebracht werden, z.B. der Effekt von Ionenkanalstörungen auf das Kurzzeitgedächtnis. Drittens können Methoden aus dem maschinellen Lernen mit diesen Modellen verbunden werden, um grosse Datensätze zu analysieren. Obwohl erste Ansätze aus dieser Forschung schon möglichen klinischen Nutzen erwiesen haben, ist das Feld noch jung. Der Artikel schliesst mit dem Vorschlag, Prozeduren aus der Entwicklung pharmazeutischer Produkte für die Validierung theoretischer Anwendungen herbeizuziehen.

Sign in / Sign up

Export Citation Format

Share Document