scholarly journals Feature-based aggregation and deep reinforcement learning: a survey and some new implementations

2019 ◽  
Vol 6 (1) ◽  
pp. 1-31 ◽  
Author(s):  
Dimitri P. Bertsekas
Keyword(s):  
Reinforcement Learning ◽  
Feature Based

scholarly journals Concept learning of ecological and artificial stimuli in rhesus macaques

2015 ◽  
Author(s):  
Drew Altschul ◽  
Greg Jensen ◽  
Herbert S Terrace
Keyword(s):  
Computer Vision ◽  
Reinforcement Learning ◽  
Concept Learning ◽  
Rhesus Macaques ◽  
Form Class ◽  
Specific Order ◽  
Feature Based ◽  
Vision Algorithm ◽  
Physical Features ◽  
Better Than

The study of concepts in animals is complicated by the possibility that performance reflects reinforcement learning of discriminative cues, which might be used to categorize of stimuli. To minimize that possibility, we trained seven rhesus macaques to respond, in a specific order, to four simultaneously presented exemplars of different perceptual concepts. These exemplars were drawn at random from large banks of images; in some conditions, the stimuli changed on every trial. Subjects nevertheless identified and ordered these stimuli correctly. Three subjects learned to correctly order ecologically relevant concepts; four subjects, to order close-up sections of paintings by four artists with distinctive styles. All subjects classified stimuli significantly better than that predicted by chance, and outperformed a feature-based computer vision algorithm, even when the exemplars were changed on every trial. Furthermore, six subjects (three using ecological stimuli and three using paintings) transferred these concepts to novel stimuli. Our results suggest that monkeys possess a flexible ability to form class-based perceptual concepts that cannot be explained as the mere discrimination of physical features.

scholarly journals Attentional Selection Can Be Predicted by Reinforcement Learning of Task-relevant Stimulus Features Weighted by Value-independent Stickiness

2016 ◽  
Vol 28 (2) ◽  
pp. 333-349 ◽  
Author(s):  
Matthew Balcarras ◽  
Salva Ardid ◽  
Daniel Kaping ◽  
Stefan Everling ◽  
Thilo Womelsdorf
Keyword(s):  
Reinforcement Learning ◽  
Choice Behavior ◽  
Selection Process ◽  
Attentional Selection ◽  
Stimulus Selection ◽  
Action Plans ◽  
Relevant Stimulus ◽  
Learning Mechanisms ◽  
Feature Based ◽  
Stimulus Features

Attention includes processes that evaluate stimuli relevance, select the most relevant stimulus against less relevant stimuli, and bias choice behavior toward the selected information. It is not clear how these processes interact. Here, we captured these processes in a reinforcement learning framework applied to a feature-based attention task that required macaques to learn and update the value of stimulus features while ignoring nonrelevant sensory features, locations, and action plans. We found that value-based reinforcement learning mechanisms could account for feature-based attentional selection and choice behavior but required a value-independent stickiness selection process to explain selection errors while at asymptotic behavior. By comparing different reinforcement learning schemes, we found that trial-by-trial selections were best predicted by a model that only represents expected values for the task-relevant feature dimension, with nonrelevant stimulus features and action plans having only a marginal influence on covert selections. These findings show that attentional control subprocesses can be described by (1) the reinforcement learning of feature values within a restricted feature space that excludes irrelevant feature dimensions, (2) a stochastic selection process on feature-specific value representations, and (3) value-independent stickiness toward previous feature selections akin to perseveration in the motor domain. We speculate that these three mechanisms are implemented by distinct but interacting brain circuits and that the proposed formal account of feature-based stimulus selection will be important to understand how attentional subprocesses are implemented in primate brain networks.

scholarly journals Kinematic Synthesis Using Reinforcement Learning

2018 ◽  
Author(s):  
Kaz Vermeer ◽  
Reinier Kuppens ◽  
Justus Herder
Keyword(s):  
Reinforcement Learning ◽  
Path Generation ◽  
Two Dimensional ◽  
Kinematic Synthesis ◽  
Prior Art ◽  
Straight Line ◽  
Feature Based ◽  
Kinematic Mechanisms ◽  
Straight Line Mechanism ◽  
Kinematic Simulations

The presented research demonstrates the synthesis of two-dimensional kinematic mechanisms using feature-based reinforcement learning. As a running example the classic challenge of designing a straight-line mechanism is adopted: a mechanism capable of tracing a straight line as part of its trajectory. This paper presents a basic framework, consisting of elements such as mechanism representations, kinematic simulations and learning algorithms, as well as some of the resulting mechanisms and a comparison to prior art. Series of successful mechanisms have been synthesized for path generation of a straight line and figure-eight.

scholarly journals Hippocampal Pattern Separation Supports Reinforcement Learning

2018 ◽  
Author(s):  
Ian Ballard ◽  
Anthony D. Wagner ◽  
Samuel M. McClure
Keyword(s):  
Reinforcement Learning ◽  
Functional Mri ◽  
The Other ◽  
Pattern Separation ◽  
Poison Ivy ◽  
Feature Based ◽  
Individual Features ◽  
Object Features

1ABSTRACTAnimals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three-leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multifeatural conjunctive representations. We tested if the hippocampus formed separable conjunctive representations that enabled learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data showed the hippocampus formed conjunctive representations that were dissociable from feature components and that these representations influenced striatal PEs. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.

Bayes factors for reinforcement-learning models of the Iowa gambling task.

Decision ◽  
2016 ◽  
Vol 3 (2) ◽  
pp. 115-131 ◽  
Author(s):  
Helen Steingroever ◽  
Ruud Wetzels ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Reinforcement Learning ◽  
Iowa Gambling Task ◽  
Bayes Factors ◽  
Gambling Task ◽  
Learning Models ◽  
Reinforcement Learning Models
Sign in / Sign up

Export Citation Format

Share Document