scholarly journals Imprecise neural computations as source of human adaptive behavior in volatile environments

2019 ◽  
Author(s):  
Charles Findling ◽  
Nicolas Chopin ◽  
Etienne Koechlin
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Everyday Life ◽  
Adaptive Behavior ◽  
Real Life ◽  
Higher Order ◽  
Low Level ◽  
Adaptive Models ◽  
Neural Computations ◽  
Counterintuitive Result

AbstractEveryday life features uncertain and ever-changing situations. In such environments, optimal adaptive behavior requires higher-order inferential capabilities to grasp the volatility of external contingencies. These capabilities however involve complex and rapidly intractable computations, so that we poorly understand how humans develop efficient adaptive behaviors in such environments. Here we demonstrate this counterintuitive result: simple, low-level inferential processes involving imprecise computations conforming to the psychophysical Weber Law actually lead to near-optimal adaptive behavior, regardless of the environment volatility. Using volatile experimental settings, we further show that such imprecise, low-level inferential processes accounted for observed human adaptive performances, unlike optimal adaptive models involving higher-order inferential capabilities, their biologically more plausible, algorithmic approximations and non-inferential adaptive models like reinforcement learning. Thus, minimal inferential capabilities may have evolved along with imprecise neural computations as contributing to near-optimal adaptive behavior in real-life environments, while leading humans to make suboptimal choices in canonical decision-making tasks.

The Role of Executive Function in Shaping Reinforcement Learning

2020 ◽  
Author(s):  
Milena Rmus ◽  
Samuel McDougle ◽  
Anne Collins
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Instrumental Behavior ◽  
Complex Environments ◽  
Neural Computations ◽  
Human Decision ◽  
Brain And Behavior ◽  
And Behavior ◽  
The Brain

Reinforcement learning (RL) models have advanced our understanding of how animals learn and make decisions, and how the brain supports some aspects of learning. However, the neural computations that are explained by RL algorithms fall short of explaining many sophisticated aspects of human decision making, including the generalization of learned information, one-shot learning, and the synthesis of task information in complex environments. Instead, these aspects of instrumental behavior are assumed to be supported by the brain’s executive functions (EF). We review recent findings that highlight the importance of EF in learning. Specifically, we advance the theory that EF sets the stage for canonical RL computations in the brain, providing inputs that broaden their flexibility and applicability. Our theory has important implications for how to interpret RL computations in the brain and behavior.

scholarly journals How the level of reward awareness changes the computational and electrophysiological signatures of reinforcement learning

2018 ◽  
Author(s):  
C.M.C. Correa ◽  
S. Noorman ◽  
J. Jiang ◽  
S. Palminteri ◽  
M.X Cohen ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Present Report ◽  
Subjective Evaluation ◽  
Computational Modelling ◽  
Learning Task ◽  
Reward Processing ◽  
Neural Activities ◽  
Electrophysiological Recordings ◽  
Neural Computations

AbstractThe extent to which subjective awareness influences reward processing, and thereby affects future decisions is currently largely unknown. In the present report, we investigated this question in a reinforcement-learning framework, combining perceptual masking, computational modeling and electroencephalographic recordings (human male and female participants). Our results indicate that degrading the visibility of the reward decreased -without completely obliterating- the ability of participants to learn from outcomes, but concurrently increased their tendency to repeat previous choices. We dissociated electrophysiological signatures evoked by the reward-based learning processes from those elicited by the reward-independent repetition of previous choices and showed that these neural activities were significantly modulated by reward visibility. Overall, this report sheds new light on the neural computations underlying reward-based learning and decision-making and highlights that awareness is beneficial for the trial-by-trial adjustment of decision-making strategies.Significance statementThe notion of reward is strongly associated with subjective evaluation, related to conscious processes such as “pleasure”, “liking” and “wanting”. Here we show that degrading reward visibility in a reinforcement learning task decreases -without completely obliterating- the ability of participants to learn from outcomes, but concurrently increases subjects tendency to repeat previous choices. Electrophysiological recordings, in combination with computational modelling, show that neural activities were significantly modulated by reward visibility. Overall, we dissociate different neural computations underlying reward-based learning and decision-making, which highlights a beneficial role of reward awareness in adjusting decision-making strategies.

Understanding reasoning: Let's describe what we really think about

2011 ◽  
Vol 34 (5) ◽  
pp. 269-270 ◽  
Author(s):  
Robert J. Sternberg
Keyword(s):  
Decision Making ◽  
Everyday Life ◽  
Deductive Reasoning ◽  
Inductive Reasoning ◽  
Real Life ◽  
Human Reasoning ◽  
Ecological Relevance ◽  
Target Article ◽  
Life Decision

AbstractI suggest psychologists would more profitably study a totally different area of human reasoning than is discussed in the target article – the inductive reasoning people use in their everyday life that matters in consequential real-life decision making, rather than the deductive reasoning that psychologists have studied meticulously but that has relatively less ecological relevance to people's lives.

scholarly journals A Recurrent Neural Network Model for Flexible and Adaptive Decision Making based on Sequence Learning

2019 ◽  
Author(s):  
Zhewei Zhang ◽  
Huzi Cheng ◽  
Tianming Yang
Keyword(s):  
Neural Network ◽  
Decision Making ◽  
Reinforcement Learning ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Adaptive Behavior ◽  
Neural Network Model ◽  
Sequence Learning ◽  
Sensory Inputs ◽  
The Brain

AbstractThe brain makes flexible and adaptive responses in the complicated and ever-changing environment for the organism’s survival. To achieve this, the brain needs to choose appropriate actions flexibly in response to sensory inputs. Moreover, the brain also has to understand how its actions affect future sensory inputs and what reward outcomes should be expected, and adapts its behavior based on the actual outcomes. A modeling approach that takes into account of the combined contingencies between sensory inputs, actions, and reward outcomes may be the key to understanding the underlying neural computation. Here, we train a recurrent neural network model based on sequence learning to predict future events based on the past event sequences that combine sensory, action, and reward events. We use four exemplary tasks that have been used in previous animal and human experiments to study different aspects of decision making and learning. We first show that the model reproduces the animals’ choice and reaction time pattern in a probabilistic reasoning task, and its units’ activities mimics the classical findings of the ramping pattern of the parietal neurons that reflects the evidence accumulation process during decision making. We further demonstrate that the model carries out Bayesian inference and may support meta-cognition such as confidence with additional tasks. Finally, we show how the network model achieves adaptive behavior with an approach distinct from reinforcement learning. Our work pieces together many experimental findings in decision making and reinforcement learning and provides a unified framework for the flexible and adaptive behavior of the brain.

scholarly journals Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning

2021 ◽  
Vol 11 (22) ◽  
pp. 10595
Author(s):  
Wenlong Zhao ◽  
Zhijun Meng ◽  
Kaipeng Wang ◽  
Jiahui Zhang ◽  
Shaoze Lu
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Tracking Control ◽  
Spatial Information ◽  
Ground Truth ◽  
Tracking Task ◽  
Learning To Learn ◽  
Low Level ◽  
Dynamic Target ◽  
High Level

Active tracking control is essential for UAVs to perform autonomous operations in GPS-denied environments. In the active tracking task, UAVs take high-dimensional raw images as input and execute motor actions to actively follow the dynamic target. Most research focuses on three-stage methods, which entail perception first, followed by high-level decision-making based on extracted spatial information of the dynamic target, and then UAV movement control, using a low-level dynamic controller. Perception methods based on deep neural networks are powerful but require considerable effort for manual ground truth labeling. Instead, we unify the perception and decision-making stages using a high-level controller and then leverage deep reinforcement learning to learn the mapping from raw images to the high-level action commands in the V-REP-based environment, where simulation data are infinite and inexpensive. This end-to-end method also has the advantages of a small parameter size and reduced effort requirements for parameter turning in the decision-making stage. The high-level controller, which has a novel architecture, explicitly encodes the spatial and temporal features of the dynamic target. Auxiliary segmentation and motion-in-depth losses are introduced to generate denser training signals for the high-level controller’s fast and stable training. The high-level controller and a conventional low-level PID controller constitute our hierarchical active tracking control framework for the UAVs’ active tracking task. Simulation experiments show that our controller trained with several augmentation techniques sufficiently generalizes dynamic targets with random appearances and velocities, and achieves significantly better performance, compared with three-stage methods.

Mental Contrasting of Future and Reality

2010 ◽  
Vol 9 (3) ◽  
pp. 138-144 ◽  
Author(s):  
Gabriele Oettingen ◽  
Doris Mayer ◽  
Babette Brinkmann
Keyword(s):  
Decision Making ◽  
Everyday Life ◽  
Brief Intervention ◽  
Middle Level ◽  
The Other ◽  
Metacognitive Strategy ◽  
Mental Contrasting ◽  
Low Expectations

Mental contrasting of a desired future with present reality leads to expectancy-dependent goal commitments, whereas focusing on the desired future only makes people commit to goals regardless of their high or low expectations for success. In the present brief intervention we randomly assigned middle-level managers (N = 52) to two conditions. Participants in one condition were taught to use mental contrasting regarding their everyday concerns, while participants in the other condition were taught to indulge. Two weeks later, participants in the mental-contrasting condition reported to have fared better in managing their time and decision making during everyday life than those in the indulging condition. By helping people to set expectancy-dependent goals, teaching the metacognitive strategy of mental contrasting can be a cost- and time-effective tool to help people manage the demands of their everyday life.

Trust: A National Project

2015 ◽  
pp. 138-146
Author(s):  
N. Rozinskaya ◽  
I. Rozinskiy
Keyword(s):  
Social Capital ◽  
Civil Society ◽  
Everyday Life ◽  
Western Europe ◽  
External Pressure ◽  
National Project ◽  
Low Level ◽  
General Trust

This article deals with the genesis of general trust and social capital in contemporary Russia, which faces the external pressure. The low level of general trust is noted, its economic, social and everyday life implications are considered, an explanation of Russia’s lower than in western Europe level of trust is provided. Considering society’s level of trust and social capital as externalia, the authors conclude that there is a necessity to "produce" trust intentionally. Promotion of collective charity is proposed as a mechanism of such "production". It is stressed that in order to activate the potential of trust in a society, there is a need for ideological and symbolic basis linked to its history. Russian People’s Unity Day, understood as the birthday of Russian civil society, is proposed to be used in this respect.

Sign in / Sign up

Export Citation Format

Share Document