scholarly journals Modeling dynamic allocation of effort in a sequential task using discounting models

2019 ◽  
Author(s):  
Cuevas Rivera Darío ◽  
Strobel Alexander ◽  
Goschke Thomas ◽  
Stefan J. Kiebel
Keyword(s):  
Model Comparison ◽  
Formal Model ◽  
Model Parameters ◽  
Sequential Decision ◽  
Dynamic Allocation ◽  
Planning Strategy ◽  
Effort Discounting ◽  
Binary Choices ◽  
Minimum Number ◽  
Neural Underpinnings

Most rewards in our lives require effort to obtain them. It is known that effort is seen by humans as carrying an intrinsic disutility which devalues the obtainable reward. Established models for effort discounting account for this by using participant-specific discounting parameters inferred from experiments. These parameters offer only a static glance into the bigger picture of effort exertion. The mechanism underlying the dynamic changes in a participant’s willingness to exert effort is still unclear and an active topic of research. Here, we modeled dynamic effort exertion as a consequence of effort- and probability-discounting mechanisms during goal reaching, sequential behavior. To do this, we developed a novel sequential decision-making task in which participants make binary choices to reach a minimum number of points. Importantly, the time points and circumstances of effort allocation are decided by participants according to their own preferences and not imposed directly by the task. Using the computational model to analyze participants’ choices, we show that the dynamics of effort exertion arise from a combination of changing task needs and forward planning. In other words, the interplay between a participant’s inferred discounting parameters is sufficient to explain the dynamic allocation of effort during goal reaching. Using formal model comparison, we also infer the forward-planning strategy used by participants. The model allows us to characterize a participant’s effort exertion in terms of only a few parameters. Moreover, the model can be adapted to a number of tasks used in establishing the neural underpinnings of forward-planning behavior and meta-control, allowing for the characterization of behavior in terms of model parameters.

An Information-theoretical Model for Breast Cancer Detection

2008 ◽  
Vol 47 (04) ◽  
pp. 322-327 ◽  
Author(s):  
D. Blokh ◽  
N. Zurgil ◽  
I. Stambler ◽  
E. Afrimzon ◽  
Y. Shafran ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Cancer Detection ◽  
Hamming Distance ◽  
Formal Model ◽  
Methodological Approach ◽  
Breast Cancer Detection ◽  
Receptor Expression ◽  
Diagnostic Model ◽  
Model Parameters ◽  
Two Stages

Summary Objectives: Formal diagnostic modeling is an important line of modern biological and medical research. The construction of a formal diagnostic model consists of two stages: first, the estimation of correlation between model parameters and the disease under consideration; and second, the construction of a diagnostic decision rule using these correlation estimates. A serious drawback of current diagnostic models is the absence of a unified mathematical methodological approach to implementing these two stages. The absence of aunified approach makesthe theoretical/biomedical substantiation of diagnostic rules difficult and reduces the efficacyofactual diagnostic model application. Methods: The present study constructs a formal model for breast cancer detection. The diagnostic model is based on information theory. Normalized mutual information is chosen as the measure of relevance between parameters and the patterns studied. The “nearest neighbor” rule is utilized for diagnosis, while the distance between elements is the weighted Hamming distance. The model concomitantly employs cellular fluorescence polarization as the quantitative input parameter and cell receptor expression as qualitative parameters. Results: Twenty-four healthy individuals and 34 patients (not including the subjects analyzed for the model construction) were tested by the model. Twenty-three healthy subjects and 34 patients were correctly diagnosed. Conclusions: The proposed diagnostic model is an open one,i.e.it can accommodate new additional parameters, which may increase its effectiveness.

scholarly journals Corticostriatal Circuits Encode the Subjective Value of Perceived Control

2019 ◽  
Vol 29 (12) ◽  
pp. 5049-5060 ◽  
Author(s):  
Kainan S Wang ◽  
Mauricio R Delgado
Keyword(s):  
Perceived Control ◽  
Well Being ◽  
Control Option ◽  
Subjective Value ◽  
Binary Choices ◽  
Mere Presence ◽  
The Difference ◽  
Neural Underpinnings ◽  
Evoked Activity

AbstractThe ability to perceive and exercise control over an outcome is both desirable and beneficial to our well-being. It has been shown that animals and humans alike exhibit behavioral bias towards seeking control and that such bias recruits the ventromedial prefrontal cortex (vmPFC) and striatum. Yet, this bias remains to be quantitatively captured and studied neurally. Here, we employed a behavioral task to measure the preference for control and characterize its neural underpinnings. Participants made a series of binary choices between having control and no-control over a game for monetary reward. The mere presence of the control option evoked activity in the ventral striatum. Importantly, we manipulated the expected value (EV) of each choice pair to extract the pairing where participants were equally likely to choose either option. The difference in EV between the options at this point of equivalence was inferred as the subjective value of control. Strikingly, perceiving control inflated the reward value of the associated option by 30% and this value inflation was tracked by the vmPFC. Altogether, these results capture the subjective value of perceived control inherent in decision making and highlight the role of corticostriatal circuitry in the perception of control.

scholarly journals The seven sisters DANCe

2018 ◽  
Vol 612 ◽  
pp. A70 ◽  
Author(s):  
J. Olivares ◽  
E. Moraux ◽  
L. M. Sarro ◽  
H. Bouy ◽  
A. Berihuete ◽  
...  
Keyword(s):  
Spatial Distribution ◽  
Model Comparison ◽  
Precise Determination ◽  
Model Parameters ◽  
Data Sets ◽  
Adequate Model ◽  
Comparison Results ◽  
Bayesian Evidence ◽  
Mass Segregation

Context. Membership analyses of the DANCe and Tycho + DANCe data sets provide the largest and least contaminated sample of Pleiades candidate members to date. Aims. We aim at reassessing the different proposals for the number surface density of the Pleiades in the light of the new and most complete list of candidate members, and inferring the parameters of the most adequate model. Methods. We compute the Bayesian evidence and Bayes Factors for variations of the classical radial models. These include elliptical symmetry, and luminosity segregation. As a by-product of the model comparison, we obtain posterior distributions for each set of model parameters. Results. We find that the model comparison results depend on the spatial extent of the region used for the analysis. For a circle of 11.5 parsecs around the cluster centre (the most homogeneous and complete region), we find no compelling reason to abandon King’s model, although the Generalised King model introduced here has slightly better fitting properties. Furthermore, we find strong evidence against radially symmetric models when compared to the elliptic extensions. Finally, we find that including mass segregation in the form of luminosity segregation in the J band is strongly supported in all our models. Conclusions. We have put the question of the projected spatial distribution of the Pleiades cluster on a solid probabilistic framework, and inferred its properties using the most exhaustive and least contaminated list of Pleiades candidate members available to date. Our results suggest however that this sample may still lack about 20% of the expected number of cluster members. Therefore, this study should be revised when the completeness and homogeneity of the data can be extended beyond the 11.5 parsecs limit. Such a study will allow for more precise determination of the Pleiades spatial distribution, its tidal radius, ellipticity, number of objects and total mass.

Hysteretic Poisson INGARCH model for integer-valued time series

2017 ◽  
Vol 17 (6) ◽  
pp. 401-422 ◽  
Author(s):  
Buu-Chau Truong ◽  
Cathy WS Chen ◽  
Songsak Sriboonchitta
Keyword(s):  
Time Series ◽  
Model Comparison ◽  
Monte Carlo Sampling ◽  
Information Criteria ◽  
Model Parameters ◽  
Modelling Framework ◽  
South Wales ◽  
Proposed Model ◽  
Over Dispersion ◽  
Ingarch Model

This study proposes a new model for integer-valued time series—the hysteretic Poisson integer-valued generalized autoregressive conditionally heteroskedastic (INGARCH) model—which has an integrated hysteresis zone in the switching mechanism of the conditional expectation. Our modelling framework provides a parsimonious representation of the salient features of integer-valued time series, such as discreteness, over-dispersion, asymmetry and structural change. We adopt Bayesian methods with a Markov chain Monte Carlo sampling scheme to estimate model parameters and utilize the Bayesian information criteria for model comparison. We then apply the proposed model to five real time series of criminal incidents recorded by the New South Wales Police Force in Australia. Simulation results and empirical analysis highlight the better performance of hysteresis in modelling the integer-valued time series.

scholarly journals Inter-trial effects in priming of pop-out: Comparison of computational updating models

2021 ◽  
Vol 17 (9) ◽  
pp. e1009332
Author(s):  
Fredrik Allenmark ◽  
Ahu Gokce ◽  
Thomas Geyer ◽  
Artyom Zinchenko ◽  
Hermann J. Müller ◽  
...  
Keyword(s):  
Model Comparison ◽  
Response Times ◽  
Target Position ◽  
Preceding Trial ◽  
Target Color ◽  
Model Parameters ◽  
Target Feature ◽  
Modeling Framework ◽  
Trial Effect ◽  
Starting Point

In visual search tasks, repeating features or the position of the target results in faster response times. Such inter-trial ‘priming’ effects occur not just for repetitions from the immediately preceding trial but also from trials further back. A paradigm known to produce particularly long-lasting inter-trial effects–of the target-defining feature, target position, and response (feature)–is the ‘priming of pop-out’ (PoP) paradigm, which typically uses sparse search displays and random swapping across trials of target- and distractor-defining features. However, the mechanisms underlying these inter-trial effects are still not well understood. To address this, we applied a modeling framework combining an evidence accumulation (EA) model with different computational updating rules of the model parameters (i.e., the drift rate and starting point of EA) for different aspects of stimulus history, to data from a (previously published) PoP study that had revealed significant inter-trial effects from several trials back for repetitions of the target color, the target position, and (response-critical) target feature. By performing a systematic model comparison, we aimed to determine which EA model parameter and which updating rule for that parameter best accounts for each inter-trial effect and the associated n-back temporal profile. We found that, in general, our modeling framework could accurately predict the n-back temporal profiles. Further, target color- and position-based inter-trial effects were best understood as arising from redistribution of a limited-capacity weight resource which determines the EA rate. In contrast, response-based inter-trial effects were best explained by a bias of the starting point towards the response associated with a previous target; this bias appeared largely tied to the position of the target. These findings elucidate how our cognitive system continually tracks, and updates an internal predictive model of, a number of separable stimulus and response parameters in order to optimize task performance.

scholarly journals Assessing model mismatch and model selection in a Bayesian uncertainty quantification analysis of a fluid-dynamics model of pulmonary blood circulation

2020 ◽  
Vol 17 (173) ◽  
pp. 20200886
Author(s):  
L. Mihaela Paun ◽  
Mitchel J. Colebank ◽  
Mette S. Olufsen ◽  
Nicholas A. Hill ◽  
Dirk Husmeier
Keyword(s):  
Model Selection ◽  
Mean Squared Error ◽  
Formal Model ◽  
Circulation Model ◽  
Noise Model ◽  
Parameter Estimates ◽  
Model Parameters ◽  
Imaging Data ◽  
Micro Computed Tomography ◽  
Model Mismatch

This study uses Bayesian inference to quantify the uncertainty of model parameters and haemodynamic predictions in a one-dimensional pulmonary circulation model based on an integration of mouse haemodynamic and micro-computed tomography imaging data. We emphasize an often neglected, though important source of uncertainty: in the mathematical model form due to the discrepancy between the model and the reality, and in the measurements due to the wrong noise model (jointly called ‘model mismatch’). We demonstrate that minimizing the mean squared error between the measured and the predicted data (the conventional method) in the presence of model mismatch leads to biased and overly confident parameter estimates and haemodynamic predictions. We show that our proposed method allowing for model mismatch, which we represent with Gaussian processes, corrects the bias. Additionally, we compare a linear and a nonlinear wall model, as well as models with different vessel stiffness relations. We use formal model selection analysis based on the Watanabe Akaike information criterion to select the model that best predicts the pulmonary haemodynamics. Results show that the nonlinear pressure–area relationship with stiffness dependent on the unstressed radius predicts best the data measured in a control mouse.

scholarly journals Neurophysiological dynamics of phrase-structure building during sentence processing

2017 ◽  
Vol 114 (18) ◽  
pp. E3669-E3678 ◽  
Author(s):  
Matthew J. Nelson ◽  
Imen El Karoui ◽  
Kristof Giber ◽  
Xiaofang Yang ◽  
Laurent Cohen ◽  
...  
Keyword(s):  
Sentence Processing ◽  
Model Comparison ◽  
Formal Model ◽  
Syntactic Structure ◽  
Brain Activity ◽  
Transition Probability ◽  
Middle Temporal Gyrus ◽  
Similar Amount ◽  
Gamma Power ◽  
The Brain

Although sentences unfold sequentially, one word at a time, most linguistic theories propose that their underlying syntactic structure involves a tree of nested phrases rather than a linear sequence of words. Whether and how the brain builds such structures, however, remains largely unknown. Here, we used human intracranial recordings and visual word-by-word presentation of sentences and word lists to investigate how left-hemispheric brain activity varies during the formation of phrase structures. In a broad set of language-related areas, comprising multiple superior temporal and inferior frontal sites, high-gamma power increased with each successive word in a sentence but decreased suddenly whenever words could be merged into a phrase. Regression analyses showed that each additional word or multiword phrase contributed a similar amount of additional brain activity, providing evidence for a merge operation that applies equally to linguistic objects of arbitrary complexity. More superficial models of language, based solely on sequential transition probability over lexical and syntactic categories, only captured activity in the posterior middle temporal gyrus. Formal model comparison indicated that the model of multiword phrase construction provided a better fit than probability-based models at most sites in superior temporal and inferior frontal cortices. Activity in those regions was consistent with a neural implementation of a bottom-up or left-corner parser of the incoming language stream. Our results provide initial intracranial evidence for the neurophysiological reality of the merge operation postulated by linguists and suggest that the brain compresses syntactically well-formed sequences of words into a hierarchy of nested phrases.

scholarly journals Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications

2019 ◽  
Vol 33 ◽  
pp. 7749-7758
Author(s):  
Daniel S. Brown ◽  
Scott Niekum
Keyword(s):  
Reinforcement Learning ◽  
Set Cover ◽  
Sequential Decision ◽  
Inverse Reinforcement Learning ◽  
Reward Function ◽  
Set Cover Problem ◽  
Efficient Approximation Algorithm ◽  
Minimum Number ◽  
Teaching Problem ◽  
Novel Applications

Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization. However, despite much recent interest in IRL, little work has been done to understand the minimum set of demonstrations needed to teach a specific sequential decisionmaking task. We formalize the problem of finding maximally informative demonstrations for IRL as a machine teaching problem where the goal is to find the minimum number of demonstrations needed to specify the reward equivalence class of the demonstrator. We extend previous work on algorithmic teaching for sequential decision-making tasks by showing a reduction to the set cover problem which enables an efficient approximation algorithm for determining the set of maximallyinformative demonstrations. We apply our proposed machine teaching algorithm to two novel applications: providing a lower bound on the number of queries needed to learn a policy using active IRL and developing a novel IRL algorithm that can learn more efficiently from informative demonstrations than a standard IRL approach.

A Micromechanical Method to Predict Macroscopic Behavior of Brittle Creep Failure in Rock

2016 ◽  
Vol 08 (08) ◽  
pp. 1650089 ◽  
Author(s):  
Xiaozhao Li ◽  
Zhushan Shao
Keyword(s):  
Model Comparison ◽  
Time Dependent ◽  
Similar Process ◽  
Model Parameters ◽  
Suggested Model ◽  
Wing Crack ◽  
Creep Failure ◽  
Brittle Creep ◽  
Deep Underground ◽  
Definition Of

Brittle creep in rock has great significance for the prediction of important geohazards and stability of deep underground excavations. A major challenge in this area is to link the time-dependent cracking with macroscopic mechanical behavior. In this paper, Ashby and Sammis’ microcrack model and Charles’ crack growth law are employed to investigate the time-dependent cracking during brittle creep in rock. Based on the macroscopic and micromechanical definition of damage in rock, a new theoretical model is suggested to establish the linkage between microcrack length and macroscopic strain. In order to verify the rationality of the suggested model, comparison between theoretical and experimental results is presented. Using this new model, brittle creep of Sanxia granite is investigated and discussed in detail. It is found that evolutions of wing crack length, strain, and damage perform a similar process during brittle creep and could be divided into three phases. Effects of model parameters on creep failure behaviors also are studied.

scholarly journals Neural computations underlying inverse reinforcement learning in the human brain

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Sven Collette ◽  
Wolfgang M Pauli ◽  
Peter Bossaerts ◽  
John O'Doherty
Keyword(s):  
Reinforcement Learning ◽  
Human Brain ◽  
Model Comparison ◽  
Formal Model ◽  
Food Preferences ◽  
Slot Machines ◽  
Inverse Reinforcement Learning ◽  
Neural Computations ◽  
Fmri Analysis ◽  
Imitation Strategy

In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.

Sign in / Sign up

Export Citation Format

Share Document