scholarly journals No substantial change in the balance between model-free and model-based control via training on the two-step task

2019 ◽  
Vol 15 (11) ◽  
pp. e1007443
Author(s):  
Elmar D. Grosskurth ◽  
Dominik R. Bach ◽  
Marcos Economides ◽  
Quentin J. M. Huys ◽  
Lisa Holper
Keyword(s):  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Substantial Change

scholarly journals Hand interception of occluded motion in humans: a test of model-based vs. on-line control

2015 ◽  
Vol 114 (3) ◽  
pp. 1577-1592 ◽  
Author(s):  
Barbara La Scaleia ◽  
Myrka Zago ◽  
Francesco Lacquaniti
Keyword(s):  
Visual Information ◽  
Target Motion ◽  
Physical Models ◽  
Model Based Control ◽  
Projectile Motion ◽  
Starting Position ◽  
Model Based ◽  
Model Free ◽  
Ball Velocity ◽  
On Line

Two control schemes have been hypothesized for the manual interception of fast visual targets. In the model-free on-line control, extrapolation of target motion is based on continuous visual information, without resorting to physical models. In the model-based control, instead, a prior model of target motion predicts the future spatiotemporal trajectory. To distinguish between the two hypotheses in the case of projectile motion, we asked participants to hit a ball that rolled down an incline at 0.2 g and then fell in air at 1 g along a parabola. By varying starting position, ball velocity and trajectory differed between trials. Motion on the incline was always visible, whereas parabolic motion was either visible or occluded. We found that participants were equally successful at hitting the falling ball in both visible and occluded conditions. Moreover, in different trials the intersection points were distributed along the parabolic trajectories of the ball, indicating that subjects were able to extrapolate an extended segment of the target trajectory. Remarkably, this trend was observed even at the very first repetition of movements. These results are consistent with the hypothesis of model-based control, but not with on-line control. Indeed, ball path and speed during the occlusion could not be extrapolated solely from the kinematic information obtained during the preceding visible phase. The only way to extrapolate ball motion correctly during the occlusion was to assume that the ball would fall under gravity and air drag when hidden from view. Such an assumption had to be derived from prior experience.

Nonlinear Model Reduction for Automotive System Descriptions

2001 ◽  
Author(s):  
Zeyu Liu ◽  
John Wagner
Keyword(s):  
Dynamic System ◽  
Model Reduction ◽  
Nonlinear Model ◽  
Order System ◽  
Control Algorithms ◽  
Model Based Control ◽  
Reduction Strategy ◽  
Nonlinear Model Reduction ◽  
Model Based ◽  
Model Free

Abstract The mathematical modeling of dynamic systems is an important task in the design, analysis, and implementation of advanced automotive control systems. Although most vehicle control algorithms tend to use model-free calibration architectures, a need exists to migrate to model-based control algorithms which offer greater operating performance. However, in many instances, the analytical descriptions are too complex for real-time powertrain and chassis model-based control algorithms. Therefore, model reduction strategies may be applied to transform the original model into a simplified lower-order form while preserving the dynamic characteristics of the original high-order system. In this paper, an empirical gramian balanced nonlinear model reduction strategy is examined for the simplification process of dynamic system descriptions. The empirical gramians may be computed using either experimental or simulation data. These gramians are then balanced and unimportant system dynamics truncated. For comparison purposes, a Taylor Series linearization will also be introduced to linearize the original nonlinear system about an equilibrium operating point and then a balanced realization linear reduction strategy will be applied. To demonstrate the functionality of each model reduction strategy, two nonlinear dynamic system models are investigated and respective transient performances compared.

scholarly journals Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments.

2021 ◽  
Author(s):  
Luca Rene Bruder ◽  
Ben Wagner ◽  
David Mathar ◽  
Jan Peters
Keyword(s):  
Pathological Gambling ◽  
Electrodermal Activity ◽  
Temporal Discounting ◽  
Sports Betting ◽  
Reduced Model ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Environment Exposure

High-performance virtual reality (VR) technology has opened new possibilities for the examination of the reactivity towards addiction-related cues (cue-reactivity) in addiction. In this preregistered study (https://osf.io/4mrta), we investigated the subjective, physiological, and behavioral effects of gambling-related VR environment exposure in participants reporting frequent or pathological gambling (n=31) as well as non-gambling controls (n=29). On two separate days, participants explored two rich and navigable VR-environments (neutral: cafe vs. gambling-related: casino/sports-betting facility), while electrodermal activity and heart rate were continuously measured using remote sensors. Within VR, participants performed a temporal discounting task and a sequential decision-making task designed to assess model-based and model-free contributions to behavior. Replicating previous findings, we found strong evidence for increased temporal discounting and reduced model-based control in participants reporting frequent or pathological gambling. Although VR gambling environment exposure increased subjective craving, there was if anything inconclusive evidence for further behavioral or physiological effects. Instead, VR exposure substantially increased physiological arousal (electrodermal activity), across groups and conditions. VR is a promising tool for the investigation of context effects in addiction, but some caution is warranted since effects of real gambling environments might not generally replicate in VR. Future studies should delineate how factors such as cognitive load and ecological validity could be balanced to create a more naturalistic VR experience.

scholarly journals Έλεγχος εξασφάλισης κύλισης σε σφαιρικά ρομποτικά ακροδάκτυλα

2016 ◽  
Author(s):  
Λεωνίδας Δρούκας
Keyword(s):  
Performance Control ◽  
Model Based Control ◽  
Prescribed Performance ◽  
Model Based ◽  
Model Free ◽  
Prescribed Performance Control ◽  
Model Free Control ◽  
Control Methodology

Όπως στην περίπτωση του ανθρώπου έτσι και στην περίπτωση των δακτύλων ενός ρομποτικού χεριού, η κίνηση μέσω κύλισης των ακροδακτύλων είναι καθοριστική για την επίτευξη της ευσταθούς σύλληψης και του ομαλού χειρισμού ενός αντικειμένου από το ρομποτικό χέρι. Σε αντίθεση με μια κίνηση ολίσθησης πάνω στην επιφάνεια με την οποία έρχεται σε επαφή το ρομποτικό δάκτυλο, η κύλιση του ακροδακτύλου βοηθάει στην ακριβέστερη τοποθέτησή του πάνω στην επιφάνεια αυτή, συμβάλλοντας έτσι στον ομαλότερο συνολικά χειρισμό του εκάστοτε αντικειμένου όπως για παράδειγμα η μετακίνηση ή η περιστροφή του. Στην υπάρχουσα βιβλιογραφία, οι περισσότεροι ελεγκτικοί νόμοι που έχουν προταθεί σχεδιάζονται με βάση ένα εξιδανικευμένο μοντέλο του συστήματος ρομποτικού δακτύλου - επιφάνειας επαφής. Το μοντέλο αυτό περιλαμβάνει ενσωματωμένους τους δεσμούς κύλισης του ακροδακτύλου, με αποτέλεσμα η κίνηση κύλισης να θεωρείται επί της ουσίας ως κάτι δεδομένο και εκ των προτέρων εξασφαλισμένο. Στην πραγματικότητα φυσικά το παραπάνω δεν ισχύει, καθώς οι εκάστοτε υπάρχουσες συνθήκες τριβής που εξαρτώνται από τα υλικά του ακροδακτύλου και της επιφάνειας μπορεί να ευνοούν λίγο ή και καθόλου την κύλιση του πρώτου πάνω στην δεύτερη. Έτσι στις περιπτώσεις αυτές, οι παραπάνω ελεγκτικές μεθοδολογίες είναι πολύ πιθανό να οδηγήσουν το ακροδάκτυλο σε μια κίνηση που θα αποτελείται από συνδυασμένη κύλιση και ολίσθηση ή ακόμη και μόνον από ολίσθηση. Στην διδακτορική αυτή διατριβή, η εξασφάλιση της κύλισης του ακροδακτύλου πάνω σε μια επιφάνεια επαφής δεν θεωρείται δεδομένη από πριν, αλλά λαμβάνεται υπόψιν ως ένας επιπλέον στόχος ελέγχου. Οι ελεγκτές που προτείνονται επιτυγχάνουν την διασφάλιση της κύλισης του σφαιρικού άκρου ενός ρομποτικού δακτύλου σε μια οποιαδήποτε επιφάνεια με άγνωστες ή και μη ευνοϊκές συνθήκες τριβής, συνδυάζοντας παράλληλα την επίτευξη του ευρέως διαδεδομένου στη ρομποτική ελέγχου της θέσης του ακροδακτύλου και του μέτρου της ασκούμενης από αυτό κάθετης δύναμης. Σχεδιάζονται ελεγκτές οι οποίοι είτε βασίζονται στην γνώση του μοντέλου του ρομποτικού δακτύλου (model based control), είτε δεν απαιτούν καμία τέτοια πληροφορία (model free control). Για την σχεδίαση των τελευταίων χρησιμοποιείται η Μέθοδος Ελέγχου Προδιαγεγραμμένης Επίδοσης (Prescribed Performance Control Methodology). Για όλους τους προτεινόμενους ελεγκτές, παρουσιάζεται η θεωρητική σχεδίαση και η προσομοιακή τους αξιολόγηση που αποδεικνύουν την εξασφάλιση του βασικότερου ελεγκτικού μας στόχου, δηλαδή της κύλισης του ρομποτικού ακροδακτύλου πάνω στην επιφάνεια επαφής, ενώ σε ορισμένες περιπτώσεις παρουσιάζονται επιπροσθέτως και πειραματικά αποτελέσματα που ενισχύουν ακόμα περισσότερο την αποτελεσματικότητα των αντίστοιχων προτεινόμενων ελεγκτών. Τέλος, η εξασφάλιση της κύλισης του ακροδακτύλου μέσω ενός από τους προτεινόμενους ελεγκτές αξιοποιείται για την εφαρμογή κατάλληλης εφαπτομενικής δύναμης ώστε να επιτευχθεί ο επιθυμητός χειρισμός χωρίς σύλληψη ενός αντικειμένου από ένα ρομποτικό δάκτυλο. Ειδικότερα, παρουσιάζεται μέσω προσομοιώσεων η επιτυχής μετακίνηση ή/και περιστροφή ενός επίπεδου και ενός κυλινδρικού αντικειμένου μέσω χειρισμού χωρίς σύλληψη από ένα ρομποτικό δάκτυλο με εκμετάλλευση της κύλισης του άκρου του πάνω στο εκάστοτε αντικείμενο.

scholarly journals Testing the Possibility of Model-based Pavlovian Control of Attention to Threat

2019 ◽  
Vol 31 (1) ◽  
pp. 36-48 ◽  
Author(s):  
Deborah Talmi ◽  
Martina Slapkova ◽  
Matthias J. Wieser
Keyword(s):  
Attention Capture ◽  
Conditioned Stimuli ◽  
Learning Stage ◽  
Internal Representations ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Role Of Information ◽  
Instrumental Control

Signals for reward or punishment attract attention preferentially, a principle termed value-modulated attention capture (VMAC). The mechanisms that govern the allocation of attention can be described with a terminology that is more often applied to the control of overt behaviors, namely, the distinction between instrumental and Pavlovian control, and between model-free and model-based control. Although instrumental control of VMAC can be either model-free or model-based, it is not known whether Pavlovian control of VMAC can be model-based. To decide whether this is possible, we measured steady-state visual evoked potentials (SSVEPs) while 20 healthy adults took part in a novel task. During the learning stage, participants underwent aversive threat conditioning with two conditioned stimuli (CSs): one that predicted pain (CS+) and one that predicted safety (CS−). Instructions given before the test stage allowed participants to infer whether novel, ambiguous CSs (new_CS+/new_CS−) were threatening or safe. Correct inference required combining stored internal representations and new propositional information, the hallmark of model-based control. SSVEP amplitudes quantified the amount of attention allocated to novel CSs on their very first presentation, before they were ever reinforced. We found that SSVEPs were higher for new_CS+ than new_CS−. This result is potentially indicative of model-based Pavlovian control of VMAC, but additional controls are necessary to verify this conclusively. This result underlines the potential transformative role of information and inference in emotion regulation.

scholarly journals Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making

2015 ◽  
Vol 112 (5) ◽  
pp. 1595-1600 ◽  
Author(s):  
Lorenz Deserno ◽  
Quentin J. M. Huys ◽  
Rebecca Boehme ◽  
Ralph Buchert ◽  
Hans-Jochen Heinze ◽  
...  
Keyword(s):  
Ventral Striatum ◽  
Behavioral Control ◽  
Striatal Dopamine ◽  
Quantitative Relation ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Neuropsychiatric Diseases ◽  
Neural Signatures

Dual system theories suggest that behavioral control is parsed between a deliberative “model-based” and a more reflexive “model-free” system. A balance of control exerted by these systems is thought to be related to dopamine neurotransmission. However, in the absence of direct measures of human dopamine, it remains unknown whether this reflects a quantitative relation with dopamine either in the striatum or other brain areas. Using a sequential decision task performed during functional magnetic resonance imaging, combined with striatal measures of dopamine using [18F]DOPA positron emission tomography, we show that higher presynaptic ventral striatal dopamine levels were associated with a behavioral bias toward more model-based control. Higher presynaptic dopamine in ventral striatum was associated with greater coding of model-based signatures in lateral prefrontal cortex and diminished coding of model-free prediction errors in ventral striatum. Thus, interindividual variability in ventral striatal presynaptic dopamine reflects a balance in the behavioral expression and the neural signatures of model-free and model-based control. Our data provide a novel perspective on how alterations in presynaptic dopamine levels might be accompanied by a disruption of behavioral control as observed in aging or neuropsychiatric diseases such as schizophrenia and addiction.

scholarly journals Fronto-striatal structures related with model-based control as an endophenotype for obsessive–compulsive disorder

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Meltem I. Kasal ◽  
Lutfullah Besiroglu ◽  
Nabi Zorlu ◽  
Nur Dikmeer ◽  
Aslıhan Bilge ◽  
...  
Keyword(s):  
Reaction Time ◽  
Obsessive Compulsive Disorder ◽  
Dorsolateral Prefrontal Cortex ◽  
Obsessive Compulsive ◽  
Model Based Control ◽  
Compulsive Disorder ◽  
Model Based ◽  
Model Free ◽  
Dorsolateral Prefrontal ◽  
Model Free Control

AbstractRecent theories suggest a shift from model-based goal-directed to model-free habitual decision-making in obsessive–compulsive disorder (OCD). However, it is yet unclear, whether this shift in the decision process is heritable. We investigated 32 patients with OCD, 27 unaffected siblings (SIBs) and 31 healthy controls (HCs) using the two-step task. We computed behavioral and reaction time analyses and fitted a computational model to assess the balance between model-based and model-free control. 80 subjects also underwent structural imaging. We observed a significant ordered effect for the shift towards model-free control in the direction OCD > SIB > HC in our computational parameter of interest. However less directed analyses revealed no shift towards model-free control in OCDs. Nonetheless, we found evidence for reduced model-based control in OCDs compared to HCs and SIBs via 2nd stage reaction time analyses. In this measure SIBs also showed higher levels of model-based control than HCs. Across all subjects these effects were associated with the surface area of the left medial/right dorsolateral prefrontal cortex. Moreover, correlations between bilateral putamen/right caudate volumes and these effects varied as a function of group: they were negative in SIBs and OCDs, but positive in HCs. Associations between fronto-striatal regions and model-based reaction time effects point to a potential endophenotype for OCD.

scholarly journals Model-based Pavlovian control of attention to threat

2018 ◽  
Author(s):  
D Talmi ◽  
M Slapkova ◽  
MJ Wieser
Keyword(s):  
Task Design ◽  
Attention Capture ◽  
Learning Stage ◽  
Internal Representations ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Role Of Information ◽  
Instrumental Control

AbstractSignals for reward or punishment attract attention preferentially, a principle termed ‘value-modulated attention capture’ (VMAC). The mechanisms that govern the allocation of attention resources can be productively described with a terminology that is more often applied to the control of overt behaviours, namely, the distinction between instrumental and Pavlovian control, and between model-free and model-based control. While instrumental control of VMAC can be either model-free or model-based, it is not known whether Pavlovian control of VMAC can be model-based. To decide whether this is possible we measured Steady-State Visual Evoked Potentials (SSVEPs) while 20 healthy adults took part in a novel task. During the learning stage participants underwent aversive threat conditioning with two CSs, one that predicted pain (CS+) and one that predicted safety (CS-). Instructions given prior to the test stage in the task allowed participants to infer whether novel, ambiguous CSs (new CS+/ new CS-) were threatening or safe. Correct inference required combining stored internal representations and new propositional information, the hallmark of model-based control. SSVEP amplitudes quantified the amount of attention allocated to novel CSs on their very first presentation, before they were ever reinforced. We found that SSVEPs were higher for new CS+ than new CS-. Because task design precluded model-free or instrumental control this result demonstrates a model-based Pavlovian control of VMAC. It confirms, in the domain of internal resource allocation, the model-based Pavlovian control of incentive behaviour and underlines the potential transformative role of information as an emotion regulation technique.

Sign in / Sign up

Export Citation Format

Share Document