scholarly journals Decision Making with STPA through Markov Decision Process, a Theoretic Framework for Safe Human-Robot Collaboration

2021 ◽  
Vol 11 (11) ◽  
pp. 5212
Author(s):  
Angeliki Zacharaki ◽  
Ioannis Kostavelis ◽  
Ioannis Dokas
Keyword(s):  
Decision Making ◽  
Situation Awareness ◽  
Process Analysis ◽  
Collaborative Environment ◽  
Current State ◽  
Safety Awareness ◽  
Markov Decision ◽  
Close Cooperation ◽  
Partially Observable ◽  
Human Robot Collaboration

During the last decades, collaborative robots capable of operating out of their cages are widely used in industry to assist humans in mundane and harsh manufacturing tasks. Although such robots are inherently safe by design, they are commonly accompanied by external sensors and other cyber-physical systems, to facilitate close cooperation with humans, which frequently render the collaborative ecosystem unsafe and prone to hazards. We introduce a method that capitalizes on partially observable Markov decision processes (POMDP) to amalgamate nominal actions of the system along with unsafe control actions posed by the System Theoretic Process Analysis (STPA). A decision-making mechanism that constantly prompts the system into a safer state is realized by providing situation awareness about the safety levels of the collaborative ecosystem by associating the system safety awareness with specific groups of selected actions. POMDP compensates the partial observability and uncertainty of the current state of the collaborative environment and creates safety screening policies that tend to make decisions that balance the system from unsafe to safe states in real time during the operational phase. The theoretical framework is assessed on a simulated human–robot collaborative scenario and proved capable of identifying loss and success scenarios.

scholarly journals Decision Making in Complex Multiagent Contexts: A Tale of Two Frameworks

AI Magazine ◽  
2012 ◽  
Vol 33 (4) ◽  
pp. 82 ◽  
Author(s):  
Prashant J. Doshi
Keyword(s):  
Decision Making ◽  
Markov Decision Process ◽  
Decision Process ◽  
Partial Information ◽  
Autonomous Systems ◽  
Relevant Research ◽  
Physical Context ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable

Decision making is a key feature of autonomous systems. It involves choosing optimally between different lines of action in various information contexts that range from perfectly knowing all aspects of the decision problem to having just partial knowledge about it. The physical context often includes other interacting autonomous systems, typically called agents. In this article, I focus on decision making in a multiagent context with partial information about the problem. Relevant research in this complex but realistic setting has converged around two complementary, general frameworks and also introduced myriad specializations on its way. I put the two frameworks, decentralized partially observable Markov decision process (Dec-POMDP) and the interactive partially observable Markov decision process (I-POMDP), in context and review the foundational algorithms for these frameworks, while briefly discussing the advances in their specializations. I conclude by examining the avenues that research pertaining to these frameworks is pursuing.

scholarly journals Bayesian Inference of Other Minds Explains Human Choices in Group Decision Making

2018 ◽  
Author(s):  
Koosha Khalvati ◽  
Seongmin A. Park ◽  
Saghar Mirbagheri ◽  
Remi Philippe ◽  
Mariateresa Sestito ◽  
...  
Keyword(s):  
Decision Making ◽  
Bayesian Inference ◽  
Markov Decision Processes ◽  
Group Decision Making ◽  
Group Decision ◽  
Decision Processes ◽  
Other Minds ◽  
Markov Decision ◽  
The Mind ◽  
Partially Observable

AbstractTo make decisions in a social context, humans have to predict the behavior of others, an ability that is thought to rely on having a model of other minds known as theory of mind. Such a model becomes especially complex when the number of people one simultaneously interacts is large and the actions are anonymous. Here, we show that in order to make decisions within a large group, humans employ Bayesian inference to model the “mind of the group,” making predictions of others’ decisions while also considering the effects of their own actions on the group as a whole. We present results from a group decision making task known as the Volunteers Dilemma and demonstrate that a Bayesian model based on partially observable Markov decision processes outperforms existing models in quantitatively explaining human behavior. Our results suggest that in group decision making, rather than acting based solely on the rewards received thus far, humans maintain a model of the group and simulate the group’s dynamics into the future in order to choose an action as a member of the group.

scholarly journals Modeling other minds: Bayesian inference explains human choices in group decision-making

2019 ◽  
Vol 5 (11) ◽  
pp. eaax8783 ◽  
Author(s):  
Koosha Khalvati ◽  
Seongmin A. Park ◽  
Saghar Mirbagheri ◽  
Remi Philippe ◽  
Mariateresa Sestito ◽  
...  
Keyword(s):  
Decision Making ◽  
Bayesian Inference ◽  
Group Decision Making ◽  
Group Decision ◽  
Other Minds ◽  
Group Interactions ◽  
Markov Decision ◽  
The Mind ◽  
Partially Observable ◽  
Large Groups

To make decisions in a social context, humans have to predict the behavior of others, an ability that is thought to rely on having a model of other minds known as “theory of mind.” Such a model becomes especially complex when the number of people one simultaneously interacts with is large and actions are anonymous. Here, we present results from a group decision-making task known as the volunteer’s dilemma and demonstrate that a Bayesian model based on partially observable Markov decision processes outperforms existing models in quantitatively predicting human behavior and outcomes of group interactions. Our results suggest that in decision-making tasks involving large groups with anonymous members, humans use Bayesian inference to model the “mind of the group,” making predictions of others’ decisions while also simulating the effects of their own actions on the group’s dynamics in the future.

scholarly journals An Active Inference Approach to Dissecting Reasons for Non-Adherence to Antidepressants

2019 ◽  
Author(s):  
Ryan Smith ◽  
Sahib Khalsa ◽  
Martin Paulus
Keyword(s):  
Decision Making ◽  
Process Model ◽  
Time Course ◽  
Antidepressant Medication ◽  
Influential Factors ◽  
Active Inference ◽  
Individual Level ◽  
Markov Decision ◽  
Decision Making Processes ◽  
Partially Observable

AbstractBackgroundAntidepressant medication adherence is among the most important problems in health care worldwide. Interventions designed to increase adherence have largely failed, pointing towards a critical need to better understand the underlying decision-making processes that contribute to adherence. A computational decision-making model that integrates empirical data with a fundamental action selection principle could be pragmatically useful in 1) making individual level predictions about adherence, and 2) providing an explanatory framework that improves our understanding of non-adherence.MethodsHere we formulate a partially observable Markov decision process model based on the active inference framework that can simulate several processes that plausibly influence adherence decisions.ResultsUsing model simulations of the day-to-day decisions to take a prescribed selective serotonin reuptake inhibitor (SSRI), we show that several distinct parameters in the model can influence adherence decisions in predictable ways. These parameters include differences in policy depth (i.e., how far into the future one considers when deciding), decision uncertainty, beliefs about the predictability (stochasticity) of symptoms, beliefs about the magnitude and time course of symptom reductions and side effects, and the strength of medication-taking habits that one has acquired.ConclusionsClarifying these influential factors will be an important first step toward empirically determining which are contributing to non-adherence to antidepressants in individual patients. The model can also be seamlessly extended to simulate adherence to other medications (by incorporating the known symptom reduction and side effect trajectories of those medications), with the potential promise of identifying which medications may be best suited for different patients.

scholarly journals Patient History-driven Framework for Healthcare Analytics

2019 ◽  
Vol 9 (1S3) ◽  
pp. 403-408
Keyword(s):  
Decision Making ◽  
Patient Care ◽  
Decision Process ◽  
Process Mining ◽  
Event Data ◽  
Medical Field ◽  
Patient History ◽  
Markov Decision ◽  
History Of ◽  
Partially Observable

Wireless body are network (WBAN) is evolving more rapidly due to the development of internet of things (IoT). Decision making is the main concern in medical field which leads to optimization. Medical evidences in patient care improve optimization in patient care. Partially observable markov decision process (POMDP) helps in making accurate decisions with the help of observations and past actions in medical field. Hence dynamic decision making makes it possible. In POMDP, the incremental method is designed to incorporate any immediate change and immediately send updates. In this paper, process mining is applied in finding the history of patients who are travelling from one country to another for in search of job or for doing a major clinical operation. Event data is very much important for handling patient’s history. Event data stores the date and time at which the patient gets consultation. Electronic medical records (EMR) are nothing but storage of all the event data of patients visiting the hospital. Event data gives the evidence of patients when they had a consultation with a doctor. Event data is present anywhere. Partially Observable Markov Decision Process for Patient-history and Careflow mining Algorithm for Heuristic Comparison are presented in this paper. Process mining gives a direct relationship by step by step evaluation and improvement of the process. It also exhibits patient care by identifying the execution errors, understanding the process heterogeneity. The online process mining tool, PROM helps finding the history of patient.

scholarly journals Column Generation Algorithms for Constrained POMDPs

2018 ◽  
Vol 62 ◽  
pp. 489-533 ◽  
Author(s):  
Erwin Walraven ◽  
Matthijs T. J. Spaan
Keyword(s):  
Column Generation ◽  
Decision Process ◽  
State Of The Art ◽  
Single Agent ◽  
Current State ◽  
Exact Evaluation ◽  
Markov Decision ◽  
Secondary Objective ◽  
Partially Observable ◽  
Planning Problems

In several real-world domains it is required to plan ahead while there are finite resources available for executing the plan. The limited availability of resources imposes constraints on the plans that can be executed, which need to be taken into account while computing a plan. A Constrained Partially Observable Markov Decision Process (Constrained POMDP) can be used to model resource-constrained planning problems which include uncertainty and partial observability. Constrained POMDPs provide a framework for computing policies which maximize expected reward, while respecting constraints on a secondary objective such as cost or resource consumption. Column generation for linear programming can be used to obtain Constrained POMDP solutions. This method incrementally adds columns to a linear program, in which each column corresponds to a POMDP policy obtained by solving an unconstrained subproblem. Column generation requires solving a potentially large number of POMDPs, as well as exact evaluation of the resulting policies, which is computationally difficult. We propose a method to solve subproblems in a two-stage fashion using approximation algorithms. First, we use a tailored point-based POMDP algorithm to obtain an approximate subproblem solution. Next, we convert this approximate solution into a policy graph, which we can evaluate efficiently. The resulting algorithm is a new approximate method for Constrained POMDPs in single-agent settings, but also in settings in which multiple independent agents share a global constraint. Experiments based on several domains show that our method outperforms the current state of the art.

Sign in / Sign up

Export Citation Format

Share Document