scholarly journals AN OPTIMAL ASSIGNMENT PROBLEM FOR MULTIPLE OBJECTS PER PERIOD-CASE OF A PARTIALLY OBSERVABLE MARKOV CHAIN

10.5109/13478 ◽  
1999 ◽  
Vol 31 (1) ◽  
pp. 23-34
Author(s):  
Toru Nakai
Keyword(s):  
Markov Chain ◽  
Assignment Problem ◽  
Optimal Assignment ◽  
Multiple Objects ◽  
Partially Observable Markov ◽  
Partially Observable

scholarly journals Task-Aware Verifiable RNN-Based Policies for Partially Observable Markov Decision Processes

2021 ◽  
Vol 72 ◽  
pp. 819-847
Author(s):  
Steven Carr ◽  
Nils Jansen ◽  
Ufuk Topcu
Keyword(s):  
Machine Learning ◽  
Markov Chain ◽  
Temporal Logic ◽  
Markov Decision Processes ◽  
Diagnostic Information ◽  
Decision Processes ◽  
Verification Methods ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable

Partially observable Markov decision processes (POMDPs) are models for sequential decision-making under uncertainty and incomplete information. Machine learning methods typically train recurrent neural networks (RNN) as effective representations of POMDP policies that can efficiently process sequential data. However, it is hard to verify whether the POMDP driven by such RNN-based policies satisfies safety constraints, for instance, given by temporal logic specifications. We propose a novel method that combines techniques from machine learning with the field of formal methods: training an RNN-based policy and then automatically extracting a so-called finite-state controller (FSC) from the RNN. Such FSCs offer a convenient way to verify temporal logic constraints. Implemented on a POMDP, they induce a Markov chain, and probabilistic verification methods can efficiently check whether this induced Markov chain satisfies a temporal logic specification. Using such methods, if the Markov chain does not satisfy the specification, a byproduct of verification is diagnostic information about the states in the POMDP that are critical for the specification. The method exploits this diagnostic information to either adjust the complexity of the extracted FSC or improve the policy by performing focused retraining of the RNN. The method synthesizes policies that satisfy temporal logic specifications for POMDPs with up to millions of states, which are three orders of magnitude larger than comparable approaches.

Sign in / Sign up

Export Citation Format

Share Document