scholarly journals Safe Policies for Factored Partially Observable Stochastic Games

Author(s):  
Steven Carr ◽  
Nils Jansen ◽  
Sudarshanan Bharadwaj ◽  
Matthijs Spaan ◽  
Ufuk Topcu
Author(s):  
Karel Horák ◽  
Branislav Bošanský

In many real-world problems, there is a dynamic interaction between competitive agents. Partially observable stochastic games (POSGs) are among the most general formal models that capture such dynamic scenarios. The model captures stochastic events, partial information of players about the environment, and the scenario does not have a fixed horizon. Solving POSGs in the most general setting is intractable.Therefore, the research has been focused on subclasses of POSGs that have a value of the game and admit designing (approximate) optimal algorithms. We propose such a subclass for two-player zero-sum games with discounted-sum objective function—POSGs with public observations (POPOSGs)—where each player is able to reconstruct beliefs of the other player over the unobserved states. Our results include: (1) theoretical analysis of PO-POSGs and their value functions showing convexity (concavity) in beliefs of maximizing (minimizing) player, (2) a novel algorithm for approximating the value of the game, and (3) a practical demonstration of scalability of our algorithm. Experimental results show that our algorithm can closely approximate the value of non-trivial games with hundreds of states.


Author(s):  
Karel Horák ◽  
Branislav Bošanský ◽  
Christopher Kiekintveld ◽  
Charles Kamhoua

Value methods for solving stochastic games with partial observability model the uncertainty of the players as a probability distribution over possible states, where the dimension of the belief space is the number of states. For many practical problems, there are exponentially many states which causes scalability problems. We propose an abstraction technique that addresses this curse of dimensionality by projecting the high-dimensional beliefs onto characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our main contributions are (1) a novel compact representation of the uncertainty in partially observable stochastic games and (2) a novel algorithm using this representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm using the compact representation dramatically increases scalability compared to the state of the art.


2019 ◽  
Vol 87 ◽  
pp. 101579 ◽  
Author(s):  
Karel Horák ◽  
Branislav Bošanský ◽  
Petr Tomášek ◽  
Christopher Kiekintveld ◽  
Charles Kamhoua

2021 ◽  
Vol 420 ◽  
pp. 36-56
Author(s):  
Roi Ceren ◽  
Keyang He ◽  
Prashant Doshi ◽  
Bikramjit Banerjee

Sign in / Sign up

Export Citation Format

Share Document