Computational Method and a Numerical Algorithm for Finding the Optimal Control Policy for a Partially Observable System

2009 ◽  
Author(s):  
Viliam Makis ◽  
Michael Jong Kim ◽  
Rui Jiang ◽  
Theodore E. Simos ◽  
George Psihoyios ◽  
...  
Keyword(s):  
Optimal Control ◽  
Numerical Algorithm ◽  
Control Policy ◽  
Computational Method ◽  
Observable System ◽  
Partially Observable System ◽  
Partially Observable

Filtering method for linear and non-linear stochastic optimal control of partially observable systems

Filomat ◽  
2017 ◽  
Vol 31 (19) ◽  
pp. 5979-5992 ◽  
Author(s):  
A. Delavarkhalafi ◽  
A. Poursherafatan
Keyword(s):  
Optimal Control ◽  
Stochastic Optimal Control ◽  
Heuristic Method ◽  
Control Factor ◽  
Filtering Method ◽  
Observable System ◽  
State Process ◽  
Non Linear ◽  
Partially Observable System ◽  
Partially Observable

This paper studies two linear methods for linear and non-linear stochastic optimal control of partially observable problem (SOCPP). At first, it introduces the general form of a SOCPP and states it as a functional matrix. A SOCPP has a payoff function which should be minimized. It also has two dynamic processes: state and observation. In this study, it is presented a deterministic method to find the control factor which has named feedback control and stated a modified complete proof of control optimality in a general SOCPP. After finding the optimal control factor, it should be substituted in the state process to make the partially observable system. Next, it introduces a linear filtering method to solve the related partially observable system with complete details. Finally, it is presented a heuristic method in discrete form for estimating non-linear SOCPPs and it is stated some examples to evaluate the performance of introducing methods.

Optimal Control of Partially Observable Semi-Markovian Failing Systems: An Analysis Using a Phase Methodology

2021 ◽  
Author(s):  
Akram Khaleghei ◽  
Michael Jong Kim
Keyword(s):  
Optimal Control ◽  
Control Policy ◽  
Computational Approach ◽  
Control Limit ◽  
New Approach ◽  
Problem Class ◽  
Optimal Value ◽  
Markov Decision ◽  
Decision Epoch ◽  
Partially Observable

In “Optimal Control of Partially Observable Semi-Markovian Failing Systems: An Analysis using a Phase Methodology,” Khaleghei and Kim study a maintenance control problem a as partially observable semi-Markov decision process (POSMDP), a problem class that is typically computationally intractable and not amenable to structural analysis. The authors develop a new approach based on a phase methodology where the idea is to view the intractable POSMDP as the limiting problem of a sequence of tractable POMDPs. They show that the optimal control policy can be represented as a control limit policy which monitors the estimated conditional reliability at each decision epoch, and, by exploiting this structure, an efficient computational approach to solve for the optimal control limit and corresponding optimal value is developed.

scholarly journals Optimal Control and Computational Method for the Resolution of Isoperimetric Problem in a Discrete-Time SIRS System

2018 ◽  
Vol 23 (4) ◽  
pp. 52 ◽  
Author(s):  
Fadwa Kihal ◽  
Imane Abouelkheir ◽  
Mostafa Rachik ◽  
Ilias Elmouki
Keyword(s):  
Optimal Control ◽  
Discrete Time ◽  
Control Function ◽  
Isoperimetric Problem ◽  
Control Policy ◽  
Computational Method ◽  
Susceptible Population ◽  
Sirs Epidemic Model ◽  
Sufficient And Necessary Conditions ◽  
Isoperimetric Constraint

We consider a discrete-time susceptible-infected-removed-susceptible “again” (SIRS) epidemic model, and we introduce an optimal control function to seek the best control policy for preventing the spread of an infection to the susceptible population. In addition, we define a new compartment, which models the dynamics of the number of controlled individuals and who are supposed not to be able to reach a long-term immunity due to the limited effect of control. Furthermore, we treat the resolution of this optimal control problem when there is a restriction on the number of susceptible people who have been controlled along the time of the control strategy. Further, we provide sufficient and necessary conditions for the existence of the sought optimal control, whose characterization is also given in accordance with an isoperimetric constraint. Finally, we present the numerical results obtained, using a computational method, which combines the secant method with discrete progressive-regressive schemes for the resolution of the discrete two-point boundary value problem.

scholarly journals Filtering method for linear and non-linear stochastic optimal control of partially observable systems II

Filomat ◽  
2018 ◽  
Vol 32 (14) ◽  
pp. 5089-5103 ◽  
Author(s):  
Ali Poursherafatan ◽  
Ali Delavarkhalafi
Keyword(s):  
Optimal Control ◽  
Stochastic Optimal Control ◽  
Payoff Function ◽  
Estimation Methods ◽  
Control Factor ◽  
Diffusion Term ◽  
Filtering Method ◽  
Partially Observable Systems ◽  
Partially Observable System ◽  
Partially Observable

In this paper we studied stochastic optimal control problem based on partially observable systems (SOCPP) with a control factor on the diffusion term. A SOCPP has state and observation processes. This kind of problem has also a minimum payoff function. The payoff function should be minimized according to the partially observable systems consist of the state and observation processes. In this regard, the filtering method is used to evaluat this kind of problem and express full consideration of it. Finally, presented estimation methods are used to simulate the solution of a partially observable system corresponding to the control factor of this problem. These methods are numerically used to solve linear and nonlinear cases.

Sign in / Sign up

Export Citation Format

Share Document