Discrete-time equivalence for constrained semi-Markov decision processes

1985 ◽  
Author(s):  
Frederick Beutler ◽  
Keith Ross
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Decision Processes ◽  
Markov Decision

scholarly journals Average optimality for Markov decision processes in borel spaces: a new condition and approach

2006 ◽  
Vol 43 (02) ◽  
pp. 318-334
Author(s):  
Xianping Guo ◽  
Quanxin Zhu
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Existence Of Solutions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Stationary Policy ◽  
Markov Decision ◽  
Optimal Stationary Policy ◽  
Optimality Inequality ◽  
Action Spaces

In this paper we study discrete-time Markov decision processes with Borel state and action spaces. The criterion is to minimize average expected costs, and the costs may have neither upper nor lower bounds. We first provide two average optimality inequalities of opposing directions and give conditions for the existence of solutions to them. Then, using the two inequalities, we ensure the existence of an average optimal (deterministic) stationary policy under additional continuity-compactness assumptions. Our conditions are slightly weaker than those in the previous literature. Also, some new sufficient conditions for the existence of an average optimal stationary policy are imposed on the primitive data of the model. Moreover, our approach is slightly different from the well-known ‘optimality inequality approach’ widely used in Markov decision processes. Finally, we illustrate our results in two examples.

scholarly journals Average optimality for Markov decision processes in borel spaces: a new condition and approach

2006 ◽  
Vol 43 (2) ◽  
pp. 318-334 ◽  
Author(s):  
Xianping Guo ◽  
Quanxin Zhu
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Existence Of Solutions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Stationary Policy ◽  
Markov Decision ◽  
Optimal Stationary Policy ◽  
Optimality Inequality ◽  
Action Spaces

In this paper we study discrete-time Markov decision processes with Borel state and action spaces. The criterion is to minimize average expected costs, and the costs may have neither upper nor lower bounds. We first provide two average optimality inequalities of opposing directions and give conditions for the existence of solutions to them. Then, using the two inequalities, we ensure the existence of an average optimal (deterministic) stationary policy under additional continuity-compactness assumptions. Our conditions are slightly weaker than those in the previous literature. Also, some new sufficient conditions for the existence of an average optimal stationary policy are imposed on the primitive data of the model. Moreover, our approach is slightly different from the well-known ‘optimality inequality approach’ widely used in Markov decision processes. Finally, we illustrate our results in two examples.

Sign in / Sign up

Export Citation Format

Share Document