scholarly journals Continuous-Time Markov Decision Processes with Exponential Utility

2017 ◽  
Vol 55 (4) ◽  
pp. 2636-2660 ◽  
Author(s):  
Yi Zhang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Exponential Utility ◽  
Markov Decision

On gradual-impulse control of continuous-time Markov decision processes with exponential utility

2021 ◽  
Vol 53 (2) ◽  
pp. 301-334
Author(s):  
Xin Guo ◽  
Aiko Kurushima ◽  
Alexey Piunovskiy ◽  
Yi Zhang
Keyword(s):  
Control Problem ◽  
Markov Decision Processes ◽  
Continuous Time ◽  
Impulse Control ◽  
Decision Processes ◽  
Exponential Utility ◽  
Impulse Control Problem ◽  
Markov Decision ◽  
The Value Function ◽  
Selection Of

AbstractWe consider a gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We show, under natural conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies that allow multiple simultaneous impulses, randomized selection of impulses with random effects, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions.

scholarly journals Denumerable state continuous time Markov decision processes with unbounded cost and transition rates under average criterion

2002 ◽  
Vol 43 (4) ◽  
pp. 541-557 ◽  
Author(s):  
Xianping Guo ◽  
Weiping Zhu
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Transition Rates ◽  
Birth And Death Processes ◽  
Optimality Equation ◽  
Average Criterion ◽  
Markov Decision ◽  
Unbounded Cost ◽  
Queue Model

AbstractIn this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission control queue model and controlled birth and death processes.

Sign in / Sign up

Export Citation Format

Share Document