Value iteration in a class of controlled Markov chains with average-criterion: unbounded costs case

2002 ◽  
Author(s):  
R. Cavazos-Cadena ◽  
E. Fernandez-Gaucherand
Keyword(s):  
Markov Chains ◽  
Value Iteration ◽  
Controlled Markov Chains ◽  
Average Criterion

Value iteration in a class of average controlled Markov chains with unbounded costs: necessary and sufficient conditions for pointwise convergence

1996 ◽  
Vol 33 (4) ◽  
pp. 986-1002 ◽  
Author(s):  
Rolando Cavazos-Cadena ◽  
Emmanuel Fernández-Gaucherand
Keyword(s):  
Markov Chains ◽  
Cost Function ◽  
Sufficient Conditions ◽  
Value Functions ◽  
Value Iteration ◽  
Controlled Markov Chains ◽  
Cost Criterion ◽  
Convergence Results ◽  
Differential Costs ◽  
Necessary And Sufficient

This work concerns controlled Markov chains with denumerable state space, (possibly) unbounded cost function, and an expected average cost criterion. Under a Lyapunov function condition, together with mild continuity-compactness assumptions, a simple necessary and sufficient criterion is given so that the relative value functions and differential costs produced by the value iteration scheme converge pointwise to the solution of the optimality equation; this criterion is applied to obtain convergence results when the cost function is bounded below or bounded above.

Value iteration in a class of average controlled Markov chains with unbounded costs: necessary and sufficient conditions for pointwise convergence

1996 ◽  
Vol 33 (04) ◽  
pp. 986-1002 ◽  
Author(s):  
Rolando Cavazos-Cadena ◽  
Emmanuel Fernández-Gaucherand
Keyword(s):  
Markov Chains ◽  
Cost Function ◽  
Sufficient Conditions ◽  
Value Functions ◽  
Value Iteration ◽  
Controlled Markov Chains ◽  
Cost Criterion ◽  
Convergence Results ◽  
Differential Costs ◽  
Necessary And Sufficient

This work concerns controlled Markov chains with denumerable state space, (possibly)unboundedcost function, and an expectedaveragecost criterion. Under aLyapunov function condition, together with mild continuity-compactness assumptions, a simplenecessary and sufficientcriterion is given so that the relative value functions and differential costs produced by thevalue iterationscheme converge pointwise to the solution of the optimality equation; this criterion is applied to obtain convergence results when the cost function is bounded below or bounded above.

Sign in / Sign up

Export Citation Format

Share Document