scholarly journals Approximate Dynamic Programming with (min; +) linear function approximation for Markov decision processes

2014 ◽  
Author(s):  
L. Chandrashekar ◽  
Shalabh Bhatnagar
Keyword(s):  
Dynamic Programming ◽  
Linear Function ◽  
Markov Decision Processes ◽  
Function Approximation ◽  
Approximate Dynamic Programming ◽  
Decision Processes ◽  
Linear Function Approximation ◽  
Markov Decision

scholarly journals Kalman Temporal Differences

2010 ◽  
Vol 39 ◽  
pp. 483-532 ◽  
Author(s):  
M. Geist ◽  
O. Pietquin
Keyword(s):  
Markov Decision Processes ◽  
Function Approximation ◽  
Approximation Scheme ◽  
State Of The Art ◽  
Decision Processes ◽  
Temporal Differences ◽  
Special Cases ◽  
Markov Decision ◽  
Biased Estimates ◽  
Q Function

Because reinforcement learning suffers from a lack of scalability, online value (and Q-) function approximation has received increasing interest this last decade. This contribution introduces a novel approximation scheme, namely the Kalman Temporal Differences (KTD) framework, that exhibits the following features: sample-efficiency, non-linear approximation, non-stationarity handling and uncertainty management. A first KTD-based algorithm is provided for deterministic Markov Decision Processes (MDP) which produces biased estimates in the case of stochastic transitions. Than the eXtended KTD framework (XKTD), solving stochastic MDP, is described. Convergence is analyzed for special cases for both deterministic and stochastic transitions. Related algorithms are experimented on classical benchmarks. They compare favorably to the state of the art while exhibiting the announced features.

Cost rate heuristics for semi-Markov decision processes

1992 ◽  
Vol 29 (03) ◽  
pp. 633-644
Author(s):  
K. D. Glazebrook ◽  
Michael P. Bailey ◽  
Lyn R. Whitaker
Keyword(s):  
Dynamic Programming ◽  
Markov Decision Processes ◽  
Preventive Maintenance ◽  
Decision Processes ◽  
Cost Rate ◽  
Backwards Induction ◽  
Optimal Policies ◽  
Markov Decision ◽  
Speed Of Evolution

In response to the computational complexity of the dynamic programming/backwards induction approach to the development of optimal policies for semi-Markov decision processes, we propose a class of heuristics resulting from an inductive process which proceeds forwards in time. These heuristics always choose actions in such a way as to minimize some measure of the current cost rate. We describe a procedure for calculating such cost rate heuristics. The quality of the performance of such policies is related to the speed of evolution (in a cost sense) of the process. A simple model of preventive maintenance is described in detail. Cost rate heuristics for this problem are calculated and assessed computationally.

Sign in / Sign up

Export Citation Format

Share Document