scholarly journals A Relative Value Iteration Algorithm for Nondegenerate Controlled Diffusions

2012 ◽  
Vol 50 (4) ◽  
pp. 1886-1902 ◽  
Author(s):  
Ari Arapostathis ◽  
Vivek S. Borkar
Keyword(s):  
Iteration Algorithm ◽  
Value Iteration ◽  
Relative Value ◽  
Controlled Diffusions ◽  
Value Iteration Algorithm

Adiabatic Markov Decision Process: Convergence of Value Iteration Algorithm

2016 ◽  
Vol 138 (6) ◽  
Author(s):  
Thai Duong ◽  
Duong Nguyen-Huu ◽  
Thinh Nguyen
Keyword(s):  
Markov Decision Process ◽  
Decision Process ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Rate Of Change ◽  
Optimal Decision ◽  
Iteration Algorithm ◽  
Value Iteration ◽  
Markov Decision ◽  
Value Iteration Algorithm

Markov decision process (MDP) is a well-known framework for devising the optimal decision-making strategies under uncertainty. Typically, the decision maker assumes a stationary environment which is characterized by a time-invariant transition probability matrix. However, in many real-world scenarios, this assumption is not justified, thus the optimal strategy might not provide the expected performance. In this paper, we study the performance of the classic value iteration algorithm for solving an MDP problem under nonstationary environments. Specifically, the nonstationary environment is modeled as a sequence of time-variant transition probability matrices governed by an adiabatic evolution inspired from quantum mechanics. We characterize the performance of the value iteration algorithm subject to the rate of change of the underlying environment. The performance is measured in terms of the convergence rate to the optimal average reward. We show two examples of queuing systems that make use of our analysis framework.

A Modified Value Iteration Algorithm for Discounted Markov Decision Processes

2015 ◽  
Vol 13 (3) ◽  
pp. 47-57 ◽  
Author(s):  
Sanaa Chafik ◽  
Cherki Daoui
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Iteration Algorithm ◽  
Value Iteration ◽  
Decomposition Technique ◽  
Artificial Data ◽  
Markov Decision ◽  
Speed Up ◽  
Value Iteration Algorithm

As many real applications need a large amount of states, the classical methods are intractable for solving large Markov Decision Processes. The decomposition technique basing on the topology of each state in the associated graph and the parallelization technique are very useful methods to cope with this problem. In this paper, the authors propose a Modified Value Iteration algorithm, adding the parallelism technique. They test their implementation on artificial data using an Open MP that offers a significant speed-up.

scholarly journals Value iteration algorithm for mean-field games

2020 ◽  
Vol 143 ◽  
pp. 104744
Author(s):  
Berkay Anahtarcı ◽  
Can Deha Karıksız ◽  
Naci Saldi
Keyword(s):  
Mean Field ◽  
Mean Field Games ◽  
Iteration Algorithm ◽  
Value Iteration ◽  
Value Iteration Algorithm

Point-based online value iteration algorithm in large POMDP

2013 ◽  
Vol 40 (3) ◽  
pp. 546-555 ◽  
Author(s):  
Bo Wu ◽  
Hong-Yan Zheng ◽  
Yan-Peng Feng
Keyword(s):  
Iteration Algorithm ◽  
Value Iteration ◽  
Value Iteration Algorithm
Sign in / Sign up

Export Citation Format

Share Document