A Hardware Approach to Value Function Iteration

2019 ◽  
Author(s):  
Alessandro Peri
Keyword(s):  
Value Function ◽  
Value Function Iteration

scholarly journals COMPUTING MARKOV-PERFECT OPTIMAL POLICIES IN BUSINESS-CYCLE MODELS

2015 ◽  
Vol 20 (7) ◽  
pp. 1850-1872 ◽  
Author(s):  
Richard Dennis ◽  
Tatiana Kirsanova
Keyword(s):  
Business Cycle ◽  
Government Spending ◽  
Value Function ◽  
Income Taxation ◽  
Shadow Prices ◽  
Value Function Iteration ◽  
Risk Sensitive ◽  
Markov Perfect ◽  
Optimal Policies ◽  
Fiscal Authority

Time inconsistency is an essential feature of many policy problems. This paper presents and compares three methods for computing Markov-perfect optimal policies in stochastic nonlinear business cycle models. The methods considered include value function iteration, generalized Euler equations, and parameterized shadow prices. In the context of a business cycle model in which a fiscal authority chooses government spending and income taxation optimally, although lacking the ability to commit, we show that the solutions obtained using value function iteration and generalized Euler equations are somewhat more accurate than that obtained using parameterized shadow prices. Among these three methods, we show that value function iteration can be applied easily, even to environments that include a risk-sensitive fiscal authority and/or inequality constraints on government spending. We show that the risk-sensitive fiscal authority lowers government spending and income taxation, reducing the disincentive to accumulate wealth that households face.

scholarly journals Value Function Iteration as a Solution Method for the Ramsey Model

2011 ◽  
Vol 231 (4) ◽  
Author(s):  
Burkhard Heer ◽  
Alfred Maußner
Keyword(s):  
Solution Method ◽  
Value Function ◽  
Spline Interpolation ◽  
Agent Model ◽  
Value Function Iteration ◽  
Speed Up ◽  
Grid Points ◽  
High Level ◽  
The Value Function ◽  
Speed And Accuracy

SummaryValue function iteration is one of the standard tools for the solution of dynamic general equilibrium models if the dimension of the state space is one ore two. We consider three kinds of models: the deterministic and the stochastic growth model and a simple heterogenous agent model. Each model is solved with six different algorithms: (1) simple value function iteration as compared to (2) smart value function iteration neglects the special structure of the problem. (3) Full and (4) modified policy iteration are methods to speed up convergence. (5) linear and (6) cubic interpolation between the grid points are methods that enhance precision and reduce the size of the grid. We evaluate the algorithms with respect to speed and accuracy. Accuracy is defined as the maximum absolute value of the residual of the Euler equation that determines the household’s savings. We demonstrate that the run time of all algorithms can be reduced substantially if the value function is initialized stepwise, starting on a coarse grid and increasing the number of grid points successively until the desired size is reached.We find that value function iteration with cubic spline interpolation between grid points dominates the other methods if a high level of accuracy is needed.

Technical Note—Cyclic Variables and Markov Decision Processes

2020 ◽  
Vol 68 (4) ◽  
pp. 1231-1237
Author(s):  
Avery Haviv
Keyword(s):  
Markov Decision Processes ◽  
Value Function ◽  
Computational Cost ◽  
Technical Note ◽  
Decision Processes ◽  
Computational Burden ◽  
Value Function Iteration ◽  
Markov Decision ◽  
Specific Order ◽  
Standard Value

Markov decision processes are commonly used to model forward-looking behavior. However, cyclic terms, including seasonality, are often omitted from these models because of the increase in computational burden. This paper develops a cyclic value function iteration (CVFI), an adjustment to the standard value function iteration. By updating states in a specific order, CVFI allows cyclic variables to be included in the state space with no increase in the computational cost. This result is proved theoretically and shown to hold closely in Monte Carlo simulations.

Sign in / Sign up

Export Citation Format

Share Document