scholarly journals Approximating High-Dimensional Dynamic Models: Sieve Value Function Iteration

2012 ◽  
Author(s):  
Peter Arcidiacono ◽  
Patrick Bayer ◽  
Federico Bugni ◽  
Jonathan James
Keyword(s):  
Value Function ◽  
Dynamic Models ◽  
High Dimensional ◽  
Value Function Iteration

scholarly journals Limits of variance-based sensitivity analysis for non-identifiability testing in high dimensional dynamic models

Automatica ◽  
2012 ◽  
Vol 48 (11) ◽  
pp. 2740-2749 ◽  
Author(s):  
Simona Dobre ◽  
Thierry Bastogne ◽  
Christophe Profeta ◽  
Muriel Barberi-Heyob ◽  
Alain Richard
Keyword(s):  
Sensitivity Analysis ◽  
Dynamic Models ◽  
High Dimensional

STATE SPACE MODELING OF YEAST GENE EXPRESSION DYNAMICS

2007 ◽  
Vol 05 (01) ◽  
pp. 31-46 ◽  
Author(s):  
OLLI HAAVISTO ◽  
HEIKKI HYÖTYNIEMI ◽  
CHRISTOPHE ROOS
Keyword(s):  
Gene Expression ◽  
State Space ◽  
Dynamic Models ◽  
System Modeling ◽  
Dynamic State ◽  
Yeast Genome ◽  
High Dimensional ◽  
Dimensional System ◽  
Yeast Saccharomyces Cerevisiae ◽  
Space Modeling

Combined interaction of all the genes forms a central part of the functional system of a cell. Thus, especially the data-based modeling of the gene expression network is currently one of the main challenges in the field of systems biology. However, the problem is an extremely high-dimensional and complex one, so that normal identification methods are usually not applicable specially if aiming at dynamic models. We propose in this paper a subspace identification approach, which is well suited for high-dimensional system modeling and the presented modified version can also handle the underdetermined case with less data samples than variables (genes). The algorithm is applied to two public stress-response data sets collected from yeast Saccharomyces cerevisiae. The obtained dynamic state space model is tested by comparing the simulation results with the measured data. It is shown that the identified model can relatively well describe the dynamics of the general stress-related changes in the expression of the complete yeast genome. However, it seems inevitable that more precise modeling of the dynamics of the whole genome would require experiments especially designed for systemic modeling.

scholarly journals Compact Representation of Value Function in Partially Observable Stochastic Games

2019 ◽  
Author(s):  
Karel Horák ◽  
Branislav Bošanský ◽  
Christopher Kiekintveld ◽  
Charles Kamhoua
Keyword(s):  
Probability Distribution ◽  
Stochastic Games ◽  
Value Function ◽  
State Of The Art ◽  
High Dimensional ◽  
Compact Representation ◽  
Lower Dimension ◽  
Partial Observability ◽  
Partially Observable ◽  
Novel Algorithm

Value methods for solving stochastic games with partial observability model the uncertainty of the players as a probability distribution over possible states, where the dimension of the belief space is the number of states. For many practical problems, there are exponentially many states which causes scalability problems. We propose an abstraction technique that addresses this curse of dimensionality by projecting the high-dimensional beliefs onto characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our main contributions are (1) a novel compact representation of the uncertainty in partially observable stochastic games and (2) a novel algorithm using this representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm using the compact representation dramatically increases scalability compared to the state of the art.

Optional Feature Vector Generation for Linear Value Function Approximation with Binary Features

2013 ◽  
Vol 756-759 ◽  
pp. 3967-3971
Author(s):  
Bo Yan Ren ◽  
Zheng Qin ◽  
Feng Fei Zhao
Keyword(s):  
Function Approximation ◽  
Value Function ◽  
Feature Vector ◽  
High Dimensional ◽  
Generation Process ◽  
Value Function Approximation ◽  
Vector Generation ◽  
Binary Features ◽  
Performance Of Algorithm ◽  
The Value Function

Linear value function approximation with binary features is important in the research of Reinforcement Learning (RL). When updating the value function, it is necessary to generate a feature vector which contains the features that should be updated. In high dimensional domains, the generation process will take lot more time, which reduces the performance of algorithm a lot. Hence, this paper introduces Optional Feature Vector Generation (OFVG) algorithm as an improved method to generate feature vectors that can be combined with any online, value-based RL method that uses and expands binary features. This paper shows empirically that OFVG performs well in high dimensional domains.

scholarly journals COMPUTING MARKOV-PERFECT OPTIMAL POLICIES IN BUSINESS-CYCLE MODELS

2015 ◽  
Vol 20 (7) ◽  
pp. 1850-1872 ◽  
Author(s):  
Richard Dennis ◽  
Tatiana Kirsanova
Keyword(s):  
Business Cycle ◽  
Government Spending ◽  
Value Function ◽  
Income Taxation ◽  
Shadow Prices ◽  
Value Function Iteration ◽  
Risk Sensitive ◽  
Markov Perfect ◽  
Optimal Policies ◽  
Fiscal Authority

Time inconsistency is an essential feature of many policy problems. This paper presents and compares three methods for computing Markov-perfect optimal policies in stochastic nonlinear business cycle models. The methods considered include value function iteration, generalized Euler equations, and parameterized shadow prices. In the context of a business cycle model in which a fiscal authority chooses government spending and income taxation optimally, although lacking the ability to commit, we show that the solutions obtained using value function iteration and generalized Euler equations are somewhat more accurate than that obtained using parameterized shadow prices. Among these three methods, we show that value function iteration can be applied easily, even to environments that include a risk-sensitive fiscal authority and/or inequality constraints on government spending. We show that the risk-sensitive fiscal authority lowers government spending and income taxation, reducing the disincentive to accumulate wealth that households face.

Sign in / Sign up

Export Citation Format

Share Document