scholarly journals Symbolic Dynamic Programming for Continuous State MDPs with Linear Program Transitions

2021 ◽  
Author(s):  
Jihwan Jeong ◽  
Parth Jaggi ◽  
Scott Sanner
Keyword(s):  
Dynamic Programming ◽  
Closed Form ◽  
Piecewise Linear ◽  
Linear Program ◽  
Symbolic Dynamic ◽  
State Transitions ◽  
Traffic Signal Control ◽  
Value Functions ◽  
Continuous State ◽  
Solution Methods

Recent advances in symbolic dynamic programming (SDP) have significantly broadened the class of MDPs for which exact closed-form value functions can be derived. However, no existing solution methods can solve complex discrete and continuous state MDPs where a linear program determines state transitions --- transitions that are often required in problems with underlying constrained flow dynamics arising in problems ranging from traffic signal control to telecommunications bandwidth planning. In this paper, we present a novel SDP solution method for MDPs with LP transitions and continuous piecewise linear dynamics by introducing a novel, fully symbolic argmax operator. On three diverse domains, we show the first automated exact closed-form SDP solution to these challenging problems and the significant advantages of our SDP approach over discretized approximations.

An investment model with entry and exit decisions

2000 ◽  
Vol 37 (2) ◽  
pp. 547-559 ◽  
Author(s):  
J. Kate Duckworth ◽  
Mihail Zervos
Keyword(s):  
Dynamic Programming ◽  
Closed Form ◽  
Production Scheduling ◽  
Analytic Solution ◽  
Investment Model ◽  
Programming Approach ◽  
Entry And Exit ◽  
Dynamic Programming Approach ◽  
Exit Decisions ◽  
Parameter Values

We consider an investment model which generalizes a number of models that have been studied in the literature. The model involves entry and exit decisions as well as decisions relating to production scheduling. We then address the problem of its valuation from the standpoint of the dynamic programming approach. Our analysis results in a closed form analytic solution that can take qualitatively different forms depending on parameter values.

New Insights Into Slip-Stick Vibrations From Compact, Closed-Form Analysis

2017 ◽  
Author(s):  
Deborah Fowler ◽  
David Peters
Keyword(s):  
Closed Form ◽  
Coulomb Friction ◽  
Sliding Friction ◽  
Piecewise Linear ◽  
Limited Range ◽  
System Behavior ◽  
Form Analysis ◽  
Mass Damper ◽  
The Times ◽  
Sticking Point

A mechanical system sliding on a moving surface with Coulomb friction is a rich area for study. Despite much past work, there is still something to be gleaned by closed-form expressions for the system behavior. Consider a spring-mass-damper system (K, M, C) with deflection x, base moving in the +x direction at velocity V, sliding friction F, and sticking friction Fs. An initial condition of x0 at rest can be considered general because all possible motions will follow. Two dimensionless schemes are used. For the abstract, we focus on the scheme normalized by x0 with variable z = x/x0, τ = (ωnt, ωn = [K/M]1/2, ζ = c/[2(KM)1/2], ν̄ = V / (ωnx0), f = F/(Kx0), and fs = Fs/(Kx0). Since the solution is piecewise linear, this allows closed-form results. For this abstract, we consider C = 0, Fs = F. (Other cases are in the paper.) There are three critical ground speeds. The first, ν̄d, is when sticking first occurs (at z = f). At the second speed, ν̄c, sticking has moved to z = −f. Thereafter, the sticking point again increases, reaching z = f at the third speed, ν̄b. For higher ν̄, there is no sticking. In this paper, closed form expressions are presented for the three critical speeds:(1)ν¯d=[(1+3f)(1−5f)]12,ν¯c=[(1+f)(1−3f)]12,ν¯b=1−f These formulas are verified by numerical simulation. The insight is that there is a limited range of f for which certain critical points can be reached. Thus, 0 < f < 1/5 has different dynamics than 1/5 < f < 1/3. Formulas are also derived for the second maximum of z, which gives an indication of decay or growth of the system. For example, with f = fs and C = 0, the second maximum z with f < 1/5 is:(2)zmax=f+((1−f)2−ν¯2−4f)2+ν¯2ν¯d<ν¯<ν¯czmax=ν¯+fν¯c<ν¯<ν¯bzmax=1ν¯>ν¯c Formulas will also be given for the times at which the maximum occurs and the times at which a transition occurs from static to sliding for all cases.

DYNAMIC POWER DISSIPATION FORMULATION FOR APPLICATION IN DYNAMIC PROGRAMMING BUFFER INSERTION ALGORITHM

2016 ◽  
Vol 78 (11) ◽  
Author(s):  
Chessda Uttraphan ◽  
Nasir Shaikh-Husin ◽  
M. Khalil-Hani
Keyword(s):  
Dynamic Programming ◽  
Closed Form ◽  
Power Dissipation ◽  
Closed Form Solution ◽  
Propagation Delay ◽  
Form Solution ◽  
Buffer Insertion ◽  
Dynamic Power ◽  
Insertion Algorithm ◽  
Series Of Experiments

Buffer insertion is a very effective technique to reduce propagation delay in nano-metre VLSI interconnects. There are two techniques for buffer insertion which are: (1) closed-form solution and (2) dynamic programming. Buffer insertion algorithm using dynamic programming is more useful than the closed-form solution as it allows the use of multiple buffer types and it can be used in tree structured interconnects. As design dimension shrinks, more buffers are needed to improve timing performance. However, the buffer itself consumes power and it has been shown that power dissipation of buffers is significant. Although there are many buffer insertion algorithms that were able to optimize propagation delay with power constraint, most of them used the closed-form solution. Hence, in this paper, we present a formulation to compute dynamic power dissipation of buffers for application in dynamic programming buffer insertion algorithm. The proposed formulation allows dynamic power dissipation of buffers to be computed incrementally. The technique is validated by comparing the formulation with the standard closed-form dynamic power equation. The advantage of the proposed formulation is demonstrated through a series of experiments where it is applied in van Ginneken’s algorithm. The results show that the output of the proposed formulation is consistent with the standard closed-form formulation. Furthermore, it also suggests that the proposed formulation is able to compute dynamic power dissipation for buffer insertion algorithm with multiple buffer types.  

scholarly journals A Robustness Check on Debt and the Pecking Order Hypothesis with Asymmetric Information

2016 ◽  
Vol 8 (6) ◽  
pp. 181
Author(s):  
George Chang
Keyword(s):  
Asymmetric Information ◽  
Closed Form ◽  
Relative Efficiency ◽  
Closed Form Solution ◽  
Form Solution ◽  
Pecking Order ◽  
Continuous State ◽  
Robustness Check ◽  
Pecking Order Hypothesis ◽  
Equity Offering

Daniel and Titman (1995) examined the incentives of firms to signal their values prior to making a new equity offering. By analyzing this issue within a simple framework that encompasses a number of models in the literature, they were able to judge the relative efficiency of various signals that have been proposed. Although their analyses offer valuable insights, the robustness of their model has yet to be checked. This paper examines the parametric assumptions of their model in the section on debt and the pecking order hypothesis. We first generalize the assumptions in the example by Daniel and Titman (1995) to allow for continuous-state of nature. We then derive the resulting closed-form solution for the equilibrium payoffs to the original shareholders of both types firms under different beliefs. Although we only examine the robustness of a particular setting, our methodology can be applied to other settings.

scholarly journals Approximate Dynamic Programming via a Smoothed Linear Program

2012 ◽  
Vol 60 (3) ◽  
pp. 655-674 ◽  
Author(s):  
Vijay V. Desai ◽  
Vivek F. Farias ◽  
Ciamac C. Moallemi
Keyword(s):  
Dynamic Programming ◽  
Approximate Dynamic Programming ◽  
Linear Program
Sign in / Sign up

Export Citation Format

Share Document