On Optimal Control by a Random Time Substitution in a Continuous Markov Process

1968 ◽  
Vol 13 (2) ◽  
pp. 343-345 ◽  
Author(s):  
Ya. A. Kogan
Keyword(s):  
Optimal Control ◽  
Markov Process ◽  
Random Time ◽  
Continuous Markov Process ◽  
Time Substitution

Homecomings of Markov processes

1973 ◽  
Vol 5 (01) ◽  
pp. 66-102 ◽  
Author(s):  
J. F. C. Kingman
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Continuous Time ◽  
Random Time ◽  
Random Set ◽  
Theoretical Interest ◽  
Different Types ◽  
Treat All ◽  
Image Position

Ifx0is a particular state for a continuous-time Markov processX, the random time setis often of both practical and theoretical interest. Ignoring trivial or pathological cases, there are four different types of structure which this random set can display. To some extent, it is possible to treat all four cases in a unified way, but they raise different questions and require different modes of description. The distributions of various random quantities associated withcan be related to one another by simple and useful formulae.

The geometry of random drift IV. Random time substitutions and stationary densities

1978 ◽  
Vol 10 (03) ◽  
pp. 563-569 ◽  
Author(s):  
Peter L. Antonelli
Keyword(s):  
Distance Measure ◽  
Spherical Geometry ◽  
Edge Length ◽  
Random Time ◽  
Random Drift ◽  
Multiple Allele ◽  
Original Process ◽  
Frequency Space ◽  
Gradient Type ◽  
Time Substitution

In the present paper a spatially homogeneous distance measure of Edwards and Cavalli-Sforza type is derived for multiple allele random genetic drift with a (possibly vanishing) symmetric mutation field, using the technique of random time substitution. Since the mutation field is of gradient type in gene frequency space, the transformed process is proved to be Brownian motion relative to a new Riemannian geometry and a new time measure. The new geometry is conformally related to spherical geometry of the original process but is not of constant curvature, generally. A formula relating the stationary density of the old and new process is derived and the edge length formula for the new geometry on the n-simplex frequency space is given and analysed.

scholarly journals Probability-Weighted Optimal Control for Nonlinear Stochastic Vibrating Systems with Random Time Delay

2018 ◽  
Vol 2018 ◽  
pp. 1-8
Author(s):  
R. C. Hu ◽  
Q. F. Lü ◽  
X. F. Wang ◽  
Z. G. Ying ◽  
R. H. Huan
Keyword(s):  
Optimal Control ◽  
Time Delay ◽  
Stochastic Averaging ◽  
Stochastic Averaging Method ◽  
Random Time ◽  
Nonlinear Stochastic System ◽  
Control Force ◽  
Markov Jump ◽  
Optimal Control Strategy ◽  
Vibrating Systems

A probability-weighted optimal control strategy for nonlinear stochastic vibrating systems with random time delay is proposed. First, by modeling the random delay as a finite state Markov process, the optimal control problem is converted into the one of Markov jump systems with finite mode. Then, upon limiting averaging principle, the optimal control force is approximately expressed as probability-weighted summation of the control force associated with different modes of the system. Then, by using the stochastic averaging method and the dynamical programming principle, the control force for each mode can be readily obtained. To illustrate the effectiveness of the proposed control, the stochastic optimal control of a two degree-of-freedom nonlinear stochastic system with random time delay is worked out as an example.

The geometry of random drift IV. Random time substitutions and stationary densities

1978 ◽  
Vol 10 (3) ◽  
pp. 563-569 ◽  
Author(s):  
Peter L. Antonelli
Keyword(s):  
Distance Measure ◽  
Spherical Geometry ◽  
Edge Length ◽  
Random Time ◽  
Random Drift ◽  
Multiple Allele ◽  
Original Process ◽  
Frequency Space ◽  
Gradient Type ◽  
Time Substitution

In the present paper a spatially homogeneous distance measure of Edwards and Cavalli-Sforza type is derived for multiple allele random genetic drift with a (possibly vanishing) symmetric mutation field, using the technique of random time substitution. Since the mutation field is of gradient type in gene frequency space, the transformed process is proved to be Brownian motion relative to a new Riemannian geometry and a new time measure. The new geometry is conformally related to spherical geometry of the original process but is not of constant curvature, generally. A formula relating the stationary density of the old and new process is derived and the edge length formula for the new geometry on the n-simplex frequency space is given and analysed.

A REMARK ON THE GENERATOR OF A RIGHT-CONTINUOUS MARKOV PROCESS

2007 ◽  
Vol 10 (04) ◽  
pp. 633-640 ◽  
Author(s):  
MICHAEL RÖCKNER ◽  
GERALD TRUTNAU
Keyword(s):  
Markov Process ◽  
Dirichlet Form ◽  
Borel Measure ◽  
Initial Condition ◽  
Full Support ◽  
Finite Borel Measure ◽  
Continuous Markov Process ◽  
Form Theory ◽  
Densely Defined ◽  
Generalized Dirichlet

Given a right-continuous Markov process (Xt)t ≥ 0 on a second countable metrizable space E with transition semigroup (pt)t ≥ 0, we prove that there exists a σ-finite Borel measure μ with full support on E, and a closed and densely defined linear operator [Formula: see text] generating (pt)t ≥ 0 on Lp (E; μ). In particular, we solve the corresponding Cauchy problem in Lp (E; μ) for any initial condition [Formula: see text]. Furthermore, for any real β > 0 we show that there exists a generalized Dirichlet form which is associated to (e-βt pt)t ≥ 0. If the β-subprocess of (Xt)t ≥ 0 corresponding to (e-βt pt)t ≥ 0, β > 0, is μ-special standard then all results from generalized Dirichlet form theory become available, and Fukushima's decomposition holds for [Formula: see text]. If (Xt)t ≥ 0 is transient, then β can be chosen to be zero.

Analysis of a discrete semi-Markov model of continuous inventory control with periodic interruptions of consumption

2015 ◽  
Vol 25 (1) ◽  
Author(s):  
Petr V. Shnurkov ◽  
Alexey V. Ivanov
Keyword(s):  
Optimal Control ◽  
Markov Process ◽  
Optimal Control Problem ◽  
Stochastic Model ◽  
Markov Model ◽  
Control Problem ◽  
Extremal Problem ◽  
Inventory Control ◽  
Optimal Policy ◽  
Semi Markov Process

AbstractWe consider a discrete stochastic model of inventory control based on a controlled semi-Markov process. Probabilistic characteristics of the semi-Markov process are found along with characteristics of a stationary cost functional connected with this process. It is proved that an optimal policy of inventory control is a deterministic one. Explicit analitical representation of stationary functional characterising the control quality is obtained. An optimal control problem is reduced to the solution of an extremal problem for a multivariate function.

Conditional Poisson processes

1972 ◽  
Vol 9 (2) ◽  
pp. 288-302 ◽  
Author(s):  
Richard F. Serfozo
Keyword(s):  
Markov Process ◽  
Poisson Process ◽  
Poisson Processes ◽  
Random Time ◽  
Time Transformation ◽  
Limiting Behavior ◽  
Independent Increments ◽  
Conditional Poisson

A conditional Poisson process (often called a double stochastic Poisson process) is characterized as a random time transformation of a Poisson process with unit intensity. This characterization is used to exhibit the jump times and sizes of these processes, and to study their limiting behavior. A conditional Poisson process, whose intensity is a function of a Markov process, is discussed. Results similar to those presented can be obtained for any process with conditional stationary independent increments.

Sign in / Sign up

Export Citation Format

Share Document