scholarly journals Geometric fluid approximation for general continuous-time Markov chains

2019 ◽  
Vol 475 (2229) ◽  
pp. 20190100 ◽  
Author(s):  
Michalis Michaelides ◽  
Jane Hillston ◽  
Guido Sanguinetti
Keyword(s):  
Population Structure ◽  
Continuous Time ◽  
Transition Matrix ◽  
Continuous Process ◽  
Gaussian Process Regression ◽  
Great Success ◽  
Fluid Approximation ◽  
Continuous Space ◽  
Continuous State ◽  
Macro Scale

Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a dynamics for the approximating process. We construct here a general method based on spectral analysis of the transition matrix of the CTMC, without the need for a population structure. Specifically, we use the popular manifold learning method of diffusion maps to analyse the transition matrix as the operator of a hidden continuous process. An embedding of states in a continuous space is recovered, and the space is endowed with a drift vector field inferred via Gaussian process regression. In this manner, we construct an ordinary differential equation whose solution approximates the evolution of the CTMC mean, mapped onto the continuous space (known as the fluid limit).

Markov processes associated with critical Galton-Watson processes with application to extinction probabilities

1976 ◽  
Vol 8 (2) ◽  
pp. 278-295 ◽  
Author(s):  
Michael Sze
Keyword(s):  
Continuous Time ◽  
Branching Process ◽  
Continuous Process ◽  
Regularly Varying Functions ◽  
Original Process ◽  
Additional Information ◽  
Embedding Technique ◽  
Extinction Probabilities ◽  
Watson Process ◽  
The Given

As an alternative to the embedding technique of T. E. Harris, S. Karlin and J. McGregor, we show that given a critical Galton–Watson process satisfying some mild assumptions, we can always construct a continuous-time Markov branching process having the same asymptotic behaviour as the given process. Thus, via the associated continuous process, additional information about the original process is obtained. We apply this technique to the study of extinction probabilities of a critical Galton–Watson process, and provide estimates for the extinction probabilities by regularly varying functions.

Particle Swarm Optimization Combined with Ant Colony Optimization for the Multiple Traveling Salesman Problem

2009 ◽  
Vol 626-627 ◽  
pp. 717-722 ◽  
Author(s):  
Hong Kui Feng ◽  
Jin Song Bao ◽  
Jin Ye
Keyword(s):  
Particle Swarm Optimization ◽  
Ant Colony Optimization ◽  
Traveling Salesman Problem ◽  
Particle Swarm ◽  
Traveling Salesman ◽  
Ant Colony ◽  
Great Success ◽  
Swarm Optimization ◽  
Continuous Space ◽  
Multiple Traveling Salesman Problem

A lot of practical problem, such as the scheduling of jobs on multiple parallel production lines and the scheduling of multiple vehicles transporting goods in logistics, can be modeled as the multiple traveling salesman problem (MTSP). Due to the combinatorial complexity of the MTSP, it is necessary to use heuristics to solve the problem, and a discrete particle swarm optimization (DPSO) algorithm is employed in this paper. Particle swarm optimization (PSO) in the continuous space has obtained great success in resolving some minimization problems. But when applying PSO for the MTSP, a difficulty rises, which is to find a suitable mapping between sequence and continuous position of particles in particle swarm optimization. For overcoming this difficulty, PSO is combined with ant colony optimization (ACO), and the mapping between sequence and continuous position of particles is established. To verify the efficiency of the DPSO algorithm, it is used to solve the MTSP and its performance is compared with the ACO and some traditional DPSO algorithms. The computational results show that the proposed DPSO algorithm is efficient.

NEURONAL DYNAMICS UNDER PERIODIC STIMULI

2002 ◽  
Vol 12 (02) ◽  
pp. 137-148
Author(s):  
K. GOPALSAMY ◽  
S. MOHAMAD
Keyword(s):  
Asymptotic Stability ◽  
Continuous Time ◽  
Temporal Pattern ◽  
Sufficient Conditions ◽  
Periodic Patterns ◽  
Neuronal Dynamics ◽  
Continuous State ◽  
State Models ◽  
Associative Recall ◽  
External Stimuli

The convergence characteristics of a single dissipative Hopfield-type neuron with self-interaction under periodic external stimuli are considered. Sufficient conditions are established for associative encoding and recall of the periodic patterns associated with the external stimuli. Both continuous-time-continuous-state and discrete-time-continuous-state models are discussed. It is shown that when the neuronal gain is dominated by the neuronal dissipation on average, associative recall of the encoded temporal pattern is guaranteed and this is achieved by the global asymptotic stability of the encoded pattern.

Markovian Analysis of a Push-Pull Merge System with Two Suppliers, An Intermediate Buffer, and Two Retailers

2017 ◽  
Vol 8 (2) ◽  
pp. 1-35 ◽  
Author(s):  
Alexandros Diamantidis ◽  
Stylianos Koukoumialos ◽  
Michael Vidalis
Keyword(s):  
Continuous Time ◽  
Transition Matrix ◽  
Control Policy ◽  
Order Quantity ◽  
Arrival Times ◽  
Continuous Review ◽  
Distribution Centre ◽  
Inventory Control Policy ◽  
Service Rates ◽  
Stationary Probabilities

This paper examines a push-pull merge system with two suppliers, two retailers and an intermediate buffer (distribution centre). Two reliable non identical suppliers performing merge operations feed a buffer that is located immediately upstream of two non-identical reliable retailers. External customers arrive to each retailer with non-identical inter-arrival times that are exponentially distributed. The amount ordered from each retailer by a customer is exactly one unit. The material flows between upstream stages (suppliers) is push type, while between downstream stages (retailers) it is driven by continuous review, reorder point/order quantity inventory control policy (s,S). Both suppliers and retailers have exponential service rates. The considered system is modelled as a continuous time Markov process with discrete states. An algorithm that generates the transition matrix for any value of the parameters of the system is developed. Once the transition matrix is known the stationary probabilities can be computed and therefore the performance measures of the model under consideration can be easily evaluated.

scholarly journals Deep Q-Network with Predictive State Models in Partially Observable Domains

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Danning Yu ◽  
Kun Ni ◽  
Yunlong Liu
Keyword(s):  
Recurrent Network ◽  
Fixed Number ◽  
The State ◽  
Great Success ◽  
Robotic Control ◽  
Continuous State Space ◽  
Continuous State ◽  
Observable Actions ◽  
State Models ◽  
Partially Observable

While deep reinforcement learning (DRL) has achieved great success in some large domains, most of the related algorithms assume that the state of the underlying system is fully observable. However, many real-world problems are actually partially observable. For systems with continuous observation, most of the related algorithms, e.g., the deep Q-network (DQN) and deep recurrent Q-network (DRQN), use history observations to represent states; however, they often make computation-expensive and ignore the information of actions. Predictive state representations (PSRs) can offer a powerful framework for modelling partially observable dynamical systems with discrete or continuous state space, which represents the latent state using completely observable actions and observations. In this paper, we present a PSR model-based DQN approach which combines the strengths of the PSR model and DQN planning. We use a recurrent network to establish the recurrent PSR model, which can fully learn dynamics of the partially continuous observable environment. Then, the model is used for the state representation and update of DQN, which makes DQN no longer rely on a fixed number of history observations or recurrent neural network (RNN) to represent states in the case of partially observable environments. The strong performance of the proposed approach is demonstrated on a set of robotic control tasks from OpenAI Gym by comparing with the technique with the memory-based DRQN and the state-of-the-art recurrent predictive state policy (RPSP) networks. Source code is available at https://github.com/RPSR-DQN/paper-code.git.

scholarly journals A Limit Theorem for Discrete Galton-Watson Branching Processes with Immigration

2006 ◽  
Vol 43 (01) ◽  
pp. 289-295 ◽  
Author(s):  
Zenghu Li
Keyword(s):  
Limit Theorem ◽  
Weak Convergence ◽  
Discrete Time ◽  
Continuous Time ◽  
Branching Processes ◽  
Sufficient Conditions ◽  
Discrete State ◽  
Simple Set ◽  
Continuous State

We provide a simple set of sufficient conditions for the weak convergence of discrete-time, discrete-state Galton-Watson branching processes with immigration to continuous-time, continuous-state branching processes with immigration.

scholarly journals Unexpected microphase transitions in flow towards nematic order of cellulose nanocrystals

Cellulose ◽  
2019 ◽  
Vol 27 (4) ◽  
pp. 2003-2014 ◽  
Author(s):  
Roland Kádár ◽  
Mina Fazilati ◽  
Tiina Nypelö
Keyword(s):  
Liquid Crystal ◽  
Shear Rate ◽  
Formation Process ◽  
Recovery Time ◽  
Continuous Process ◽  
Cellulose Nanocrystal ◽  
Nematic Order ◽  
Chiral Nematic ◽  
Macro Scale

Abstract Organization of nanoparticles is essential in order to control their light-matter interactions. We present cellulose nanocrystal suspension organization in flow towards a unidirectional state. Visualization of evolving polarization patterns of the cellulose nanocrystal suspensions is combined with steady and oscillatory shear rheology. Elucidation of the chiral nematic mesophase in a continuous process towards unidirectional order enables control of alignment in a suspension precursor for structural films and reveals thus far in situ unrevealed transition states that were not detectable by rheology alone. The coupled analytics enabled the suspensions of interest to be divided into rheological gels and rheological liquid crystal fluids with detailed information on the microtransition phases. Both populations experienced submicron organization and reached macro-scale homogeneity with unidirectional ordering in continued shear. We quantify the time, shear rate, and recovery time after shear to design an optimizing formation process for controlled wet structures as precursors for dry products. Graphic abstract

Asymptotic properties of super-critical branching processes II: Crump-Mode and Jirina processes

1975 ◽  
Vol 7 (01) ◽  
pp. 66-82 ◽  
Author(s):  
N. H. Bingham ◽  
R. A. Doney
Keyword(s):  
State Space ◽  
Continuous Time ◽  
Regular Variation ◽  
Branching Processes ◽  
Asymptotic Properties ◽  
Random Variables ◽  
The Other ◽  
Continuous State Space ◽  
Continuous State ◽  
Integer Exponent

We obtain results connecting the distribution of the random variablesYandWin the supercritical generalized branching processes introduced by Crump and Mode. For example, if β > 1,EYβandEWβconverge or diverge together and regular variation of the tail of one ofY, Wwith non-integer exponent β > 1 is equivalent to regular variation of the other. We also prove analogous results for the continuous-time continuous state-space branching processes introduced by Jirina.

Asymptotic properties of super-critical branching processes II: Crump-Mode and Jirina processes

1975 ◽  
Vol 7 (1) ◽  
pp. 66-82 ◽  
Author(s):  
N. H. Bingham ◽  
R. A. Doney
Keyword(s):  
State Space ◽  
Continuous Time ◽  
Regular Variation ◽  
Branching Processes ◽  
Asymptotic Properties ◽  
Random Variables ◽  
The Other ◽  
Continuous State Space ◽  
Continuous State ◽  
Integer Exponent

We obtain results connecting the distribution of the random variables Y and W in the supercritical generalized branching processes introduced by Crump and Mode. For example, if β > 1, EYβ and EWβ converge or diverge together and regular variation of the tail of one of Y, W with non-integer exponent β > 1 is equivalent to regular variation of the other. We also prove analogous results for the continuous-time continuous state-space branching processes introduced by Jirina.

Sign in / Sign up

Export Citation Format

Share Document