scholarly journals A Semi-Deterministic Random Walk with Resetting

Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 825
Author(s):  
Javier Villarroel ◽  
Miquel Montero ◽  
Juan Antonio Vega
Keyword(s):  
Random Walk ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Escape Time ◽  
First Passage ◽  
Double Barrier ◽  
Starting Position ◽  
Random Times ◽  
Deterministic Random Walk ◽  
Passage Times

We consider a discrete-time random walk (xt) which, at random times, is reset to the starting position and performs a deterministic motion between them. We show that the quantity Prxt+1=n+1|xt=n,n→∞ determines if the system is averse, neutral or inclined towards resetting. It also classifies the stationary distribution. Double barrier probabilities, first passage times and the distribution of the escape time from intervals are determined.

Geometric Markov chains

1995 ◽  
Vol 32 (02) ◽  
pp. 349-374
Author(s):  
William Rising
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Nearest Neighbor ◽  
First Passage Times ◽  
First Passage ◽  
Death Rates ◽  
Passage Times ◽  
Birth Death

A generalization of the familiar birth–death chain, called the geometric chain, is introduced and explored. By the introduction of two families of parameters in addition to the infinitesimal birth and death rates, the geometric chain allows transitions beyond the nearest neighbor, but is shown to retain the simple computational formulas of the birth–death chain for the stationary distribution and the expected first-passage times between states. It is also demonstrated that even when not reversible, a reversed geometric chain is again a geometric chain.

scholarly journals SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

2007 ◽  
Vol 24 (06) ◽  
pp. 813-829 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Linear Equations ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
The Mean ◽  
Computational Procedures ◽  
Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

Some asymptotic results for transient random walks

1996 ◽  
Vol 28 (1) ◽  
pp. 207-226 ◽  
Author(s):  
J. Bertoin ◽  
R. A. Doney
Keyword(s):  
Random Walk ◽  
Random Walks ◽  
Limit Theorems ◽  
First Passage Times ◽  
Asymptotic Results ◽  
First Passage ◽  
Cramér’S Condition ◽  
Tail Behaviour ◽  
Passage Times ◽  
Asymptotic Tail

We consider a real-valued random walk S which drifts to –∞ and is such that E(exp θS1) < ∞ for some θ > 0, but for which Cramér's condition fails. We investigate the asymptotic tail behaviour of the distributions of the all time maximum, the upwards and downwards first passage times and the last passage times. As an application, we obtain new limit theorems for certain conditional laws.

Some asymptotic results for transient random walks with applications to insurance risk

2001 ◽  
Vol 38 (01) ◽  
pp. 108-121 ◽  
Author(s):  
Aleksandras Baltrūnas
Keyword(s):  
Random Walk ◽  
Ruin Probability ◽  
Risk Theory ◽  
Insurance Risk ◽  
Asymptotic Results ◽  
First Passage ◽  
Tail Behaviour ◽  
Heavy Tailed ◽  
Exact Rate Of Convergence ◽  
Passage Times

We consider a real-valued random walk which drifts to -∞ and is such that the step distribution is heavy tailed, say, subexponential. We investigate the asymptotic tail behaviour of the distribution of the upwards first passage times. As an application, we obtain the exact rate of convergence for the ruin probability in finite time. Our result supplements similar theorems in risk theory.

scholarly journals MUSICAL MARKOV CHAINS

2012 ◽  
Vol 16 ◽  
pp. 116-135 ◽  
Author(s):  
DIMA VOLCHENKOV ◽  
JEAN RENÉ DAWIN
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Classical Music ◽  
First Passage Times ◽  
Transition Matrices ◽  
First Passage ◽  
Passage Times

A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.

Applications of generalized inverses to Markov chains

1991 ◽  
Vol 23 (2) ◽  
pp. 293-302 ◽  
Author(s):  
William Rising
Keyword(s):  
Stationary Distribution ◽  
Infinitesimal Generator ◽  
First Passage Times ◽  
Generalized Inverses ◽  
Single Server ◽  
First Passage ◽  
Triangular Matrices ◽  
Special Case ◽  
Irreducible Markov Chain ◽  
Passage Times

First it is shown that any generalized inverse of the infinitesimal generator of an irreducible Markov chain can be used to compute the exact stationary distribution and all the expected first-passage times of the chain. In the special case of a single-server queue this allows all computations to be done with upper-triangular matrices.Next it is shown that the effect of a perturbation of the infinitesimal generator on the stationary distribution and expected first-passage times can also be computed using generalized inverses. These results extend and generalize Schweitzer's [9] original work using fundamental matrices. It is then shown that any perturbation can be broken up into a series of perturbations each involving a single state.

Sign in / Sign up

Export Citation Format

Share Document