scholarly journals On strong stationary times and approximation of Markov chain hitting times by geometric sums

2019 ◽  
Vol 150 ◽  
pp. 74-80
Author(s):  
Fraser Daly
Keyword(s):  
Markov Chain ◽  
Hitting Times ◽  
Geometric Sums

On identifiability and order of continuous-time aggregated Markov chains, Markov-modulated Poisson processes, and phase-type distributions

1996 ◽  
Vol 33 (3) ◽  
pp. 640-653 ◽  
Author(s):  
Tobias Rydén
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Poisson Processes ◽  
Hitting Times ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Minimum Number ◽  
Markov Modulated ◽  
Two Parameters

An aggregated Markov chain is a Markov chain for which some states cannot be distinguished from each other by the observer. In this paper we consider the identifiability problem for such processes in continuous time, i.e. the problem of determining whether two parameters induce identical laws for the observable process or not. We also study the order of a continuous-time aggregated Markov chain, which is the minimum number of states needed to represent it. In particular, we give a lower bound on the order. As a by-product, we obtain results of this kind also for Markov-modulated Poisson processes, i.e. doubly stochastic Poisson processes whose intensities are directed by continuous-time Markov chains, and phase-type distributions, which are hitting times in finite-state Markov chains.

scholarly journals Undiscounted Markov Chain BSDEs to Stopping Times

2014 ◽  
Vol 51 (1) ◽  
pp. 262-281
Author(s):  
Samuel N. Cohen
Keyword(s):  
Markov Chain ◽  
Differential Equations ◽  
Continuous Time ◽  
Stopping Time ◽  
Stopping Times ◽  
Hitting Times ◽  
Continuous Time Markov Chain ◽  
Countable State ◽  
Growth Bound ◽  
Novel Applications

We consider backward stochastic differential equations in a setting where noise is generated by a countable state, continuous time Markov chain, and the terminal value is prescribed at a stopping time. We show that, given sufficient integrability of the stopping time and a growth bound on the terminal value and BSDE driver, these equations admit unique solutions satisfying the same growth bound (up to multiplication by a constant). This holds without assuming that the driver is monotone in y, that is, our results do not require that the terminal value be discounted at some uniform rate. We show that the conditions are satisfied for hitting times of states of the chain, and hence present some novel applications of the theory of these BSDEs.

scholarly journals On the Moments of Hitting Times for Random Walks on Trees

2009 ◽  
Vol 2009 ◽  
pp. 1-4 ◽  
Author(s):  
José Luis Palacios
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Random Walks ◽  
Simple Random Walk ◽  
Hitting Times ◽  
Natural Numbers ◽  
Ergodic Markov Chain

Using classical arguments we derive a formula for the moments of hitting times for an ergodic Markov chain. We apply this formula to the case of simple random walk on trees and show, with an elementary electric argument, that all the moments are natural numbers.

scholarly journals Markov Chain–Based Stochastic Strategies for Robotic Surveillance

2020 ◽  
Vol 4 (1) ◽  
Author(s):  
Xiaoming Duan ◽  
Francesco Bullo
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Mobile Robots ◽  
Performance Metrics ◽  
Optimization Problems ◽  
Autonomous Systems ◽  
Hitting Times ◽  
Annual Review ◽  
Publication Date ◽  
Motion Models

This article surveys recent advancements in strategy designs for persistent robotic surveillance tasks, with a focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way, where the efficiency is defined with respect to relevant underlying performance metrics. We start by reviewing the basics of Markov chains, which are the primary motion models for stochastic robotic surveillance. We then discuss the two main criteria regarding the speed and unpredictability of surveillance strategies. The central objects that appear throughout the treatment are the hitting times of Markov chains, their distributions, and their expectations. We formulate various optimization problems based on the relevant metrics in different scenarios and establish their respective properties. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Markov chains with binomial time change

1986 ◽  
Vol 23 (02) ◽  
pp. 519-523
Author(s):  
Kyle Siegrist
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Time Variable ◽  
Time Change ◽  
Hitting Times ◽  
Potential Operator ◽  
Limiting Behavior ◽  
Binomial Parameter

The effect of a binomial time change on a given Markov chain is studied. Results are obtained for the hitting times, potential operator, and transience and recurrence properties of the time-changed chain. The limiting behavior is considered as the binomial parameter approaches 0 and the time variable approaches∞.

Entropy of killed-resurrected stationary Markov chains

2021 ◽  
Vol 58 (1) ◽  
pp. 177-196
Author(s):  
Servet Martínez
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Probability Measure ◽  
Markov Chains ◽  
Stochastic Matrix ◽  
Hitting Times ◽  
Stationary Distributions ◽  
Natural Factors ◽  
Absorbing States ◽  
Stationary Representation

AbstractWe consider a strictly substochastic matrix or a stochastic matrix with absorbing states. By using quasi-stationary distributions we show that there is an associated canonical Markov chain that is built from the resurrected chain, the absorbing states, and the hitting times, together with a random walk on the absorbing states, which is necessary for achieving time stationarity. Based upon the 2-stringing representation of the resurrected chain, we supply a stationary representation of the killed and the absorbed chains. The entropies of these representations have a clear meaning when one identifies the probability measure of natural factors. The balance between the entropies of these representations and the entropy of the canonical chain serves to check the correctness of the whole construction.

Markov chains with binomial time change

1986 ◽  
Vol 23 (2) ◽  
pp. 519-523 ◽  
Author(s):  
Kyle Siegrist
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Time Variable ◽  
Time Change ◽  
Hitting Times ◽  
Potential Operator ◽  
Limiting Behavior ◽  
Binomial Parameter

The effect of a binomial time change on a given Markov chain is studied. Results are obtained for the hitting times, potential operator, and transience and recurrence properties of the time-changed chain. The limiting behavior is considered as the binomial parameter approaches 0 and the time variable approaches∞.

On spectral gap estimates of a Markov chain via hitting times and coupling

1999 ◽  
Vol 36 (2) ◽  
pp. 310-319 ◽  
Author(s):  
Christian Meise
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Spectral Gap ◽  
Sharp Estimate ◽  
Contraction Principle ◽  
Hitting Times ◽  
Nearest Neighbour ◽  
Discrete Time Markov Chain ◽  
Strongly Connected ◽  
Dimensional Cube

Well-known inequalities for the spectral gap of a discrete-time Markov chain, such as Poincaré's and Cheeger's inequalities, do not perform well if the transition graph of the Markov chain is strongly connected. For example in the case of nearest-neighbour random walk on the n-dimensional cube Poincaré's and Cheeger's inequalities are off by a factor n. Using a coupling technique and a contraction principle lower bounds on the spectral gap can be derived. Finally, we show that using the contraction principle yields a sharp estimate for nearest-neighbour random walk on the n-dimensional cube.

Sign in / Sign up

Export Citation Format

Share Document