scholarly journals Series Expansions for Finite-State Markov Chains

2005 ◽  
Author(s):  
Bernd Heidergott ◽  
Arie Hordijk ◽  
Miranda Van Uitert
Keyword(s):  
Markov Chains ◽  
Series Expansions ◽  
Finite State

scholarly journals SERIES EXPANSIONS FOR FINITE-STATE MARKOV CHAINS

2007 ◽  
Vol 21 (3) ◽  
pp. 381-400 ◽  
Author(s):  
Bernd Heidergott ◽  
Arie Hordijk ◽  
Miranda van Uitert
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Numerical Algorithm ◽  
Series Expansions ◽  
Finite Markov Chain ◽  
Numerical Examples ◽  
Finite State

This article provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

An extension of a convergence theorem for Markov chains arising in population genetics

2016 ◽  
Vol 53 (3) ◽  
pp. 953-956 ◽  
Author(s):  
Martin Möhle ◽  
Morihiro Notohara
Keyword(s):  
Population Genetics ◽  
Markov Chains ◽  
Convergence Theorem ◽  
Transition Matrix ◽  
Initial Distribution ◽  
Generator Matrix ◽  
Finite Dimensional ◽  
Finite State ◽  
Initial Distributions ◽  
Finite State Space

AbstractAn extension of a convergence theorem for sequences of Markov chains is derived. For every positive integer N let (XN(r))r be a Markov chain with the same finite state space S and transition matrix ΠN=I+dNBN, where I is the unit matrix, Q a generator matrix, (BN)N a sequence of matrices, limN℩∞cN= limN→∞dN=0 and limN→∞cN∕dN=0. Suppose that the limits P≔limm→∞(I+dNQ)m and G≔limN→∞PBNP exist. If the sequence of initial distributions PXN(0) converges weakly to some probability measure μ, then the finite-dimensional distributions of (XN([t∕cN))t≥0 converge to those of the Markov process (Xt)t≥0 with initial distribution μ, transition matrix PetG and limN→∞(I+dNQ+cNBN)[t∕cN]

Extreme values, range and weak convergence of integrals of Markov chains

1982 ◽  
Vol 19 (02) ◽  
pp. 272-288 ◽  
Author(s):  
P. J. Brockwell ◽  
S. I. Resnick ◽  
N. Pacheco-Santiago
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
State Space ◽  
Extreme Values ◽  
Finite State ◽  
Integral Process ◽  
Maximum Minimum ◽  
Finite State Space

A study is made of the maximum, minimum and range on [0,t] of the integral processwhereSis a finite state-space Markov chain. Approximate results are derived by establishing weak convergence of a sequence of such processes to a Wiener process. For a particular family of two-state stationary Markov chains we show that the corresponding centered integral processes exhibit the Hurst phenomenon to a remarkable degree in their pre-asymptotic behaviour.

scholarly journals A thermodynamically consistent model of finite-state machines

2018 ◽  
Vol 8 (6) ◽  
pp. 20180037 ◽  
Author(s):  
Dominique Chu ◽  
Richard E. Spinney
Keyword(s):  
Markov Chains ◽  
Entropy Production ◽  
Error Probability ◽  
Energy Levels ◽  
Finite State Machines ◽  
State Machines ◽  
Consistent Model ◽  
Resource Requirements ◽  
Finite State ◽  
Time Required

Finite-state machines (FSMs) are a theoretically and practically important model of computation. We propose a general, thermodynamically consistent model of FSMs and characterize the resource requirements of these machines. We model FSMs as time-inhomogeneous Markov chains. The computation is driven by instantaneous manipulations of the energy levels of the states. We calculate the entropy production of the machine, its error probability, and the time required to complete one update step. We find that a sequence of generalized bit-setting operations is sufficient to implement any FSM.

An invariant of representations of phase-type distributions and some applications

1996 ◽  
Vol 33 (2) ◽  
pp. 368-381 ◽  
Author(s):  
C. Commault ◽  
J. P. Chemla
Keyword(s):  
Markov Chains ◽  
Laplace Transform ◽  
Rational Functions ◽  
Absorbing State ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Finite State ◽  
The Laplace Transform ◽  
The Difference ◽  
Real Poles

In this paper we consider phase-type distributions, their Laplace transforms which are rational functions and their representations which are finite-state Markov chains with an absorbing state. We first prove that, in any representation, the minimal number of states which are visited before absorption is equal to the difference of degree between denominator and numerator in the Laplace transform of the distribution. As an application, we prove that when the Laplace transform has a denominator with n real poles and a numerator of degree less than or equal to one the distribution has order n. We show that, in general, this result can be extended neither to the case where the numerator has degree two nor to the case of non-real poles.

Sign in / Sign up

Export Citation Format

Share Document