Energy of Markov chains

1979 ◽  
Vol 11 (3) ◽  
pp. 542-575 ◽  
Author(s):  
R. Syski
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Potential Theory ◽  
Excessive Function ◽  
Theory Section ◽  
Increasing Process ◽  
A Charge

After preliminaries on Markov chains, supermartingales and potential theory (Section 1), the energy of a potential supermartingale generated by an increasing process is defined. The paper examines some properties of the energy of potentials of the formUt=p(Xt) wherepis a purely excessive function (which is also a potential of a charge) for a Markov chain (Xt). Also, the mutual energy of two potentials associated with the same Markov chain is discussed. Finally, several applications and examples are worked out in detail (Section 3).

Energy of Markov chains

1979 ◽  
Vol 11 (03) ◽  
pp. 542-575 ◽  
Author(s):  
R. Syski
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Potential Theory ◽  
Excessive Function ◽  
Theory Section ◽  
Increasing Process ◽  
A Charge

After preliminaries on Markov chains, supermartingales and potential theory (Section 1), the energy of a potential supermartingale generated by an increasing process is defined. The paper examines some properties of the energy of potentials of the form U t = p(X t ) where p is a purely excessive function (which is also a potential of a charge) for a Markov chain (X t ). Also, the mutual energy of two potentials associated with the same Markov chain is discussed. Finally, several applications and examples are worked out in detail (Section 3).

Asymptotic behaviour of sample weighted circuits representing recurrent Markov chains

1990 ◽  
Vol 27 (03) ◽  
pp. 545-556 ◽  
Author(s):  
S. Kalpazidou
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Asymptotic Behaviour ◽  
Probabilistic Interpretation ◽  
Stationary Markov Chain

The asymptotic behaviour of the sequence (𝒞 n (ω), wc,n (ω)/n), is studied where 𝒞 n (ω) is the class of all cycles c occurring along the trajectory ωof a recurrent strictly stationary Markov chain (ξ n ) until time n and wc,n (ω) is the number of occurrences of the cycle c until time n. The previous sequence of sample weighted classes converges almost surely to a class of directed weighted cycles (𝒞∞, ω c ) which represents uniquely the chain (ξ n ) as a circuit chain, and ω c is given a probabilistic interpretation.

On the absorption probabilities and mean time for absorption for discrete Markov chains

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Nikolaos Halidias
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Generating Function ◽  
Discrete Time ◽  
Probability Generating Function ◽  
The Mean ◽  
Mean Time ◽  
Absorption Probabilities

Abstract In this note we study the probability and the mean time for absorption for discrete time Markov chains. In particular, we are interested in estimating the mean time for absorption when absorption is not certain and connect it with some other known results. Computing a suitable probability generating function, we are able to estimate the mean time for absorption when absorption is not certain giving some applications concerning the random walk. Furthermore, we investigate the probability for a Markov chain to reach a set A before reach B generalizing this result for a sequence of sets A 1 , A 2 , … , A k {A_{1},A_{2},\dots,A_{k}} .

scholarly journals Proportional lumpability and proportional bisimilarity

2021 ◽  
Author(s):  
Andrea Marin ◽  
Carla Piazza ◽  
Sabina Rossi
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Upper And Lower Bounds ◽  
General Definition ◽  
Performance Indices ◽  
State Aggregation ◽  
Structural Regularity ◽  
Original Definition ◽  
Aggregation Technique ◽  
Definition Of

AbstractIn this paper, we deal with the lumpability approach to cope with the state space explosion problem inherent to the computation of the stationary performance indices of large stochastic models. The lumpability method is based on a state aggregation technique and applies to Markov chains exhibiting some structural regularity. Moreover, it allows one to efficiently compute the exact values of the stationary performance indices when the model is actually lumpable. The notion of quasi-lumpability is based on the idea that a Markov chain can be altered by relatively small perturbations of the transition rates in such a way that the new resulting Markov chain is lumpable. In this case, only upper and lower bounds on the performance indices can be derived. Here, we introduce a novel notion of quasi-lumpability, named proportional lumpability, which extends the original definition of lumpability but, differently from the general definition of quasi-lumpability, it allows one to derive exact stationary performance indices for the original process. We then introduce the notion of proportional bisimilarity for the terms of the performance process algebra PEPA. Proportional bisimilarity induces a proportional lumpability on the underlying continuous-time Markov chains. Finally, we prove some compositionality results and show the applicability of our theory through examples.

scholarly journals A Bayesian model for binary Markov chains

2004 ◽  
Vol 2004 (8) ◽  
pp. 421-429 ◽  
Author(s):  
Souad Assoudou ◽  
Belkheir Essebbar
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Bayesian Estimation ◽  
Bayesian Model ◽  
Transition Probabilities ◽  
Simulated Data ◽  
Bayesian Estimator ◽  
Jeffreys Prior ◽  
Data Set

This note is concerned with Bayesian estimation of the transition probabilities of a binary Markov chain observed from heterogeneous individuals. The model is founded on the Jeffreys' prior which allows for transition probabilities to be correlated. The Bayesian estimator is approximated by means of Monte Carlo Markov chain (MCMC) techniques. The performance of the Bayesian estimates is illustrated by analyzing a small simulated data set.

scholarly journals Infinitely divisible random transition probabilities with application to dependent Markov chains

1984 ◽  
Vol 25 (4) ◽  
pp. 463-472
Author(s):  
Peter L. Chesson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
White Noise ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Rates ◽  
Infinitely Divisible ◽  
Random Transition ◽  
Transition Probability Matrices ◽  
Probability Matrices

AbstractRandom transition probability matrices with stationary independent factors define “white noise” environment processes for Markov chains. Two examples are considered in detail. Such environment processes can be used to construct several Markov chains which are dependent, have the same transition probabilities and are jointly a Markov chain. Transition rates for such processes are evaluated. These results have application to the study of animal movements.

On determining absorption probabilities for Markov chains in random environments

1981 ◽  
Vol 13 (2) ◽  
pp. 369-387 ◽  
Author(s):  
Richard D. Bourgin ◽  
Robert Cogburn
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Random Environment ◽  
Efficient Method ◽  
General Framework ◽  
Random Environments ◽  
Extinction Probabilities ◽  
Environmental Process ◽  
Branching Chains ◽  
Absorption Probabilities

The general framework of a Markov chain in a random environment is presented and the problem of determining extinction probabilities is discussed. An efficient method for determining absorption probabilities and criteria for certain absorption are presented in the case that the environmental process is a two-state Markov chain. These results are then applied to birth and death, queueing and branching chains in random environments.

Markov chains in small time intervals

1981 ◽  
Vol 18 (3) ◽  
pp. 747-751
Author(s):  
Stig I. Rosenlund
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Transition Probability ◽  
Small Time ◽  
Exact Order ◽  
Time Intervals ◽  
Continuous Parameter ◽  
Minimum Number

For a time-homogeneous continuous-parameter Markov chain we show that as t → 0 the transition probability pn,j (t) is at least of order where r(n, j) is the minimum number of jumps needed for the chain to pass from n to j. If the intensities of passage are bounded over the set of states which can be reached from n via fewer than r(n, j) jumps, this is the exact order.

Strong ergodicity for continuous-time, non-homogeneous Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 692-694 ◽  
Author(s):  
Mark Scott ◽  
Barry C. Arnold ◽  
Dean L. Isaacson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Homogeneous Markov Chain ◽  
Strong Ergodicity

Characterizations of strong ergodicity for Markov chains using mean visit times have been found by several authors (Huang and Isaacson (1977), Isaacson and Arnold (1978)). In this paper a characterization of uniform strong ergodicity for a continuous-time non-homogeneous Markov chain is given. This extends the characterization, using mean visit times, that was given by Isaacson and Arnold.

Sign in / Sign up

Export Citation Format

Share Document