scholarly journals Recursive filters for partially observable finite Markov chains

2005 ◽  
Vol 42 (3) ◽  
pp. 684-697 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Arrival Process ◽  
Estimation Of Parameters ◽  
Batch Markovian Arrival Process ◽  
Recursive Filters ◽  
Conditional Expectations ◽  
Finite Markov Chains ◽  
Partially Observable ◽  
Recursive Equations

In this note, we consider discrete-time finite Markov chains and assume that they are only partly observed. We obtain finite-dimensional normalized filters for basic statistics associated with such processes. Recursive equations for these filters are derived by means of simple computations involving conditional expectations. An application to the estimation of parameters of the so-called discrete-time batch Markovian arrival process is outlined.

scholarly journals Recursive filters for partially observable finite Markov chains

2005 ◽  
Vol 42 (03) ◽  
pp. 684-697 ◽  
Author(s):  
James Ledoux
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Arrival Process ◽  
Estimation Of Parameters ◽  
Batch Markovian Arrival Process ◽  
Recursive Filters ◽  
Conditional Expectations ◽  
Finite Markov Chains ◽  
Partially Observable ◽  
Recursive Equations

In this note, we consider discrete-time finite Markov chains and assume that they are only partly observed. We obtain finite-dimensional normalized filters for basic statistics associated with such processes. Recursive equations for these filters are derived by means of simple computations involving conditional expectations. An application to the estimation of parameters of the so-called discrete-time batch Markovian arrival process is outlined.

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

Lumpability for non-irreducible finite markov chains

1984 ◽  
Vol 21 (03) ◽  
pp. 567-574 ◽  
Author(s):  
Atef M. Abdel-Moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
The State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

Conditions under which a function of a finite, discrete-time Markov chain, X(t), is again Markov are given, when X(t) is not irreducible. These conditions are given in terms of an interrelationship between two partitions of the state space of X(t), the partition induced by the minimal essential classes of X(t) and the partition with respect to which lumping is to be considered.

scholarly journals Elucidating the Short Term Loss Behavior of Markovian-Modulated Batch-Service Queueing Model with Discrete-Time Batch Markovian Arrival Process

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Yung-Chung Wang ◽  
Dong-Liang Cai ◽  
Li-Hsin Chiang ◽  
Cheng-Wei Hu
Keyword(s):  
Discrete Time ◽  
Packet Loss ◽  
Loss Probability ◽  
Markovian Arrival Process ◽  
Arrival Process ◽  
Service Process ◽  
Batch Service ◽  
Temporal Behavior ◽  
Packet Loss Probability ◽  
Batch Markovian Arrival Process

This paper applies a matrix-analytical approach to analyze the temporal behavior of Markovian-modulated batch-service queue with discrete-time batch Markovian arrival process (DBMAP). The service process is correlated and its structure is presented through discrete-time batch Markovian service process (DBMSP). We examine the temporal behavior of packet loss by means of conditional statistics with respect to congested and noncongested periods that occur in an alternating manner. The congested period corresponds to having more than a certain number of packets in the buffer; noncongested period corresponds to the opposite. All of the four related performance measures are derived, including probability distributions of a congested and noncongested periods, the probability that the system stays in a congested period, the packet loss probability during congested period, and the long term packet loss probability. Queueing systems of this type arise in the domain of wireless communications.

General transition probabilities for finite Markov chains

1964 ◽  
Vol 60 (1) ◽  
pp. 83-91 ◽  
Author(s):  
Marcel F. Neuts
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Finite Number ◽  
Discrete Time ◽  
Transition Probabilities ◽  
Initial State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

We consider a stationary discrete-time Markov chain with a finite number m of possible states which we designate by 1,…,m. We assume that at time t = 0 the process is in an initial state i with probability (i = 1,…, m) and such that and .

On Quasi-Stationary distributions in absorbing discrete-time finite Markov chains

1965 ◽  
Vol 2 (1) ◽  
pp. 88-100 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Stationary Distributions ◽  
Transient States ◽  
Reverse Process ◽  
Finite Markov Chains ◽  
Quasi Stationary Distribution

The time to absorption from the set T of transient states of a Markov chain may be sufficiently long for the probability distribution over T to settle down in some sense to a “quasi-stationary” distribution. Various analogues of the stationary distribution of an irreducible chain are suggested and compared. The reverse process of an absorbing chain is found to be relevant.

On Quasi-Stationary distributions in absorbing discrete-time finite Markov chains

1965 ◽  
Vol 2 (01) ◽  
pp. 88-100 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Discrete Time ◽  
Stationary Distributions ◽  
Transient States ◽  
Reverse Process ◽  
Finite Markov Chains ◽  
Quasi Stationary Distribution

The time to absorption from the set T of transient states of a Markov chain may be sufficiently long for the probability distribution over T to settle down in some sense to a “quasi-stationary” distribution. Various analogues of the stationary distribution of an irreducible chain are suggested and compared. The reverse process of an absorbing chain is found to be relevant.

Sign in / Sign up

Export Citation Format

Share Document