The Martin boundary for Polya's urn scheme, and an application to stochastic population growth

1964 ◽  
Vol 1 (2) ◽  
pp. 284-296 ◽  
Author(s):  
David Blackwell ◽  
David Kendall
Keyword(s):  
Markov Process ◽  
Population Growth ◽  
Transition Probability ◽  
Martin Boundary ◽  
Transition Probability Matrix ◽  
Stochastic Population Growth ◽  
Urn Scheme ◽  
Pólya’S Urn ◽  
Image Position ◽  
Positive Integers

1. In 1923 Eggenberger and Pólya introduced the following ‘urn scheme’ as a model for the development of a contagious phenomenon. A box contains b black and r red balls, and a ball is drawn from it at random with ‘double replacement’ (i.e. whatever ball is drawn, it is returned to the box together with a fresh ball of the same colour); the procedure is then continued indefinitely. A slightly more complicated version with m-fold replacement is sometimes discussed, but it will be sufficient for our purposes to keep m = 2 and it will be convenient further to simplify the scheme by taking b = r = 1 as the initial condition. We shall however generalise the scheme in another direction by allowing an arbitrary number k(≧2) of colours. Thus initially the box will contain k differently coloured balls and successive random drawings will be followed by double replacement as before. We write sn (a k-vector with jth component ) for the numerical composition of the box immediately after the nth replacement, so that and we observe that is a Markov process for which the state-space consists of all ordered k-ads of positive integers, the (constant) transition-probability matrix having elements determined by where Sn is the sum of the components of sn and (e(i))j = δij. We shall calculate the Martin boundary for this Markov process, and point out some applications to stochastic models for population growth.

The Martin boundary for Polya's urn scheme, and an application to stochastic population growth

1964 ◽  
Vol 1 (02) ◽  
pp. 284-296 ◽  
Author(s):  
David Blackwell ◽  
David Kendall
Keyword(s):  
Markov Process ◽  
Population Growth ◽  
Transition Probability ◽  
Martin Boundary ◽  
Transition Probability Matrix ◽  
Stochastic Population Growth ◽  
Urn Scheme ◽  
Pólya’S Urn ◽  
Image Position ◽  
Positive Integers

1. In 1923 Eggenberger and Pólya introduced the following ‘urn scheme’ as a model for the development of a contagious phenomenon. A box containsbblack andrred balls, and a ball is drawn from it at random with ‘double replacement’ (i.e. whatever ball is drawn, it is returned to the box together with a fresh ballof the same colour); the procedure is then continued indefinitely. A slightly more complicated version with m-fold replacement is sometimes discussed, but it will be sufficient for our purposes to keepm= 2 and it will be convenient further to simplify the scheme by takingb=r= 1 as the initial condition. We shall however generalise the scheme in another direction by allowing an arbitrary numberk(≧2) of colours. Thus initially the box will containkdifferently coloured balls and successive random drawings will be followed by double replacement as before. We writesn(ak-vector withjth component) for the numerical composition of the box immediately after the nth replacement, so thatand we observe thatis a Markov process for which the state-space consists of all orderedk-ads of positive integers, the (constant) transition-probability matrix having elements determined bywhere Snis the sum of the components of snand (e(i))j= δij. We shall calculate the Martin boundary for this Markov process, and point out some applications to stochastic models for population growth.

On Null-Recurrent Markov Chains

1960 ◽  
Vol 12 ◽  
pp. 278-288 ◽  
Author(s):  
John Lamperti
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Invariant Measure ◽  
Discrete Time ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Positive Vector ◽  
Scalar Multiple ◽  
Basic Facts ◽  
Image Position

Throughout this paper, the symbol P = [Pij] will represent the transition probability matrix of an irreducible, null-recurrent Markov process in discrete time. Explanation of this terminology and basic facts about such chains may be found in (6, ch. 15). It is known (3) that for each such matrix P there is a unique (except for a positive scalar multiple) positive vector Q = {qi} such that QP = Q, or1this vector is often called the "invariant measure" of the Markov chain.The first problem to be considered in this paper is that of determining for which vectors U(0) = {μi(0)} the vectors U(n) converge, or are summable, to the invariant measure Q, where U(n) = U(0)Pn has components2In § 2, this problem is attacked for general P. The main result is a negative one, and shows how to form U(0) for which U(n) will not be (termwise) Abel summable.

On calculating extinction probabilities for branching processes in random environments

1969 ◽  
Vol 6 (03) ◽  
pp. 478-492 ◽  
Author(s):  
William E. Wilkinson
Keyword(s):  
Generating Functions ◽  
Infinite Sequence ◽  
Branching Processes ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Random Environments ◽  
Real Numbers ◽  
Discrete Time Markov Chain ◽  
Extinction Probabilities ◽  
Image Position

Consider a discrete time Markov chain {Zn } whose state space is the non-negative integers and whose transition probability matrix ║Pij ║ possesses the representation where {Pr }, r = 1,2,…, is a finite or denumerably infinite sequence of non-negative real numbers satisfying , and , is a corresponding sequence of probability generating functions. It is assumed that Z 0 = k, a finite positive integer.

Quasi-stationary distributions in semi-Markov processes

1970 ◽  
Vol 7 (02) ◽  
pp. 388-399 ◽  
Author(s):  
C. K. Cheong
Keyword(s):  
Markov Process ◽  
State Space ◽  
Markov Processes ◽  
Transition Probability ◽  
Main Concern ◽  
Initial State ◽  
Stationary Distributions ◽  
Semi Markov Process ◽  
Image Position ◽  
Semi Markov Processes

Our main concern in this paper is the convergence, as t → ∞, of the quantities i, j ∈ E; where Pij (t) is the transition probability of a semi-Markov process whose state space E is irreducible but not closed (i.e., escape from E is possible), and rj is the probability of eventual escape from E conditional on the initial state being i. The theorems proved here generalize some results of Seneta and Vere-Jones ([8] and [11]) for Markov processes.

On the differential equations for the transition probabilities of Markov processes with enumerably many states

1953 ◽  
Vol 49 (2) ◽  
pp. 247-262 ◽  
Author(s):  
G. E. H. Reuter ◽  
W. Ledermann ◽  
M. S. Bartlett
Keyword(s):  
Markov Process ◽  
Differential Equations ◽  
Conditional Probability ◽  
Markov Processes ◽  
Transition Probabilities ◽  
Regularity Conditions ◽  
Kolmogorov Equations ◽  
Image Position ◽  
Positive Integers

Let pik (s, t) (i, k = 1, 2, …; s ≤ t) be the transition probabilities of a Markov process in a system with an enumerable set of states. The states are labelled by positive integers, and pik (s, t) is the conditional probability that the system be in state k at time t, given that it was in state i at an earlier time s. If certain regularity conditions are imposed on the pik, they can be shown to satisfy the well-known Kolmogorov equations§

Quasi-stationary distributions in semi-Markov processes

1970 ◽  
Vol 7 (2) ◽  
pp. 388-399 ◽  
Author(s):  
C. K. Cheong
Keyword(s):  
Markov Process ◽  
State Space ◽  
Markov Processes ◽  
Transition Probability ◽  
Main Concern ◽  
Initial State ◽  
Stationary Distributions ◽  
Semi Markov Process ◽  
Image Position ◽  
Semi Markov Processes

Our main concern in this paper is the convergence, as t → ∞, of the quantities i, j ∈ E; where Pij(t) is the transition probability of a semi-Markov process whose state space E is irreducible but not closed (i.e., escape from E is possible), and rj is the probability of eventual escape from E conditional on the initial state being i. The theorems proved here generalize some results of Seneta and Vere-Jones ([8] and [11]) for Markov processes.

On calculating extinction probabilities for branching processes in random environments

1969 ◽  
Vol 6 (3) ◽  
pp. 478-492 ◽  
Author(s):  
William E. Wilkinson
Keyword(s):  
Generating Functions ◽  
Infinite Sequence ◽  
Branching Processes ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Random Environments ◽  
Real Numbers ◽  
Discrete Time Markov Chain ◽  
Extinction Probabilities ◽  
Image Position

Consider a discrete time Markov chain {Zn} whose state space is the non-negative integers and whose transition probability matrix ║Pij║ possesses the representation where {Pr}, r = 1,2,…, is a finite or denumerably infinite sequence of non-negative real numbers satisfying , and , is a corresponding sequence of probability generating functions. It is assumed that Z0 = k, a finite positive integer.

A note on cumulative sums of markovian variables

1965 ◽  
Vol 5 (2) ◽  
pp. 285-287 ◽  
Author(s):  
R. M. Phatarfod
Keyword(s):  
Markov Chain ◽  
Probability Distribution ◽  
Finite Number ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
The Matrix ◽  
Initial Probability Distribution ◽  
Regular Markov Chain ◽  
Image Position ◽  
Cumulative Sums

Consider a positive regular Markov chain X0, X1, X2,… with s(s finite) number of states E1, E2,… E8, and a transition probability matrix P = (pij) where = , and an initial probability distribution given by the vector p0. Let {Zr} be a sequence of random variables such that and consider the sum SN = Z1+Z2+ … ZN. It can easily be shown that (cf. Bartlett [1] p. 37), where λ1(t), λ2(t)…λ1(t) are the latent roots of P(t) ≡ (pijethij) and si(t) and t′i(t) are the column and row vectors corresponding to λi(t), and so constructed as to give t′i(t)Si(t) = 1 and t′i(t), si(o) = si where t′i(t) and si are the corresponding column and row vectors, considering the matrix .

scholarly journals The limiting behavior of transient birth and death processes conditioned on survival

1968 ◽  
Vol 8 (4) ◽  
pp. 716-722 ◽  
Author(s):  
Phillip Good
Keyword(s):  
Transition Probability ◽  
Transition Probability Matrix ◽  
Death Process ◽  
Initial Condition ◽  
Birth And Death Process ◽  
Birth And Death Processes ◽  
Limiting Behavior ◽  
Image Position ◽  
Number Of Individuals ◽  
Over Time

The development of a population over time can often be simulated by the behavior of a birth and death process, whose transition probability matrix P(t) = (Pij(t), where X(t) denotes the number of individuals at time t, satisfies the differential equations and the initial condition

Sign in / Sign up

Export Citation Format

Share Document