Partially observable semi-Markov reward processes

1993 ◽  
Vol 30 (3) ◽  
pp. 548-560 ◽  
Author(s):  
Yasushi Masuda
Keyword(s):  
Markov Process ◽  
Counting Process ◽  
Probabilistic Interpretation ◽  
Real Domain ◽  
Semi Markov Process ◽  
Reward Process ◽  
Markov Reward ◽  
Partially Observable ◽  
Joint Behavior

The main objective of this paper is to investigate the conditional behavior of the multivariate reward process given the number of certain signals where the underlying system is described by a semi-Markov process and the signal is defined by a counting process. To this end, we study the joint behavior of the multivariate reward process and the multivariate counting process in detail. We derive transform results as well as the corresponding real domain expressions, thus providing clear probabilistic interpretation.

Partially observable semi-Markov reward processes

1993 ◽  
Vol 30 (03) ◽  
pp. 548-560 ◽  
Author(s):  
Yasushi Masuda
Keyword(s):  
Markov Process ◽  
Counting Process ◽  
Probabilistic Interpretation ◽  
Real Domain ◽  
Semi Markov Process ◽  
Reward Process ◽  
Markov Reward ◽  
Partially Observable ◽  
Joint Behavior

The main objective of this paper is to investigate the conditional behavior of the multivariate reward process given the number of certain signals where the underlying system is described by a semi-Markov process and the signal is defined by a counting process. To this end, we study the joint behavior of the multivariate reward process and the multivariate counting process in detail. We derive transform results as well as the corresponding real domain expressions, thus providing clear probabilistic interpretation.

Analysis of a counting process associated with a semi-Markov process: number of entries into a subset of state space

1987 ◽  
Vol 19 (4) ◽  
pp. 767-783 ◽  
Author(s):  
Yasushi Masuda ◽  
Ushio Sumita
Keyword(s):  
Markov Process ◽  
State Space ◽  
Counting Process ◽  
Probabilistic Interpretation ◽  
First Passage ◽  
Marginal Process ◽  
The Laplace Transform ◽  
Semi Markov Process ◽  
Age Process ◽  
Passage Times

Let N(t) be a finite semi-Markov process on 𝒩 and let X(t) be the associated age process. Of interest is the counting process M(t) for transitions of the semi-Markov process from a subset G of 𝒩 to another subset B where 𝒩 = B ∪ G and B ∩ G = ∅. By studying the trivariate process Y(t) =[N(t), M(t), X(t)] in its state space, new transform results are derived. By taking M(t) as a marginal process of Y(t), the Laplace transform generating function of M(t) is then obtained. Furthermore, this result is recaptured in the context of first-passage times of the semi-Markov process, providing a simple probabilistic interpretation. The asymptotic behavior of the moments of M(t) as t → ∞ is also discussed. In particular, an asymptotic expansion for E[M(t)] and the limit for Var [M(t)]/t as t → ∞ are given explicitly.

Analysis of a counting process associated with a semi-Markov process: number of entries into a subset of state space

1987 ◽  
Vol 19 (04) ◽  
pp. 767-783 ◽  
Author(s):  
Yasushi Masuda ◽  
Ushio Sumita
Keyword(s):  
Markov Process ◽  
State Space ◽  
Counting Process ◽  
Probabilistic Interpretation ◽  
First Passage ◽  
Marginal Process ◽  
The Laplace Transform ◽  
Semi Markov Process ◽  
Age Process ◽  
Passage Times

Let N(t) be a finite semi-Markov process on 𝒩 and let X(t) be the associated age process. Of interest is the counting process M(t) for transitions of the semi-Markov process from a subset G of 𝒩 to another subset B where 𝒩 = B ∪ G and B ∩ G = ∅. By studying the trivariate process Y(t) =[N(t), M(t), X(t)] in its state space, new transform results are derived. By taking M(t) as a marginal process of Y(t), the Laplace transform generating function of M(t) is then obtained. Furthermore, this result is recaptured in the context of first-passage times of the semi-Markov process, providing a simple probabilistic interpretation. The asymptotic behavior of the moments of M(t) as t → ∞ is also discussed. In particular, an asymptotic expansion for E[M(t)] and the limit for Var [M(t)]/t as t → ∞ are given explicitly.

Reward processes with nonlinear reward functions

1996 ◽  
Vol 33 (4) ◽  
pp. 1011-1017 ◽  
Author(s):  
A. Reza Soltani
Keyword(s):  
Markov Process ◽  
Explicit Formula ◽  
Sojourn Time ◽  
Time Distribution ◽  
Reward Function ◽  
Sojourn Time Distribution ◽  
Semi Markov Process ◽  
Reward Process ◽  
Reward Functions

Based on a semi-Markov process J(t), t ≧ 0, a reward process Z(t), t ≧ 0, is introduced where it is assumed that the reward function, p(k, x) is nonlinear; if the reward function is linear, i.e. ρ (k, x) = kx, the reward process Z(t), t ≧ 0, becomes the classical one, which has been considered by many authors. An explicit formula for E(Z(t)) is given in terms of the moments of the sojourn time distribution at t, when the reward function is a polynomial.

A multivariate reward process defined on a semi-Markov process and its first-passage-time distributions

1991 ◽  
Vol 28 (2) ◽  
pp. 360-373 ◽  
Author(s):  
Yasushi Masuda ◽  
Ushio Sumita
Keyword(s):  
Asymptotic Behavior ◽  
Markov Process ◽  
First Passage Time ◽  
Passage Time ◽  
First Passage ◽  
Renewal Equations ◽  
Supplementary Variables ◽  
Semi Markov Process ◽  
Reward Process

A multivariate reward process defined on a semi-Markov process is studied. Transform results for the distributions of the multivariate reward and related processes are derived through the method of supplementary variables and the Markov renewal equations. These transform results enable the asymptotic behavior to be analyzed. A class of first-passage time distributions of the multivariate reward processes is also investigated.

A multivariate reward process defined on a semi-Markov process and its first-passage-time distributions

1991 ◽  
Vol 28 (02) ◽  
pp. 360-373 ◽  
Author(s):  
Yasushi Masuda ◽  
Ushio Sumita
Keyword(s):  
Asymptotic Behavior ◽  
Markov Process ◽  
First Passage Time ◽  
Passage Time ◽  
First Passage ◽  
Renewal Equations ◽  
Supplementary Variables ◽  
Semi Markov Process ◽  
Reward Process

A multivariate reward process defined on a semi-Markov process is studied. Transform results for the distributions of the multivariate reward and related processes are derived through the method of supplementary variables and the Markov renewal equations. These transform results enable the asymptotic behavior to be analyzed. A class of first-passage time distributions of the multivariate reward processes is also investigated.

Reward processes with nonlinear reward functions

1996 ◽  
Vol 33 (04) ◽  
pp. 1011-1017 ◽  
Author(s):  
A. Reza Soltani
Keyword(s):  
Markov Process ◽  
Explicit Formula ◽  
Sojourn Time ◽  
Time Distribution ◽  
Reward Function ◽  
Sojourn Time Distribution ◽  
Semi Markov Process ◽  
Reward Process ◽  
Reward Functions

Based on a semi-Markov process J(t), t ≧ 0, a reward process Z(t), t ≧ 0, is introduced where it is assumed that the reward function, p(k, x) is nonlinear; if the reward function is linear, i.e. ρ (k, x) = kx, the reward process Z(t), t ≧ 0, becomes the classical one, which has been considered by many authors. An explicit formula for E(Z(t)) is given in terms of the moments of the sojourn time distribution at t, when the reward function is a polynomial.

Sign in / Sign up

Export Citation Format

Share Document