Estimation and control in Markov chains

1974 ◽  
Vol 6 (1) ◽  
pp. 40-60 ◽  
Author(s):  
P. Mandl
Keyword(s):  
Markov Chains ◽  
Asymptotic Properties ◽  
Control Policy ◽  
Contrast Method ◽  
The Central Limit Theorem ◽  
Large Numbers ◽  
Estimation And Control ◽  
And Control ◽  
Stationary Control ◽  
Optimal Stationary Control

We consider a finite controlled Markov chain, the description of which depends on an unknown parameter a, and investigate the following control policy. To each a an optimal stationary control is associated. a is estimated recurrently from the trajectory by the minimum contrast method, and the optimal stationary control corresponding to the estimate is used. We present asymptotic properties of the estimate and of the criterion function. They follow from the law of large numbers and from the central limit theorem for controlled Markov chains derived with the aid of martingales.

Estimation and control in Markov chains

1974 ◽  
Vol 6 (01) ◽  
pp. 40-60 ◽  
Author(s):  
P. Mandl
Keyword(s):  
Markov Chains ◽  
Asymptotic Properties ◽  
Control Policy ◽  
Contrast Method ◽  
The Central Limit Theorem ◽  
Large Numbers ◽  
Estimation And Control ◽  
And Control ◽  
Stationary Control ◽  
Optimal Stationary Control

We consider a finite controlled Markov chain, the description of which depends on an unknown parameter a, and investigate the following control policy. To each a an optimal stationary control is associated. a is estimated recurrently from the trajectory by the minimum contrast method, and the optimal stationary control corresponding to the estimate is used. We present asymptotic properties of the estimate and of the criterion function. They follow from the law of large numbers and from the central limit theorem for controlled Markov chains derived with the aid of martingales.

On controlled one-dimensional diffusion processes with unknown parameter

1977 ◽  
Vol 9 (01) ◽  
pp. 105-124 ◽  
Author(s):  
Věra Dufková
Keyword(s):  
Unknown Parameter ◽  
Diffusion Processes ◽  
Asymptotic Properties ◽  
Control Policy ◽  
Controlled Diffusion ◽  
The Central Limit Theorem ◽  
Large Numbers ◽  
The Law ◽  
Stationary Control ◽  
Optimal Stationary Control

We consider a controlled diffusion process, the description of which depends on an unknown parameter α, and investigate the following control policy. To each α an optimal stationary control is associated. α is estimated recurrently from the trajectory by Bayes' method, and the optimal stationary control corresponding to the estimate is used. We establish the consistency of the estimate, and present asymptotic properties of the criterion function. They follow from the central limit theorem, from the law of large numbers and from the law of the iterated logarithm for local martingales.

On controlled one-dimensional diffusion processes with unknown parameter

1977 ◽  
Vol 9 (1) ◽  
pp. 105-124 ◽  
Author(s):  
Věra Dufková
Keyword(s):  
Unknown Parameter ◽  
Diffusion Processes ◽  
Asymptotic Properties ◽  
Control Policy ◽  
Controlled Diffusion ◽  
The Central Limit Theorem ◽  
Large Numbers ◽  
The Law ◽  
Stationary Control ◽  
Optimal Stationary Control

We consider a controlled diffusion process, the description of which depends on an unknown parameter α, and investigate the following control policy. To each α an optimal stationary control is associated. α is estimated recurrently from the trajectory by Bayes' method, and the optimal stationary control corresponding to the estimate is used. We establish the consistency of the estimate, and present asymptotic properties of the criterion function. They follow from the central limit theorem, from the law of large numbers and from the law of the iterated logarithm for local martingales.

Asymptotic Properties of a Supercritical Branching Process with Immigration in a Random Environment

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Yanqing Wang ◽  
Quansheng Liu
Keyword(s):  
Random Environment ◽  
Limit Theorems ◽  
Branching Process ◽  
Law Of Large Numbers ◽  
Large Deviation ◽  
Asymptotic Properties ◽  
Moderate Deviation ◽  
The Central Limit Theorem ◽  
Large Numbers ◽  
Branching Process With Immigration

Abstract This is a short survey about asymptotic properties of a supercritical branching process ( Z n ) (Z_{n}) with immigration in a stationary and ergodic or independent and identically distributed random environment. We first present basic properties of the fundamental submartingale ( W n ) (W_{n}) , about the a.s. convergence, the non-degeneracy of its limit 𝑊, the convergence in L p L^{p} for p ≥ 1 p\geq 1 , and the boundedness of the harmonic moments E ⁢ W n - a \mathbb{E}W_{n}^{-a} , a > 0 a>0 . We then present limit theorems and large deviation results on log ⁡ Z n \log Z_{n} , including the law of large numbers, large and moderate deviation principles, the central limit theorem with Berry–Esseen’s bound, and Cramér’s large deviation expansion. Some key ideas of the proofs are also presented.

Sign in / Sign up

Export Citation Format

Share Document