Identification and Adaptive Control of Markov Chains

1982 ◽  
Vol 20 (4) ◽  
pp. 470-489 ◽  
Author(s):  
Vivek Borkar ◽  
Pravin Varaiya
Keyword(s):  
Adaptive Control ◽  
Markov Chains

Adaptive control of Markov chains with local updates

1990 ◽  
Vol 14 (3) ◽  
pp. 209-218 ◽  
Author(s):  
Ahmad Jalali ◽  
Michael Ferguson
Keyword(s):  
Adaptive Control ◽  
Markov Chains

Minimizing the learning loss in adaptive control of Markov chains under the weak accessibility condition

1991 ◽  
Vol 28 (04) ◽  
pp. 779-790 ◽  
Author(s):  
Rajeev Agrawal
Keyword(s):  
Adaptive Control ◽  
Maximum Likelihood ◽  
Markov Chains ◽  
Upper Bound ◽  
Unknown Parameter ◽  
Learning Loss ◽  
Probability Of Error ◽  
Control Scheme ◽  
Rate Of Decay ◽  
Randomized Control

We consider the adaptive control of Markov chains under the weak accessibility condition with a view to minimizing the learning loss. A certainty equivalence control with a forcing scheme is constructed. We use a stationary randomized control scheme for forcing and compute a maximum likelihood estimate of the unknown parameter from the resulting observations. We obtain an exponential upper bound on the rate of decay of the probability of error. This allows us to choose the rate of forcing appropriately, whereby we achieve a o(f(n) log n) learning loss for any function as .

Sign in / Sign up

Export Citation Format

Share Document