Optimal Control of Random Walks.

This is a study of simple random walks, birth and death processes, and M/M/s queues that have transition probabilities and rates that are sequentially controlled at jump times of the processes. Each control action yields a one-step reward depending on the chosen probabilities or transition rates and the state of the process. The aim is to find control policies that maximize the total discounted or average reward. Conditions are given for these processes to have certain natural monotone optimal policies. Under such a policy for the M/M/s queue, for example, the service and arrival rates are non-decreasing and non-increasing functions, respectively, of the queue length. Properties of these policies and a linear program for computing them are also discussed.

Download Full-text

Optimal control of random walks, birth and death processes, and queues

Advances in Applied Probability ◽

10.2307/1426467 ◽

1981 ◽

Vol 13 (1) ◽

pp. 61-83 ◽

Cited By ~ 52

Author(s):

Richard Serfozo

Keyword(s):

Optimal Control ◽

Random Walks ◽

Queue Length ◽

Transition Probabilities ◽

Transition Rates ◽

Average Reward ◽

Birth And Death Processes ◽

Optimal Policies ◽

One Step ◽

Increasing Functions

This is a study of simple random walks, birth and death processes, and M/M/s queues that have transition probabilities and rates that are sequentially controlled at jump times of the processes. Each control action yields a one-step reward depending on the chosen probabilities or transition rates and the state of the process. The aim is to find control policies that maximize the total discounted or average reward. Conditions are given for these processes to have certain natural monotone optimal policies. Under such a policy for the M/M/s queue, for example, the service and arrival rates are non-decreasing and non-increasing functions, respectively, of the queue length. Properties of these policies and a linear program for computing them are also discussed.

Download Full-text