Optimal service-rate control of M/G/1 queueing systems using phase methods

1983 ◽  
Vol 15 (3) ◽  
pp. 616-637 ◽  
Author(s):  
Kyung Y. Jo ◽  
Shaler Stidham
Keyword(s):  
Partial Information ◽  
Queueing Systems ◽  
Infinite Horizon ◽  
Service Rate ◽  
Erlang Distribution ◽  
Work Requirement ◽  
Average Return ◽  
Long Run ◽  
Markov Decision ◽  
Method Of Phases

A new approach to the optimal control of the service rate in M/G/1 queues is introduced using the method of phases. Each customer's work requirement is approximated by a random number of exponential phases with (possibly) different parameters (a generalized hyper-Erlang distribution). Using a semi-Markov decision-process formulation, we establish monotonicity properties of optimal policies for the finite-horizon problem, by induction on the horizon length. The analysis is then extended to the discounted infinite-horizon and the long-run average-return problems. In contrast to the models in previous papers, our model is appropriate for situations where the system controller has partial information about the work requirement of a customer, specifically the number of phases (tasks) to be performed. Because it requires a multidimensional state description, the analysis of the phase-type control model may be viewed as a first step toward the solution of control models for networks of queues.

Optimal service-rate control of M/G/1 queueing systems using phase methods

1983 ◽  
Vol 15 (03) ◽  
pp. 616-637 ◽  
Author(s):  
Kyung Y. Jo ◽  
Shaler Stidham
Keyword(s):  
Partial Information ◽  
Queueing Systems ◽  
Infinite Horizon ◽  
Service Rate ◽  
Erlang Distribution ◽  
Work Requirement ◽  
Average Return ◽  
Long Run ◽  
Markov Decision ◽  
Method Of Phases

A new approach to the optimal control of the service rate in M/G/1 queues is introduced using the method of phases. Each customer's work requirement is approximated by a random number of exponential phases with (possibly) different parameters (a generalized hyper-Erlang distribution). Using a semi-Markov decision-process formulation, we establish monotonicity properties of optimal policies for the finite-horizon problem, by induction on the horizon length. The analysis is then extended to the discounted infinite-horizon and the long-run average-return problems. In contrast to the models in previous papers, our model is appropriate for situations where the system controller has partial information about the work requirement of a customer, specifically the number of phases (tasks) to be performed. Because it requires a multidimensional state description, the analysis of the phase-type control model may be viewed as a first step toward the solution of control models for networks of queues.

scholarly journals LTL-Constrained Steady-State Policy Synthesis

2021 ◽  
Author(s):  
Jan Křetínský
Keyword(s):  
Steady State ◽  
State Policy ◽  
Infinite Horizon ◽  
Long Run ◽  
Markov Decision ◽  
Quantitative Properties ◽  
The One ◽  
The Given ◽  
Type Specification ◽  
Ltl Model Checking

Decision-making policies for agents are often synthesized with the constraint that a formal specification of behaviour is satisfied. Here we focus on infinite-horizon properties. On the one hand, Linear Temporal Logic (LTL) is a popular example of a formalism for qualitative specifications. On the other hand, Steady-State Policy Synthesis (SSPS) has recently received considerable attention as it provides a more quantitative and more behavioural perspective on specifications, in terms of the frequency with which states are visited. Finally, rewards provide a classic framework for quantitative properties. In this paper, we study Markov decision processes (MDP) with the specification combining all these three types. The derived policy maximizes the reward among all policies ensuring the LTL specification with the given probability and adhering to the steady-state constraints. To this end, we provide a unified solution reducing the multi-type specification to a multi-dimensional long-run average reward. This is enabled by Limit-Deterministic Büchi Automata (LDBA), recently studied in the context of LTL model checking on MDP, and allows for an elegant solution through a simple linear programme. The algorithm also extends to the general omega-regular properties and runs in time polynomial in the sizes of the MDP as well as the LDBA.

Perturbation theory for Markov reward processes with applications to queueing systems

1988 ◽  
Vol 20 (1) ◽  
pp. 79-98 ◽  
Author(s):  
Nico M. Van Dijk ◽  
Martin L. Puterman
Keyword(s):  
First Passage Time ◽  
Queueing Systems ◽  
Infinite Horizon ◽  
Passage Time ◽  
Decision Processes ◽  
Markov Decision ◽  
Reward Process ◽  
Reward Functions ◽  
Markov Reward ◽  
Time Bounds

We study the effect of perturbations in the data of a discrete-time Markov reward process on the finite-horizon total expected reward, the infinite-horizon expected discounted and average reward and the total expected reward up to a first-passage time. Bounds for the absolute errors of these reward functions are obtained. The results are illustrated for a finite as well as infinite queueing systems (M/M/1/S and ). Extensions to Markov decision processes and other settings are discussed.

Arrival control of load-sharing queueing systems

1984 ◽  
Vol 16 (1) ◽  
pp. 9-9
Author(s):  
Kyung Y. Jo ◽  
Arnold Greenland
Keyword(s):  
Customer Service ◽  
Markov Decision Process ◽  
Decision Process ◽  
Original Problem ◽  
Partial Information ◽  
Queueing Systems ◽  
Work Load ◽  
Optimal Solutions ◽  
Holding Costs ◽  
Markov Decision

We consider a system of three queues in which arriving customers are assigned to other queues if rejected from entry to one queue; and thus the work load of each queue is shared. The objective function considered is a combination of holding costs, routing costs, and customer service rewards. We first establish the characteristics of optimal control policies via a Markov decision process formulation. Next, the decomposed problems with partial information for each server are considered and the results compared with the original problem are shown. Appropriate combinations of optimal solutions for the decomposed problems are then used either in approximating the centralized optimal policy or in determining a good starting policy for successive approximation of the multidimensional Markov decision process. Numerical results of specific models are also presented.

Perturbation theory for Markov reward processes with applications to queueing systems

1988 ◽  
Vol 20 (01) ◽  
pp. 79-98 ◽  
Author(s):  
Nico M. Van Dijk ◽  
Martin L. Puterman
Keyword(s):  
First Passage Time ◽  
Queueing Systems ◽  
Infinite Horizon ◽  
Passage Time ◽  
Decision Processes ◽  
Markov Decision ◽  
Reward Process ◽  
Reward Functions ◽  
Markov Reward ◽  
Time Bounds

We study the effect of perturbations in the data of a discrete-time Markov reward process on the finite-horizon total expected reward, the infinite-horizon expected discounted and average reward and the total expected reward up to a first-passage time. Bounds for the absolute errors of these reward functions are obtained. The results are illustrated for a finite as well as infinite queueing systems (M/M/1/S and ). Extensions to Markov decision processes and other settings are discussed.

scholarly journals Analysis of the Optimal Resource Allocation for a Tandem Queueing System

2017 ◽  
Vol 2017 ◽  
pp. 1-10 ◽  
Author(s):  
Zaiming Liu ◽  
Wei Deng ◽  
Gang Chen
Keyword(s):  
Optimal Policy ◽  
Optimal Allocation ◽  
Queueing System ◽  
Service Rate ◽  
Allocation Policy ◽  
Long Run ◽  
Markov Decision ◽  
Upstream Station ◽  
Optimal Resource ◽  
Resource Cost

We study a controllable two-station tandem queueing system, where customers (jobs) must first be processed at upstream station and then the downstream station. A manager dynamically allocates the service resource to each station to adjust the service rate, leading to a tradeoff between the holding cost and resource cost. The goal of the manager is to find the optimal policy to minimize the long-run average costs. The problem is constructed as a Markov decision process (MDP). In this paper, we consider the model in which the resource cost and service rate functions are more general than linear. We derive the monotonicity of the optimal allocation policy by the quasiconvexity properties of the value function. Furthermore, we obtain the relationship between the two stations’ optimal policy and conditions under which the optimal policy is unique and has the bang-bang control property. Finally, we provide some numerical experiments to illustrate these results.

scholarly journals A Taylor Series Approach for Service-Coupled Queueing Systems with Intermediate Load

2017 ◽  
Vol 2017 ◽  
pp. 1-10 ◽  
Author(s):  
Ekaterina Evdokimova ◽  
Sabine Wittevrongel ◽  
Dieter Fiems
Keyword(s):  
Performance Measures ◽  
Queueing Systems ◽  
Poisson Processes ◽  
Series Expansions ◽  
Service Rate ◽  
Single Server ◽  
Queueing Model ◽  
State Space Explosion ◽  
Finite Queues ◽  
Service Rates

This paper investigates the performance of a queueing model with multiple finite queues and a single server. Departures from the queues are synchronised or coupled which means that a service completion leads to a departure in every queue and that service is temporarily interrupted whenever any of the queues is empty. We focus on the numerical analysis of this queueing model in a Markovian setting: the arrivals in the different queues constitute Poisson processes and the service times are exponentially distributed. Taking into account the state space explosion problem associated with multidimensional Markov processes, we calculate the terms in the series expansion in the service rate of the stationary distribution of the Markov chain as well as various performance measures when the system is (i) overloaded and (ii) under intermediate load. Our numerical results reveal that, by calculating the series expansions of performance measures around a few service rates, we get accurate estimates of various performance measures once the load is above 40% to 50%.

Sign in / Sign up

Export Citation Format

Share Document