Fixed-Confidence, Fixed-Tolerance Guarantees for Ranking-and-Selection Procedures

2021 ◽  
Vol 31 (2) ◽  
pp. 1-33
Author(s):  
David J. Eckman ◽  
Shane G. Henderson
Keyword(s):  
Sufficient Conditions ◽  
Subset Selection ◽  
Ranking And Selection ◽  
Correct Selection ◽  
Selection Procedures ◽  
Indifference Zone ◽  
Second Best ◽  
Probability Of Correct Selection ◽  
Statistical Ranking ◽  
Ranking And Selection Procedures

Ever since the conception of the statistical ranking-and-selection (R8S) problem, a predominant approach has been the indifference-zone (IZ) formulation. Under the IZ formulation, R8S procedures are designed to provide a guarantee on the probability of correct selection (PCS) whenever the performance of the best system exceeds that of the second-best system by a specified amount. We discuss the shortcomings of this guarantee and argue that providing a guarantee on the probability of good selection (PGS)—selecting a system whose performance is within a specified tolerance of the best—is a more justifiable goal. Unfortunately, this form of fixed-confidence, fixed-tolerance guarantee has received far less attention within the simulation community. We present an overview of the PGS guarantee with the aim of reorienting the simulation community toward this goal. We examine numerous techniques for proving the PGS guarantee, including sufficient conditions under which selection and subset-selection procedures that deliver the IZ-inspired PCS guarantee also deliver the PGS guarantee.

Algorithm for Calculating the Initial Sample Size in a Fully Sequential Ranking and Selection Procedure

2020 ◽  
Vol 37 (03) ◽  
pp. 2050015
Author(s):  
Ruijing Wu ◽  
Shaoxuan Liu ◽  
Zhenyang Shi
Keyword(s):  
Sample Size ◽  
Selection Procedure ◽  
Total Sample ◽  
Ranking And Selection ◽  
Correct Selection ◽  
Initial Sample ◽  
Selection Procedures ◽  
Probability Of Correct Selection ◽  
Ranking And Selection Procedures ◽  
The Impact

In some fully sequential ranking and selection procedures, such as the KN procedure and Rinott’s procedure, some initial samples must be taken to estimate the variance. We analyze the impact of the initial sample size (ISS) on the total sample size and propose an algorithm to calculate the ISS in this type of procedure. To better illustrate our approach, we implement this algorithm on the KN procedure and propose the KN-ISS procedure. Comprehensive numerical experiments reveal that this procedure can significantly improve the efficiency compared with the KN procedure and still deliver the desired probability of correct selection.

scholarly journals An overview on selection methods

1982 ◽  
Vol 1 (2) ◽  
pp. 91-96 ◽  
Author(s):  
J. W. H. Swanepoel
Keyword(s):  
Major Part ◽  
Ranking And Selection ◽  
Regression Analyses ◽  
Selection Methods ◽  
Selection Procedures ◽  
Indifference Zone ◽  
Selection Problems ◽  
Fixed Sample ◽  
Sequential Procedures ◽  
Ranking And Selection Procedures

In many studies the experimenter has under consideration several (two or more) alternatives, and is studying them in order to determine which is the best (with regard to certain specified criteria of “goodness”). Such an experimenter does not wish basically to test hypotheses, or construct confidence intervals, or perform regression analyses (though these may be appropriate parts of his analysis); he does wish to select the best of several alternatives, and the major part of his analysis should therefore be directed towards this goal. It is precisely for this problem that ranking and selection procedures were developed. This paper presents an overview of some recent work in this field, with emphasis on aspects important to experimenters confronted with selection problems. Fixed sample size and sequential procedures for both the indifference zone and subset formulations of the selection problem are discussed.

Selecting the Best Alternative Based on Its Quantile

2020 ◽  
Author(s):  
Demet Batur ◽  
F. Fred Choobineh
Keyword(s):  
Value At Risk ◽  
Decision Makers ◽  
Correct Selection ◽  
Zone Size ◽  
Indifference Zone ◽  
Probability Of Correct Selection ◽  
A Value ◽  
Conscious Decision ◽  
Target Probability ◽  
Sequential Procedures

A value-at-risk, or quantile, is widely used as an appropriate investment selection measure for risk-conscious decision makers. We present two quantile-based sequential procedures—with and without consideration of equivalency between alternatives—for selecting the best alternative from a set of simulated alternatives. These procedures asymptotically guarantee a user-defined target probability of correct selection within a prespecified indifference zone. Experimental results demonstrate the trade-off between the indifference-zone size and the number of simulation iterations needed to render a correct selection while satisfying a desired probability of correct selection.

Sign in / Sign up

Export Citation Format

Share Document