Sequential Probability Ratio Tests: Conservative and Robust

2017 ◽  
Author(s):  
Jack P. C. Kleijnen ◽  
Wen Shi
Keyword(s):  
Probability Ratio ◽  
Sequential Probability Ratio Tests ◽  
Sequential Probability

scholarly journals Properties of Generalized Sequential Probability Ratio Tests

1976 ◽  
Vol 4 (2) ◽  
pp. 237-251 ◽  
Author(s):  
Bennett Eisenberg ◽  
B. K. Ghosh ◽  
Gordon Simons
Keyword(s):  
Probability Ratio ◽  
Sequential Probability Ratio Tests ◽  
Sequential Probability

Sequential probability ratio tests: conservative and robust

SIMULATION ◽  
2020 ◽  
Vol 97 (1) ◽  
pp. 33-43
Author(s):  
Jack P C Kleijnen ◽  
Wen Shi
Keyword(s):  
Error Rates ◽  
Normal Distributions ◽  
Probability Ratio ◽  
Monte Carlo Experiments ◽  
Actual Error ◽  
Sequential Probability Ratio Tests ◽  
Statistical Hypotheses ◽  
Sequential Probability ◽  
The Mean ◽  
Pilot Sample

Because computers (except for parallel computers) generate simulation outputs sequentially, we recommend sequential probability ratio tests (SPRTs) for the statistical analysis of these outputs. However, until now simulation analysts have ignored SPRTs. To change this situation, we review SPRTs for the simplest case; namely, the case of choosing between two hypothesized values for the mean simulation output. For this case, the classic SPRT of Wald (Wald A. Sequential tests of statistical hypotheses. Ann Math Stat 1945; 16: 117–186) allows general types of distribution, including normal distributions with known variances. A modification permits unknown variances that are estimated. Hall (Hall WJ. Some sequential analogs of Stein’s two-stage test. Biometrika 1962; 49: 367–378) developed a SPRT that assumes normal distributions with unknown variances estimated from a pilot sample. A modification uses a fully sequential variance estimator. In this paper, we quantify the performance of the various SPRTs, using several Monte Carlo experiments. In experiment #1, simulation outputs are normal. Whereas Wald’s SPRT with estimated variance gives too high error rates, Hall’s original and modified SPRTs are “conservative”; that is, the actual error rates are smaller than those prespecified (nominal). Furthermore, our experiment shows that the most efficient SPRT is Hall’s modified SPRT. In experiment #2, we estimate the robustness of these SPRTs for non-normal output. For these two experiments, we provide details on their design and analysis; these details may also be useful for simulation experiments in general.

Sign in / Sign up

Export Citation Format

Share Document