Detecting Fraudulent Erasures at an Aggregate Level

2017 ◽  
Vol 43 (3) ◽  
pp. 286-315 ◽  
Author(s):  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Null Distribution ◽  
Real Data ◽  
Error Rates ◽  
Standard Normal Distribution ◽  
Type I ◽  
Group Level ◽  
Nominal Level ◽  
Type I Error Rates ◽  
Standard Normal

Wollack, Cohen, and Eckerly suggested the “erasure detection index” (EDI) to detect fraudulent erasures for individual examinees. Wollack and Eckerly extended the EDI to detect fraudulent erasures at the group level. The EDI at the group level was found to be slightly conservative. This article suggests two modifications of the EDI for the group level. The asymptotic null distribution of the two modified indices is proved to be the standard normal distribution. In a simulation study, the modified indices are shown to have Type I error rates close to the nominal level and larger power than the index of Wollack and Eckerly. A real data example is also included.

scholarly journals Three New Methods for Analysis of Answer Changes

2016 ◽  
Vol 77 (1) ◽  
pp. 54-81 ◽  
Author(s):  
Sandip Sinharay ◽  
Matthew S. Johnson
Keyword(s):  
Normal Distribution ◽  
Null Hypothesis ◽  
Type I Error ◽  
Error Rates ◽  
Standard Normal Distribution ◽  
Type I ◽  
Data Set ◽  
Continuity Correction ◽  
Type I Error Rates ◽  
Standard Normal

In a pioneering research article, Wollack and colleagues suggested the “erasure detection index” (EDI) to detect test tampering. The EDI can be used with or without a continuity correction and is assumed to follow the standard normal distribution under the null hypothesis of no test tampering. When used without a continuity correction, the EDI often has inflated Type I error rates. When used with a continuity correction, the EDI has satisfactory Type I error rates, but smaller power compared with the EDI without a continuity correction. This article suggests three methods for detecting test tampering that do not rely on the assumption of a standard normal distribution under the null hypothesis. It is demonstrated in a detailed simulation study that the performance of each suggested method is slightly better than that of the EDI. The EDI and the suggested methods were applied to a real data set. The suggested methods, although more computation intensive than the EDI, seem to be promising in detecting test tampering.

scholarly journals A comparative analysis of cell-type adjustment methods for epigenome-wide association studies based on simulated and real data sets

2018 ◽  
Vol 20 (6) ◽  
pp. 2055-2065 ◽  
Author(s):  
Johannes Brägelmann ◽  
Justo Lorenzo Bermejo
Keyword(s):  
Statistical Power ◽  
Type I Error ◽  
Association Studies ◽  
Real Data ◽  
Error Rates ◽  
Data Sets ◽  
Type I ◽  
Cell Type ◽  
Type I Error Rates

Abstract Technological advances and reduced costs of high-density methylation arrays have led to an increasing number of association studies on the possible relationship between human disease and epigenetic variability. DNA samples from peripheral blood or other tissue types are analyzed in epigenome-wide association studies (EWAS) to detect methylation differences related to a particular phenotype. Since information on the cell-type composition of the sample is generally not available and methylation profiles are cell-type specific, statistical methods have been developed for adjustment of cell-type heterogeneity in EWAS. In this study we systematically compared five popular adjustment methods: the factored spectrally transformed linear mixed model (FaST-LMM-EWASher), the sparse principal component analysis algorithm ReFACTor, surrogate variable analysis (SVA), independent SVA (ISVA) and an optimized version of SVA (SmartSVA). We used real data and applied a multilayered simulation framework to assess the type I error rate, the statistical power and the quality of estimated methylation differences according to major study characteristics. While all five adjustment methods improved false-positive rates compared with unadjusted analyses, FaST-LMM-EWASher resulted in the lowest type I error rate at the expense of low statistical power. SVA efficiently corrected for cell-type heterogeneity in EWAS up to 200 cases and 200 controls, but did not control type I error rates in larger studies. Results based on real data sets confirmed simulation findings with the strongest control of type I error rates by FaST-LMM-EWASher and SmartSVA. Overall, ReFACTor, ISVA and SmartSVA showed the best comparable statistical power, quality of estimated methylation differences and runtime.

Type I Error Rates and Power Estimates for Selected Two-Sample Tests of Scale

1989 ◽  
Vol 14 (4) ◽  
pp. 373-384 ◽  
Author(s):  
James Algina ◽  
Stephen Olejnik ◽  
Romer Ocanto
Keyword(s):  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Sample Sizes ◽  
Nominal Level ◽  
Nonnormal Distributions ◽  
Group Design ◽  
Type I Error Rates

Estimated Type I error rates and power are reported for a modified Fligner-Killeen test, O’Brien’s test, O’Brien’s test using Welch’s modified ANOVA, the Brown-Forsythe test, and two tests developed by Tiku. Normal and nonnormal distributions and a two-group design were investigated. O’Brien’s test and the Brown-Forsythe test had estimated Type I error rates near the nominal level for all conditions investigated. To maximize power when the sample sizes are equal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe test with leptokurtic distributions. Either test can be used when the kurtosis is zero. When the sample sizes are unequal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe with symmetric-leptokurtic distributions. With other distributions, the tests have similar power.

scholarly journals Compound Bias due to Measurement Error When Comparing Regression Coefficients

2019 ◽  
Vol 80 (3) ◽  
pp. 548-577
Author(s):  
William M. Murrah
Keyword(s):  
Measurement Error ◽  
Multiple Regression ◽  
Simulation Study ◽  
Type I Error ◽  
Real Data ◽  
Error Rates ◽  
Regression Coefficients ◽  
Type I ◽  
Type I Error Rates ◽  
The Impact

Multiple regression is often used to compare the importance of two or more predictors. When the predictors being compared are measured with error, the estimated coefficients can be biased and Type I error rates can be inflated. This study explores the impact of measurement error on comparing predictors when one is measured with error, followed by a simulation study to help quantify the bias and Type I error rates for common research situations. Two methods used to adjust for measurement error are demonstrated using a real data example. This study adds to the literature documenting the impact of measurement error on regression modeling, identifying issues particular to the use of multiple regression for comparing predictors, and offers recommendations for researchers conducting such studies.

scholarly journals Detection of Item Preknowledge Using Response Times

2020 ◽  
Vol 44 (5) ◽  
pp. 376-392
Author(s):  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Response Times ◽  
Real Data ◽  
Standard Normal Distribution ◽  
Major Type ◽  
Type I ◽  
Standard Normal ◽  
The Difference ◽  
Item Scores ◽  
Item Preknowledge

Benefiting from item preknowledge is a major type of fraudulent behavior during educational assessments. This article suggests a new statistic that can be used for detecting the examinees who may have benefited from item preknowledge using their response times. The statistic quantifies the difference in speed between the compromised items and the non-compromised items of the examinees. The distribution of the statistic under the null hypothesis of no preknowledge is proved to be the standard normal distribution. A simulation study is used to evaluate the Type I error rate and power of the suggested statistic. A real data example demonstrates the usefulness of the new statistic that is found to provide information that is not provided by statistics based only on item scores.

A Maximum Test for Scale: Type I Error Rates and Power

1995 ◽  
Vol 20 (1) ◽  
pp. 27-39 ◽  
Author(s):  
James Algina ◽  
R. Clifford Blair ◽  
William T. Coombs
Keyword(s):  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Test Statistics ◽  
Test Statistic ◽  
Nominal Level ◽  
Scale Type ◽  
Type I Error Rates ◽  
Maximum Test ◽  
Study Type

A maximum test in which the test statistic is the more extreme of the Brown-Forsythe and O’Brien’s test statistics is developed. Estimated Type I error rates and power are presented for the Brown-Forsythe test, O’Brien’s test, and the maximum test. For the conditions included in the study, Type I error rates for the maximum test are near the nominal level. In all conditions, the power of the maximum test tended to be equal to or greater than that of the test—O’Brien or Brown-Forsythe—that had the larger power.

The Use of Theory of Linear Mixed-Effects Models to Detect Fraudulent Erasures at an Aggregate Level

2021 ◽  
pp. 001316442199489
Author(s):  
Luyao Peng ◽  
Sandip Sinharay
Keyword(s):  
Type I Error ◽  
Real Data ◽  
Mixed Effects ◽  
Error Rates ◽  
Mixed Effects Models ◽  
Type I ◽  
Aggregate Level ◽  
Linear Mixed Effects Models ◽  
Linear Mixed Effects ◽  
Best Linear Unbiased

Wollack et al. (2015) suggested the erasure detection index (EDI) for detecting fraudulent erasures for individual examinees. Wollack and Eckerly (2017) and Sinharay (2018) extended the index of Wollack et al. (2015) to suggest three EDIs for detecting fraudulent erasures at the aggregate or group level. This article follows up on the research of Wollack and Eckerly (2017) and Sinharay (2018) and suggests a new aggregate-level EDI by incorporating the empirical best linear unbiased predictor from the literature of linear mixed-effects models (e.g., McCulloch et al., 2008). A simulation study shows that the new EDI has larger power than the indices of Wollack and Eckerly (2017) and Sinharay (2018). In addition, the new index has satisfactory Type I error rates. A real data example is also included.

Sign in / Sign up

Export Citation Format

Share Document