scholarly journals A Monte Carlo Simulation Study for Kolmogorov-Smirnov Two-Sample Test Under the Precondition of Heterogeneity: Upon the Changes on the Probabilities of Statistical Power and Type I Error Rates with Respect to Skewness Measure

2013 ◽  
Author(s):  
ttken Senger ◽  
Ali Kemal elik
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Statistical Power ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Monte Carlo Simulation Study ◽  
Type I Error Rates ◽  
Sample Test ◽  
Kolmogorov Smirnov

Comparison of methods to account for autocorrelation in correlation analyses of fish data

1998 ◽  
Vol 55 (9) ◽  
pp. 2127-2140 ◽  
Author(s):  
Brian J Pyper ◽  
Randall M Peterman
Keyword(s):  
Monte Carlo ◽  
Hypothesis Testing ◽  
Type I Error ◽  
Low Frequency ◽  
Error Rates ◽  
Type I ◽  
Testing Procedures ◽  
Type I Error Rates ◽  
Fish Recruitment ◽  
Correlation Analyses

Autocorrelation in fish recruitment and environmental data can complicate statistical inference in correlation analyses. To address this problem, researchers often either adjust hypothesis testing procedures (e.g., adjust degrees of freedom) to account for autocorrelation or remove the autocorrelation using prewhitening or first-differencing before analysis. However, the effectiveness of methods that adjust hypothesis testing procedures has not yet been fully explored quantitatively. We therefore compared several adjustment methods via Monte Carlo simulation and found that a modified version of these methods kept Type I error rates near . In contrast, methods that remove autocorrelation control Type I error rates well but may in some circumstances increase Type II error rates (probability of failing to detect some environmental effect) and hence reduce statistical power, in comparison with adjusting the test procedure. Specifically, our Monte Carlo simulations show that prewhitening and especially first-differencing decrease power in the common situations where low-frequency (slowly changing) processes are important sources of covariation in fish recruitment or in environmental variables. Conversely, removing autocorrelation can increase power when low-frequency processes account for only some of the covariation. We therefore recommend that researchers carefully consider the importance of different time scales of variability when analyzing autocorrelated data.

scholarly journals A comparative analysis of cell-type adjustment methods for epigenome-wide association studies based on simulated and real data sets

2018 ◽  
Vol 20 (6) ◽  
pp. 2055-2065 ◽  
Author(s):  
Johannes Brägelmann ◽  
Justo Lorenzo Bermejo
Keyword(s):  
Statistical Power ◽  
Type I Error ◽  
Association Studies ◽  
Real Data ◽  
Error Rates ◽  
Data Sets ◽  
Type I ◽  
Cell Type ◽  
Type I Error Rates

Abstract Technological advances and reduced costs of high-density methylation arrays have led to an increasing number of association studies on the possible relationship between human disease and epigenetic variability. DNA samples from peripheral blood or other tissue types are analyzed in epigenome-wide association studies (EWAS) to detect methylation differences related to a particular phenotype. Since information on the cell-type composition of the sample is generally not available and methylation profiles are cell-type specific, statistical methods have been developed for adjustment of cell-type heterogeneity in EWAS. In this study we systematically compared five popular adjustment methods: the factored spectrally transformed linear mixed model (FaST-LMM-EWASher), the sparse principal component analysis algorithm ReFACTor, surrogate variable analysis (SVA), independent SVA (ISVA) and an optimized version of SVA (SmartSVA). We used real data and applied a multilayered simulation framework to assess the type I error rate, the statistical power and the quality of estimated methylation differences according to major study characteristics. While all five adjustment methods improved false-positive rates compared with unadjusted analyses, FaST-LMM-EWASher resulted in the lowest type I error rate at the expense of low statistical power. SVA efficiently corrected for cell-type heterogeneity in EWAS up to 200 cases and 200 controls, but did not control type I error rates in larger studies. Results based on real data sets confirmed simulation findings with the strongest control of type I error rates by FaST-LMM-EWASher and SmartSVA. Overall, ReFACTor, ISVA and SmartSVA showed the best comparable statistical power, quality of estimated methylation differences and runtime.

Summarizing Monte Carlo Results in Methodological Research

1992 ◽  
Vol 17 (4) ◽  
pp. 297-313 ◽  
Author(s):  
Michael R. Harwell
Keyword(s):  
Monte Carlo ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rates ◽  
Power Of A Test ◽  
Methodological Research ◽  
Assumption Violations ◽  
Monte Carlo Studies ◽  
Effective Use

Monte Carlo studies provide information that can assist researchers in selecting a statistical test when underlying assumptions of the test are violated. Effective use of this literature is hampered by the lack of an overarching theory to guide the interpretation of Monte Carlo studies. The problem is exacerbated by the impressionistic nature of the studies, which can lead different readers to different conclusions. These shortcomings can be addressed using meta-analytic methods to integrate the results of Monte Carlo studies. Quantitative summaries of the effects of assumption violations on the Type I error rate and power of a test can assist researchers in selecting the best test for their data. Such summaries can also be used to evaluate the validity of previously published statistical results. This article provides a methodological framework for quantitatively integrating Type I error rates and power values for Monte Carlo studies. An example is provided using Monte Carlo studies of Bartlett’s (1937) test of equality of variances. The importance of relating meta-analytic results to exact statistical theory is emphasized.

Performance of Monte Carlo Permutation and Approximate Tests for Multivariate Means Comparisons With Small Sample Sizes When Parametric Assumptions are Violated

Methodology ◽  
2009 ◽  
Vol 5 (2) ◽  
pp. 60-70 ◽  
Author(s):  
W. Holmes Finch ◽  
Teresa Davenport
Keyword(s):  
Monte Carlo ◽  
Type I Error ◽  
Permutation Tests ◽  
Error Rates ◽  
Covariance Matrices ◽  
Small Sample ◽  
Type I ◽  
Permutation Testing ◽  
Sample Sizes ◽  
Type I Error Rates

Permutation testing has been suggested as an alternative to the standard F approximate tests used in multivariate analysis of variance (MANOVA). These approximate tests, such as Wilks’ Lambda and Pillai’s Trace, have been shown to perform poorly when assumptions of normally distributed dependent variables and homogeneity of group covariance matrices were violated. Because Monte Carlo permutation tests do not rely on distributional assumptions, they may be expected to work better than their approximate cousins when the data do not conform to the assumptions described above. The current simulation study compared the performance of four standard MANOVA test statistics with their Monte Carlo permutation-based counterparts under a variety of conditions with small samples, including conditions when the assumptions were met and when they were not. Results suggest that for sample sizes of 50 subjects, power is very low for all the statistics. In addition, Type I error rates for both the approximate F and Monte Carlo tests were inflated under the condition of nonnormal data and unequal covariance matrices. In general, the performance of the Monte Carlo permutation tests was slightly better in terms of Type I error rates and power when both assumptions of normality and homogeneous covariance matrices were not met. It should be noted that these simulations were based upon the case with three groups only, and as such results presented in this study can only be generalized to similar situations.

Silverstein's Nonparametric Many-One Test for a One-Way Design: A Monte Carlo Study

1981 ◽  
Vol 48 (1) ◽  
pp. 19-22 ◽  
Author(s):  
James D. Church ◽  
Edward L. Wike
Keyword(s):  
Monte Carlo ◽  
Type I Error ◽  
Monte Carlo Study ◽  
Error Rates ◽  
Wilcoxon Test ◽  
Type I ◽  
Type I Error Rates ◽  
Wilcoxon Rank Sum Test

A Monte Carlo study was done to find the Type I error rates for three nonparametric procedures for making k − 1 many-one comparisons in a one-way design. The tests ( t) were the Silverstein and Steel many-one ranks tests and the two-sample Wilcoxon rank-sum test, k = 3, 5, 7, and 10 treatments were crossed with n = 7, 10, and 15 replicates with 1000 simulations per k, n combination. Analyses of four Type I error rates showed that: (1) The Wilcoxon test had the best comparisonwise error rates; (2) none of the tests functioned well as protected tests; and (3) the Silverstein test had the best experimentwise error rates and was the recommended procedure for many-one tests for a one-way layout.

scholarly journals A Monte Carlo Study of an Iterative Wald Test Procedure for DIF Analysis

2016 ◽  
Vol 77 (1) ◽  
pp. 104-118 ◽  
Author(s):  
Mengyang Cao ◽  
Louis Tay ◽  
Yaowu Liu
Keyword(s):  
Monte Carlo ◽  
Type I Error ◽  
Test Procedure ◽  
Monte Carlo Study ◽  
Error Rates ◽  
Type I ◽  
Iterative Approach ◽  
Response Options ◽  
Type I Error Rates ◽  
Dichotomous Data

This study examined the performance of a proposed iterative Wald approach for detecting differential item functioning (DIF) between two groups when preknowledge of anchor items is absent. The iterative approach utilizes the Wald-2 approach to identify anchor items and then iteratively tests for DIF items with the Wald-1 approach. Monte Carlo simulation was conducted across several conditions including the number of response options, test length, sample size, percentage of DIF items, DIF effect size, and type of cumulative DIF. Results indicated that the iterative approach performed well for polytomous data in all conditions, with well-controlled Type I error rates and high power. For dichotomous data, the iterative approach also exhibited better control over Type I error rates than the Wald-2 approach without sacrificing the power in detecting DIF. However, inflated Type I error rates were found for the iterative approach in conditions with dichotomous data, noncompensatory DIF, large percentage of DIF items, and medium to large DIF effect sizes. Nevertheless, the Type I error rates were substantially less inflated in those conditions compared with the Wald-2 approach.

Using Monte Carlo Software to Teach Abstract Statistical Concepts: A Case Study

2005 ◽  
Vol 32 (3) ◽  
pp. 193-195 ◽  
Author(s):  
Holly Raffle ◽  
Gordon P. Brooks
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Type I Error ◽  
Computer Software ◽  
Course Evaluation ◽  
Error Rates ◽  
Type I ◽  
Statistics Course ◽  
Instructor's Manual

Violations of assumptions, inflated Type I error rates, and robustness are important concepts for students to learn in an introductory statistics course. However, these abstract ideas can be difficult for students to understand. Monte Carlo simulation methods can provide a concrete way for students to learn abstract statistical concepts. This article describes the MC4G computer software (Brooks, 2004) and the accompanying instructor's manual (Raffle, 2004). It also provides a case study that includes both assessment and course evaluation data supporting the effectiveness of Monte Carlo simulation exercises in a graduate-level statistics course.

Sign in / Sign up

Export Citation Format

Share Document