REQUIRED SAMPLE SIZE FOR COMPARING PROPORTIONS IN TWO INDEPENDENT SAMPLES

This paper continues our series of articles on required sample size for the most common basic statistical tests used in biomedical research. Sample size calculations are rarely performed in research planning in Russia often resulting in Type II errors, i.e. on acceptance on false null hypothesis due to insufficient sample size. The most common statistical test for analyzing proportions in independent samples is Pearson’s chi-squared test. In this paper we present a simple algorithm for calculating required sample size for comparing two independent proportions. In addition to manual calculations we present a step-by-step guide on how to use WinPepi and Stata software for calculating sample size for independent proportions. In addition, we present a table for junior researchers with already calculated sample sizes for comparing proportions from 0,1 to 0,9 by 0,1 with 95% confidence level and 80% statistical power.

Required sample size for comparing two independent means

Sample size calculation in a planning phase is still uncommon in Russian research practice. This situation threatens validity of the conclusions and may introduce Type I error when the false null hypothesis is accepted due to lack of statistical power to detect the existing difference between the means. Comparing two means using unpaired Students’ ttests is the most common statistical procedure in the Russian biomedical literature. However, calculations of the minimal required sample size or retrospective calculation of the statistical power were observed only in very few publications. In this paper we demonstrate how to calculate required sample size for comparing means in unpaired samples using WinPepi and Stata software. In addition, we produced tables for minimal required sample size for studies when two means have to be compared and body mass index and blood pressure are the variables of interest. The tables were constructed for unpaired samples for different levels of statistical power and standard deviations obtained from the literature.

Should forest regeneration studies have more replications?

When it comes to testing for differences in seedling survival, researchers sometimes make a Type II statistical error (i.e. failure to reject a false null hypothesis) due to the inherent variability associated with survival in tree planting studies. For example, in one trial (with five replications) first-year survival of seedlings planted in October (42%) was not significantly different (alpha = 0.05) from those planted in December (69%). Did planting in a dry October truly have no effect on survival? Authors who make a Type II error might not be aware that as seedling survival decreases (down to an overall average of 50% survival), statistical power declines. As a result, the ability to declare an 8% difference as “significant” is very difficult when survival averages 90% or less. We estimate that about half of regeneration trials (average survival of pines <90%) cannot declare a 12% difference as statistically significant (alpha = 0.05). When researchers realize their tree planting trials have low statistical power, they should consider using more replications. Other ways to increase power include: (1) use a one-tailed test (2) use a potentially more powerful contrast test (instead of an overall treatment F-test) and (3) conduct survival trials under a roof.

Wine, Women, Men, and Type II Error

More than forty published works show that women and men differ in their taste preferences for sweet, salt, sour, bitter, fruit, and other flavors. Despite those differences, dozens of state fair and other wine competitions determine winners' ribbons, medals, scores, and ranks by pooling the opinions of female and male judges. This article examines twenty-three blind wine tastings during which female and male judges scored more than nine hundred wines. Two-sample t-test results show that the gender-specific distributions of scores do have similar means and standard deviations. Exact p-values for two-sample chi-square tests show that the distributions of men's and women's scores are not significantly different, and exact p-values for likelihood ratio tests of Plackett-Luce model results show that the genders' preference orders are not significantly different. The correlation coefficient between women's and men's scores is weakly positive in 90 percent of the tastings. On that evidence, indications that the genders prefer different wines are difficult to detect. If such differences do exist, as the nonwine literature implies, the results of this analysis show that those differences are small compared to non-gender-related idiosyncratic differences between individuals and random expressions of preference. The potential for accept-a-false-null-hypothesis Type II error when pooling female and male judges' wine-related opinions appears to be small. (JEL Classifications: A10, C10, C00, C12, D12)

Using Bootstrap Method to Evaluate the Power of Tests for Non-Linearity in Asymmetric Price Relationship

This paper introduces and applies the bootstrap method to compare the power of the test for asymmetry in the Granger and Lee (1989) and Von Cramon-Taubadel and Loy (1996) models. The results of the bootstrap simulations indicate that the power of the test for asymmetry depends on various conditions such as the bootstrap sample size, model complexity, difference in adjustment speeds and the amount of noise in the data generating process used in the application. The true model achieves greater power when compared with the complex model. With small bootstrap sample size or large noise, both models display low power in rejecting the (false) null hypothesis of symmetry.

Problems Resulting from Multiple Tests

The consequences of multiple tests within a study have received much attention in recent years. In spite of many detailed considerations, there remains controversy concerning the seriousness of these consequences.To appreciate fully the arguments that abound in this controversy, it is necessary to be aware of several different definitions of errors that can occur in hypothesis testing. The definitions of Type I error (rejection of a true null hypothesis) and Type II error (failure to reject a false null hypothesis) remain basic. Consider initially the Type I error, which receives the major emphasis of arguments concerning the effects of multiple testing. Its usual definition is based on the fact that for a single variable and a single comparison, there is a probability (α) that the hypothesis of no effect erroneously can be rejected. If all studies consisted of only one variable and if only one comparison were of interest (for example, only two treatments or two groups were studied), there would be no multiple testing controversy. Of course, few, if any, studies are so limited in their intent. Thus, the consequences of multiple testing apply to nearly all studies.