Small sample sizes in the study of ontogenetic allometry; implications for palaeobiology

Williams and Bargh (2008) reported that holding a hot cup of coffee caused participants to judge a person’s personality as warmer, and that holding a therapeutic heat pad caused participants to choose rewards for other people rather than for themselves. These experiments featured large effects (r = .28 and .31), small sample sizes (41 and 53 participants), and barely statistically significant results. We attempted to replicate both experiments in field settings with more than triple the sample sizes (128 and 177) and double-blind procedures, but found near-zero effects (r = –.03 and .02). In both cases, Bayesian analyses suggest there is substantially more evidence for the null hypothesis of no effect than for the original physical warmth priming hypothesis.

Download Full-text

G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

Scientific Reports ◽

10.1038/s41598-021-81110-0 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Florent Le Borgne ◽

Arthur Chatton ◽

Maxime Léger ◽

Rémi Lenain ◽

Yohann Foucher

Keyword(s):

Machine Learning ◽

Support Vector Machine ◽

Statistical Power ◽

Small Sample ◽

Causal Effects ◽

Small Samples ◽

Support Vector ◽

Sample Sizes ◽

Super Learner ◽

Small Sample Sizes

AbstractIn clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

Download Full-text

What can we Learn from Studies Based on Small Sample Sizes? Comment on Regan, Lakhanpal, and Anguiano (2012)

Psychological Reports ◽

10.2466/21.02.07.pr0.113x12z8 ◽

2013 ◽

Vol 113 (1) ◽

pp. 221-224 ◽

Cited By ~ 3

Author(s):

David R. Johnson ◽

Lauren K. Bachan

Keyword(s):

Sample Size ◽

Reporting rigorous qualitative results: Moving beyond small sample sizes

Australian Occupational Therapy Journal ◽

10.1111/1440-1630.12475 ◽

2018 ◽

Vol 65 (2) ◽

pp. 77-78

Author(s):

Genevieve Pepin

Keyword(s):

Small Sample ◽

Sample Sizes ◽

Small Sample Sizes

Download Full-text

Implications of Small Samples for Generalization: Adjustments and Rules of Thumb

Evaluation Review ◽

10.1177/0193841x16655665 ◽

2016 ◽

Vol 41 (5) ◽

pp. 472-505 ◽

Cited By ~ 16

Author(s):

Elizabeth Tipton ◽

Kelly Hallberg ◽

Larry V. Hedges ◽

Wendy Chan

Keyword(s):

Observational Studies ◽

Small Sample ◽

Average Treatment Effect ◽

Small Samples ◽

Sample Sizes ◽

Random Samples ◽

Rules Of Thumb ◽

Large Populations ◽

Small Sample Sizes ◽

Combine Information

Background: Policy makers and researchers are frequently interested in understanding how effective a particular intervention may be for a specific population. One approach is to assess the degree of similarity between the sample in an experiment and the population. Another approach is to combine information from the experiment and the population to estimate the population average treatment effect (PATE). Method: Several methods for assessing the similarity between a sample and population currently exist as well as methods estimating the PATE. In this article, we investigate properties of six of these methods and statistics in the small sample sizes common in education research (i.e., 10–70 sites), evaluating the utility of rules of thumb developed from observational studies in the generalization case. Result: In small random samples, large differences between the sample and population can arise simply by chance and many of the statistics commonly used in generalization are a function of both sample size and the number of covariates being compared. The rules of thumb developed in observational studies (which are commonly applied in generalization) are much too conservative given the small sample sizes found in generalization. Conclusion: This article implies that sharp inferences to large populations from small experiments are difficult even with probability sampling. Features of random samples should be kept in mind when evaluating the extent to which results from experiments conducted on nonrandom samples might generalize.

Download Full-text

Small sample sizes in high-throughput miRNA screens: A common pitfall for the identification of miRNA biomarkers

Biomolecular Detection and Quantification ◽

10.1016/j.bdq.2017.11.002 ◽

2018 ◽

Vol 15 ◽

pp. 1-5 ◽

Cited By ~ 23

Author(s):

M.G.M. Kok ◽

M.W.J. de Ronde ◽

P.D. Moerland ◽

J.M. Ruijter ◽

E.E. Creemers ◽

...

Keyword(s):

High Throughput ◽

Small Sample ◽

Sample Sizes ◽

Common Pitfall ◽

Small Sample Sizes

Download Full-text

Variable selection in omics data: A practical evaluation of small sample sizes

PLoS ONE ◽

10.1371/journal.pone.0197910 ◽

2018 ◽

Vol 13 (6) ◽

pp. e0197910 ◽

Cited By ~ 17

Author(s):

Alexander Kirpich ◽

Elizabeth A. Ainsworth ◽

Jessica M. Wedow ◽

Jeremy R. B. Newman ◽

George Michailidis ◽

...

Keyword(s):

Variable Selection ◽

Small Sample ◽

Omics Data ◽

Sample Sizes ◽

Practical Evaluation ◽

Small Sample Sizes

Download Full-text

Sample Size Determination and Optimal Design of Simple Pretest-Posttest Experimental Designs: Introduction, Software, and Illustrations

10.35542/osf.io/k5ey8 ◽

2021 ◽

Author(s):

Metin Bulus

Keyword(s):

Optimal Design ◽

Sample Size ◽

Small Sample ◽

Experimental Designs ◽

Sample Size Determination ◽

Size Determination ◽

Sample Sizes ◽

Control Groups ◽

Small Sample Sizes ◽

And Control

A recent systematic review of experimental studies conducted in Turkey between 2010 and 2020 reported that small sample sizes had been a significant drawback (Bulus and Koyuncu, 2021). A small chunk of the studies were small-scale true experiments (subjects randomized into the treatment and control groups). The remaining studies consisted of quasi-experiments (subjects in treatment and control groups were matched on pretest or other covariates) and weak experiments (neither randomized nor matched but had the control group). They had an average sample size below 70 for different domains and outcomes. These small sample sizes imply a strong (and perhaps erroneous) assumption about the minimum relevant effect size (MRES) of intervention before an experiment is conducted; that is, a standardized intervention effect of Cohen’s d < 0.50 is not relevant to education policy or practice. Thus, an introduction to sample size determination for pretest-posttest simple experimental designs is warranted. This study describes nuts and bolts of sample size determination, derives expressions for optimal design under differential cost per treatment and control units, provide convenient tables to guide sample size decisions for MRES values between 0.20 ≤ Cohen’s d ≤ 0.50, and describe the relevant software along with illustrations.

Download Full-text