Low power, often the result of an insufficient number of participants, contributes to replication failures (e.g. Button et al., 2013). It is usually recommended that studies have at least an 80% chance of identifying an effect. However, following Cohen’s (1962) work, many reviews have reported that studies are under-powered. We calculated the power to detect small, medium and large effects of a test from the first study in all relevant Psychonomic Society journal articles published in 2017. Overall, mean power to detect medium sized effects was only 65%, and two-thirds of papers failed to achieve 80% power. Nearly half of papers in Psychonomic Bulletin & Review and Memory & Cognition achieved the 80% criteria, but other journals were much lower. The mean power to detect small effects was 22%, indicating that they would almost always be missed. Larger samples are generally needed to avoiding missing real effects.
The Challenge of the Psychonomic Society Guidelines on Statistical Issues (2012)