scholarly journals An old problem in the spotlight again: The mistaken practice of the null-hypothesis significance test

2016 ◽  
Vol 94 (1) ◽  
pp. 52-75 ◽  
Author(s):  
Anna Bárdits ◽  
Renáta Németh ◽  
Győző Terplán
Keyword(s):  
Null Hypothesis ◽  
Significance Test ◽  
Null Hypothesis Significance Test

The null-hypothesis significance-test procedure is still warranted

1998 ◽  
Vol 21 (2) ◽  
pp. 228-235 ◽  
Author(s):  
Siu L. Chow
Keyword(s):  
Null Hypothesis ◽  
Statistical Power ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Test Procedure ◽  
Significance Test ◽  
Null Hypothesis Significance Test ◽  
Wide Range ◽  
Statistical Hypotheses ◽  
Alternative Hypotheses

Entertaining diverse assumptions about empirical research, commentators give a wide range of verdicts on the NHSTP defence in Statistical significance. The null-hypothesis significance-test procedure (NHSTP) is defended in a framework in which deductive and inductive rules are deployed in theory corroboration in the spirit of Popper's Conjectures and refutations (1968b). The defensible hypothetico-deductive structure of the framework is used to make explicit the distinctions between (1) substantive and statistical hypotheses, (2) statistical alternative and conceptual alternative hypotheses, and (3) making statistical decisions and drawing theoretical conclusions. These distinctions make it easier to show that (1) H0 can be true, (2) the effect size is irrelevant to theory corroboration, and (3) “strong” hypotheses make no difference to NHSTP. Reservations about statistical power, meta-analysis, and the Bayesian approach are still warranted.

The Ouroboros of Psychological Methodology: The Case of Effect Sizes (Mechanical Objectivity vs. Expertise)

2018 ◽  
Vol 22 (4) ◽  
pp. 469-476 ◽  
Author(s):  
Ian J. Davidson
Keyword(s):  
Effect Size ◽  
Null Hypothesis ◽  
Significance Test ◽  
Effect Sizes ◽  
Circular Path ◽  
Bottom Up ◽  
Psychological Science ◽  
Null Hypothesis Significance Test ◽  
Theoretical Tool ◽  
Mechanical Objectivity

The reporting and interpretation of effect sizes is often promoted as a panacea for the ramifications of institutionalized statistical rituals associated with the null-hypothesis significance test. Mechanical objectivity—conflating the use of a method with the obtainment of truth—is a useful theoretical tool for understanding the possible failure of effect size reporting ( Porter, 1995 ). This article helps elucidate the ouroboros of psychological methodology. This is the cycle of improved tools to produce trustworthy knowledge, leading to their institutionalization and adoption as forms of thinking, leading to methodologists eventually admonishing researchers for relying too heavily on rituals, finally leading to the production of more new improved quantitative tools that may follow along this circular path. Despite many critiques and warnings, research psychologists’ superficial adoption of effect sizes might preclude expert interpretation much like in the null-hypothesis significance test as widely received. One solution to this situation is bottom-up: promoting a balance of mechanical objectivity and expertise in the teaching of methods and research. This would require the acceptance and encouragement of expert interpretation within psychological science.

Meta-analysis, power analysis, and the null-hypothesis significance-test procedure

1998 ◽  
Vol 21 (2) ◽  
pp. 216-217 ◽  
Author(s):  
Joseph S. Rossi
Keyword(s):  
Null Hypothesis ◽  
Power Analysis ◽  
Meta Analysis ◽  
Verbal Learning ◽  
Test Procedure ◽  
Significance Test ◽  
Size Estimation ◽  
Null Hypothesis Significance Test ◽  
Effect Size Estimation ◽  
History Of

Chow's (1996) defense of the null-hypothesis significance-test procedure (NHSTP) is thoughtful and compelling in many respects. Nevertheless, techniques such as meta-analysis, power analysis, effect size estimation, and confidence intervals can be useful supplements to NHSTP in furthering the cumulative nature of behavioral research, as illustrated by the history of research on the spontaneous recovery of verbal learning.

Significance tests cannot be justified in theory-corroboration experiments

1998 ◽  
Vol 21 (2) ◽  
pp. 213-213
Author(s):  
Marks R. Nester
Keyword(s):  
Null Hypothesis ◽  
Test Procedure ◽  
Significance Test ◽  
Effect Sizes ◽  
Standard Errors ◽  
Confidence Limits ◽  
Significance Tests ◽  
Null Hypothesis Significance Test

Chow's one-tailed null-hypothesis significance-test procedure, with its rationale based on the elimination of chance influences, is not appropriate for theory-corroboration experiments. Estimated effect sizes and their associated standard errors or confidence limits will always suffice.

scholarly journals Overview on the Null Hypothesis Significance Test

2021 ◽  
Author(s):  
Noah N'Djaye Nikolai van Dongen ◽  
Leonie van Grootel
Keyword(s):  
Systematic Review ◽  
Null Hypothesis ◽  
Significance Test ◽  
Ongoing Debate ◽  
Thematic Synthesis ◽  
Null Hypothesis Significance Test ◽  
Problems And Solutions ◽  
Replication Crisis

For decades, waxing and waning, there has been an ongoing debate on the values and problems of the ubiquitously used null hypothesis significance test (NHST). With the start of the replication crisis, this debate has flared-up once again, especially in the psychology and psychological methods literature. Arguing for or against the NHST method usually takes place in essay and opinion pieces that cover some, but not all the qualities and problems of the method. The NHST literature landscape is vast, a clear overview is lacking, and participants in the debate seem to be talking past one another. To contribute to a resolution, we conducted a systematic review on essay literature concerning NHST published in psychology and psychological methods journals between 2011 and 2018. We extracted all arguments in defense of (20) and against (70) NHST, and we extracted the solutions (33) that were proposed to remedy (some of) the perceived problems of NHST. Unfiltered, these 123 items form a landscape that is prohibitively difficult to keep in one’s sights. Our contribution to the resolution of the NHST debate is twofold. 1) We performed a thematic synthesis of the arguments and solutions that carves the landscape in a framework of three zones: mild, moderate, and critical. This reduction summarizes groups of arguments and solutions, thus offering a manageable overview of NHST’s qualities, problems, and solutions. 2) We provide the data on the arguments and solutions as a resource for those who will carry-on the debate and/or study the use of NHST.

The null-hypothesis significance-test procedure: Can't live with it, can't live without it

1998 ◽  
Vol 21 (2) ◽  
pp. 194-195
Author(s):  
Charles F. Blaich
Keyword(s):  
Null Hypothesis ◽  
Test Procedure ◽  
Significance Test ◽  
Null Hypothesis Significance Test

If the NHSTP procedure is essential for controlling for chance, why is there little, if any, discussion of the nature of chance by Chow and other advocates of the procedure. Also, many criticisms that Chow takes to be aimed against the NHSTP (null-hypothesis significance-test) procedure are actually directed against the kind of theory that is tested by the procedure.

Sign in / Sign up

Export Citation Format

Share Document