Problems Resulting from Multiple Tests

1991 ◽  
Vol 12 (6) ◽  
pp. 373-374
Author(s):  
Leon F. Burmeister
Keyword(s):  
Null Hypothesis ◽  
Multiple Testing ◽  
Type I Error ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Usual Definition ◽  
Multiple Tests ◽  
Major Emphasis ◽  
False Null Hypothesis

The consequences of multiple tests within a study have received much attention in recent years. In spite of many detailed considerations, there remains controversy concerning the seriousness of these consequences.To appreciate fully the arguments that abound in this controversy, it is necessary to be aware of several different definitions of errors that can occur in hypothesis testing. The definitions of Type I error (rejection of a true null hypothesis) and Type II error (failure to reject a false null hypothesis) remain basic. Consider initially the Type I error, which receives the major emphasis of arguments concerning the effects of multiple testing. Its usual definition is based on the fact that for a single variable and a single comparison, there is a probability (α) that the hypothesis of no effect erroneously can be rejected. If all studies consisted of only one variable and if only one comparison were of interest (for example, only two treatments or two groups were studied), there would be no multiple testing controversy. Of course, few, if any, studies are so limited in their intent. Thus, the consequences of multiple testing apply to nearly all studies.

When Studies are in Error: Basic Statistical Vocabulary Needed to Understand Clinical Studies

1996 ◽  
Vol 1 (1) ◽  
pp. 25-28 ◽  
Author(s):  
Martin A. Weinstock
Keyword(s):  
Null Hypothesis ◽  
Statistical Power ◽  
Critical Appraisal ◽  
Type I Error ◽  
Statistical Significance ◽  
P Value ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Error Type

Background: Accurate understanding of certain basic statistical terms and principles is key to critical appraisal of published literature. Objective: This review describes type I error, type II error, null hypothesis, p value, statistical significance, a, two-tailed and one-tailed tests, effect size, alternate hypothesis, statistical power, β, publication bias, confidence interval, standard error, and standard deviation, while including examples from reports of dermatologic studies. Conclusion: The application of the results of published studies to individual patients should be informed by an understanding of certain basic statistical concepts.

scholarly journals Első- és másodfajú etikai kudarcok (Type I and type II ethical errors)

2012 ◽  
pp. 56-63
Author(s):  
Zsuzsanna Győri
Keyword(s):  
Common Good ◽  
Null Hypothesis ◽  
Type I Error ◽  
Holistic Approach ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Opportunistic Behaviour ◽  
Self Interest ◽  
The Government

A cikkben a szerző a piac és a kormányzat kudarcaiból kiindulva azonosítja a közjó elérését célzó harmadik rendszer, az etikai felelősség kudarcait. Statisztikai analógiát használva elsőfajú kudarcként azonosítja, mikor az etikát nem veszik figyelembe, pedig szükség van rá. Ugyanakkor másodfajú kudarcként kezeli az etika profitnövelést célzó használatát, mely megtéveszti az érintetteteket, így még szélesebb utat enged az opportunista üzleti tevékenységnek. Meglátása szerint a három rendszer egymást nemcsak kiegészíti, de kölcsönösen korrigálja is. Ez az elsőfajú kudarc esetében általánosabb, a másodfajú kudarc megoldásához azonban a gazdasági élet alapvetéseinek átfogalmazására, az önérdek és az egydimenziós teljesítményértékelés helyett egy új, holisztikusabb szemléletű közgazdaságra van szükség. _______ In the article the author identifies the errors of ethical responsibility. That is the third system to attain common good, but have similar failures like the other two: the hands of the market and the government. Using statistical analogy the author identifies Type I error when ethics are not considered but it should be (null hypothesis is rejected however it’s true). She treats the usage of ethics to extend profit as Type II error. This misleads the stakeholders and makes room for opportunistic behaviour in business (null hypothesis is accepted in turn it’s false). In her opinion the three systems: the hand of the market, the government and the ethical management not only amend but interdependently correct each other. In the case of Type I error it is more general. Nevertheless to solve the Type II error we have to redefine the core principles of business. We need a more holistic approach in economics instead of self-interest and one-dimensional interpretation of value.

scholarly journals Analysis of type I and II error rates of Bayesian and frequentist parametric and nonparametric two-sample hypothesis tests under preliminary assessment of normality

2020 ◽  
Author(s):  
Riko Kelter
Keyword(s):  
Null Hypothesis ◽  
Error Control ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Preliminary Assessment ◽  
Type I Error Rates ◽  
Practical Research

Abstract Testing for differences between two groups is among the most frequently carried out statistical methods in empirical research. The traditional frequentist approach is to make use of null hypothesis significance tests which use p values to reject a null hypothesis. Recently, a lot of research has emerged which proposes Bayesian versions of the most common parametric and nonparametric frequentist two-sample tests. These proposals include Student’s two-sample t-test and its nonparametric counterpart, the Mann–Whitney U test. In this paper, the underlying assumptions, models and their implications for practical research of recently proposed Bayesian two-sample tests are explored and contrasted with the frequentist solutions. An extensive simulation study is provided, the results of which demonstrate that the proposed Bayesian tests achieve better type I error control at slightly increased type II error rates. These results are important, because balancing the type I and II errors is a crucial goal in a variety of research, and shifting towards the Bayesian two-sample tests while simultaneously increasing the sample size yields smaller type I error rates. What is more, the results highlight that the differences in type II error rates between frequentist and Bayesian two-sample tests depend on the magnitude of the underlying effect.

A Forum for Researchers: A Note on the Power of Statistical Tests

1977 ◽  
Vol 8 (5) ◽  
pp. 385-389
Author(s):  
Robert C. Clark
Keyword(s):  
Null Hypothesis ◽  
Type I Error ◽  
A Priori ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Error Type ◽  
Significant Difference ◽  
The Difference ◽  
A Priori Probability

In the classical decision-making model, the experimenter seeks to demonstrate a difference between the distributions of two or more samples (or a sample and a population) for some parameter. A null hypothesis is stated that there is no “significant” difference between the distributions, and probabilistic models are used to determine the probability that any difference is due to chance. Philosophically, the researcher decides in advance that he is only willing to accept a probability less than a given size for making the error of assuming that the difference is real when it is actually due to chance. The a priori probability of such an error (Type I) is designated α. But the possibility of another error exists: the error of failing to reject a null hypothesis when the difference is real. (The a priori probability of such an error is designated β.) Whereas the probability of a Type I error has been controlled by the choice of a significance criterion, the Type II error (failing to reject a false null hypothesis) has seldom been controlled. However, the use of the technique of power analysis now makes it possible to control the probability of a Type II error with little more difficulty than the technique used to control the probability of a Type I error.

scholarly journals BANKRUPTCY PREDICTION MODEL WITH ZETAc OPTIMAL CUT-OFF SCORE TO CORRECT TYPE I ERRORS

2005 ◽  
Vol 7 (1) ◽  
pp. 41 ◽  
Author(s):  
Mohamad Iwan
Keyword(s):  
Type I Error ◽  
Bankruptcy Prediction ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Prior Probabilities ◽  
Prediction Result ◽  
Optimum Cutting ◽  
Classification Errors ◽  
One Year

This research examines financial ratios that distinguish between bankrupt and non-bankrupt companies and make use of those distinguishing ratios to build a one-year prior to bankruptcy prediction model. This research also calculates how many times the type I error is more costly compared to the type II error. The costs of type I and type II errors (cost of misclassification errors) in conjunction to the calculation of prior probabilities of bankruptcy and non-bankruptcy are used in the calculation of the ZETAc optimal cut-off score. The bankruptcy prediction result using ZETAc optimal cut-off score is compared to the bankruptcy prediction result using a cut-off score which does not consider neither cost of classification errors nor prior probabilities as stated by Hair et al. (1998), and for later purposes will be referred to Hair et al. optimum cutting score. Comparison between the prediction results of both cut-off scores is purported to determine the better cut-off score between the two, so that the prediction result is more conservative and minimizes expected costs, which may occur from classification errors.  This is the first research in Indonesia that incorporates type I and II errors and prior probabilities of bankruptcy and non-bankruptcy in the computation of the cut-off score used in performing bankruptcy prediction. Earlier researches gave the same weight between type I and II errors and prior probabilities of bankruptcy and non-bankruptcy, while this research gives a greater weigh on type I error than that on type II error and prior probability of non-bankruptcy than that on prior probability of bankruptcy.This research has successfully attained the following results: (1) type I error is in fact 59,83 times more costly compared to type II error, (2) 22 ratios distinguish between bankrupt and non-bankrupt groups, (3) 2 financial ratios proved to be effective in predicting bankruptcy, (4) prediction using ZETAc optimal cut-off score predicts more companies filing for bankruptcy within one year compared to prediction using Hair et al. optimum cutting score, (5) Although prediction using Hair et al. optimum cutting score is more accurate, prediction using ZETAc optimal cut-off score proved to be able to minimize cost incurred from classification errors.

Is It Really Robust?

Methodology ◽  
2010 ◽  
Vol 6 (4) ◽  
pp. 147-151 ◽  
Author(s):  
Emanuel Schmider ◽  
Matthias Ziegler ◽  
Erik Danay ◽  
Luzi Beyer ◽  
Markus Bühner
Keyword(s):  
Goodness Of Fit ◽  
Type I Error ◽  
Effect Sizes ◽  
Random Numbers ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Normality Assumption ◽  
Different Types ◽  
Factor Type

Empirical evidence to the robustness of the analysis of variance (ANOVA) concerning violation of the normality assumption is presented by means of Monte Carlo methods. High-quality samples underlying normally, rectangularly, and exponentially distributed basic populations are created by drawing samples which consist of random numbers from respective generators, checking their goodness of fit, and allowing only the best 10% to take part in the investigation. A one-way fixed-effect design with three groups of 25 values each is chosen. Effect-sizes are implemented in the samples and varied over a broad range. Comparing the outcomes of the ANOVA calculations for the different types of distributions, gives reason to regard the ANOVA as robust. Both, the empirical type I error α and the empirical type II error β remain constant under violation. Moreover, regression analysis identifies the factor “type of distribution” as not significant in explanation of the ANOVA results.

Automatic Real-Time Identification of Fingerprint Images Using Wavelets and Gradient of Gaussian

1997 ◽  
Vol 07 (05) ◽  
pp. 433-440 ◽  
Author(s):  
Woo Kyu Lee ◽  
Jae Ho Chung
Keyword(s):  
Wavelet Transform ◽  
Real Time ◽  
Type I Error ◽  
Recognition Algorithm ◽  
Fingerprint Recognition ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Local Orientation ◽  
Real Time Identification

In this paper, a fingerprint recognition algorithm is suggested. The algorithm is developed based on the wavelet transform, and the dominant local orientation which is derived from the coherence and the gradient of Gaussian. By using the wavelet transform, the algorithm does not require conventional preprocessing procedures such as smoothing, binarization, thining and restoration. Computer simulation results show that when the rate of Type II error — Incorrect recognition of two different fingerprints as identical fingerprints — is held at 0.0%, the rate of Type I error — Incorrect recognition of two identical fingerprints as different ones — turns out as 2.5% in real time.

A New and Simpler Approximation for ANOVA Under Variance Heterogeneity

1994 ◽  
Vol 19 (2) ◽  
pp. 91-101 ◽  
Author(s):  
Ralph A. Alexander ◽  
Diane M. Govern
Keyword(s):  
Type I Error ◽  
Order Approximation ◽  
Error Rates ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Tail Probabilities ◽  
Type I Error Rates ◽  
Heterogeneity Of Variance ◽  
The Face

A new approximation is proposed for testing the equality of k independent means in the face of heterogeneity of variance. Monte Carlo simulations show that the new procedure has Type I error rates that are very nearly nominal and Type II error rates that are quite close to those produced by James’s (1951) second-order approximation. In addition, it is computationally the simplest approximation yet to appear, and it is easily applied to Scheffé (1959) -type multiple contrasts and to the calculation of approximate tail probabilities.

scholarly journals Symmetry's Mandate: Constraining the Politicization of American Administrative Law

2020 ◽  
pp. 455
Author(s):  
Daniel Walters
Keyword(s):  
Type I Error ◽  
Theoretical Perspective ◽  
Administrative Law ◽  
Political Conflict ◽  
Regulatory Action ◽  
Positive Case ◽  
Type I ◽  
Type Ii ◽  
Type Ii Error ◽  
Call Type

Recent years have seen the rise of pointed and influential critiques of deference doctrines in administrative law. What many of these critiques have in common is a view that judges, not agencies, should resolve interpretive disputes over the meaning of statutes—disputes the critics take to be purely legal and almost always resolvable using lawyerly tools of statutory construction. In this Article, I take these critiques, and the relatively formalist assumptions behind them, seriously and show that the critics have not acknowledged or advocated the full reform vision implied by their theoretical premises. Specifically, critics have extended their critique of judicial abdication only to what I call Type I statutory errors (that is, agency interpretations that regulate more conduct than the best reading of the statute would allow the agency to regulate) and do not appear to accept or anticipate that their theory of interpretation would also extend to what I call Type II statutory errors (that is, agency failures to regulate as much conduct as the best reading of the statute would require). As a consequence, critics have been more than willing to entertain an end to Chevron deference, an administrative law doctrine that is mostly invoked to justify Type I error, but have not shown any interest in adjusting administrative law doctrine to remedy agencies’ commission of Type II error. The result is a vision of administrative law’s future that is precariously slanted against legislative and regulatory action. I critique this asymmetry in administrative law and address potential justifications of systemic asymmetries in the doctrine, such as concern about the remedial implications of addressing Type II error, finding them all wanting from a legal and theoretical perspective. I also lay out the positive case for adhering to symmetry in administrative law doctrine. In a time of deep political conflict over regulation and administration, symmetry plays, or at the very least could play, an important role in depoliticizing administrative law, clarifying what is at stake in debates about the proper level of deference to agency legal interpretations, and disciplining partisan gamesmanship. I suggest that when the conversation is so disciplined, an administrative law without deference to both Type I and Type II error is hard to imagine due to the high judicial costs of minimizing Type II error, but if we collectively choose to discard deference notwithstanding these costs, it would be a more sustainable political choice for administrative law than embracing the current, one-sided critique of deference.

Sign in / Sign up

Export Citation Format

Share Document