scholarly journals Data-driven hypothesis weighting increases detection power in multiple testing

2015 ◽  
Author(s):  
Nikolaos Ignatiadis ◽  
Bernd Klaus ◽  
Judith Zaugg ◽  
Wolfgang Huber
Keyword(s):  
Multiple Testing ◽  
Statistical Power ◽  
Prior Probability ◽  
Large Datasets ◽  
Data Driven ◽  
Dependent Manner ◽  
Test Statistic ◽  
Additional Insight ◽  
False Discovery ◽  
Powerful Approach

AbstractHypothesis weighting is a powerful approach for improving the power of data analyses that employ multiple testing. However, in general it is not evident how to choose the weights in a data-dependent manner. We describe independent hypothesis weighting (IHW), a method for making use of informative covariates that are independent of the test statistic under the null, but informative of each test’s power or prior probability of the null hypothesis. Covariates can be continuous or categorical and need not fulfill any particular assumptions. The method increases statistical power in applications while controlling the false discovery rate (FDR) and produces additional insight by revealing the covariate-weight relationship. Independent hypothesis weighting is a practical approach to discovery of associations in large datasets.

scholarly journals 2dFDR: a new approach to confounder adjustment substantially increases detection power in omics association studies

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Sangyoon Yi ◽  
Xianyang Zhang ◽  
Lu Yang ◽  
Jinyan Huang ◽  
Yuanhang Liu ◽  
...  
Keyword(s):  
Multiple Testing ◽  
Statistical Power ◽  
Association Studies ◽  
Control Procedure ◽  
Multiple Testing Correction ◽  
New Approach ◽  
False Discovery ◽  
Traditional Procedure ◽  
Extensive Evaluation ◽  
Confounder Adjustment

AbstractOne challenge facing omics association studies is the loss of statistical power when adjusting for confounders and multiple testing. The traditional statistical procedure involves fitting a confounder-adjusted regression model for each omics feature, followed by multiple testing correction. Here we show that the traditional procedure is not optimal and present a new approach, 2dFDR, a two-dimensional false discovery rate control procedure, for powerful confounder adjustment in multiple testing. Through extensive evaluation, we demonstrate that 2dFDR is more powerful than the traditional procedure, and in the presence of strong confounding and weak signals, the power improvement could be more than 100%.

On “Field Significance” and the False Discovery Rate

2006 ◽  
Vol 45 (9) ◽  
pp. 1181-1189 ◽  
Author(s):  
D. S. Wilks
Keyword(s):  
False Discovery Rate ◽  
Statistical Power ◽  
Statistical Significance ◽  
Significance Test ◽  
P Value ◽  
Test Statistic ◽  
Global Test ◽  
Additional Advantage ◽  
Counting Procedure ◽  
False Discovery

Abstract The conventional approach to evaluating the joint statistical significance of multiple hypothesis tests (i.e., “field,” or “global,” significance) in meteorology and climatology is to count the number of individual (or “local”) tests yielding nominally significant results and then to judge the unusualness of this integer value in the context of the distribution of such counts that would occur if all local null hypotheses were true. The sensitivity (i.e., statistical power) of this approach is potentially compromised both by the discrete nature of the test statistic and by the fact that the approach ignores the confidence with which locally significant tests reject their null hypotheses. An alternative global test statistic that has neither of these problems is the minimum p value among all of the local tests. Evaluation of field significance using the minimum local p value as the global test statistic, which is also known as the Walker test, has strong connections to the joint evaluation of multiple tests in a way that controls the “false discovery rate” (FDR, or the expected fraction of local null hypothesis rejections that are incorrect). In particular, using the minimum local p value to evaluate field significance at a level αglobal is nearly equivalent to the slightly more powerful global test based on the FDR criterion. An additional advantage shared by Walker’s test and the FDR approach is that both are robust to spatial dependence within the field of tests. The FDR method not only provides a more broadly applicable and generally more powerful field significance test than the conventional counting procedure but also allows better identification of locations with significant differences, because fewer than αglobal × 100% (on average) of apparently significant local tests will have resulted from local null hypotheses that are true.

Brain networks construction using Bayes FDR and average power function

2019 ◽  
Vol 29 (3) ◽  
pp. 866-878
Author(s):  
Piero Quatto ◽  
Nicolò Margaritella ◽  
Isa Costantini ◽  
Francesca Baglio ◽  
Massimo Garegnani ◽  
...  
Keyword(s):  
Time Series ◽  
False Discovery Rate ◽  
Correlation Coefficient ◽  
Power Function ◽  
Multiple Testing ◽  
Statistical Power ◽  
Brain Connectivity ◽  
Average Power ◽  
Statistical Error ◽  
False Discovery

Brain functional connectivity is a widely investigated topic in neuroscience. In recent years, the study of brain connectivity has been largely aided by graph theory. The link between time series recorded at multiple locations in the brain and the construction of a graph is usually an adjacency matrix. The latter converts a measure of the connectivity between two time series, typically a correlation coefficient, into a binary choice on whether the two brain locations are functionally connected or not. As a result, the choice of a threshold τ over the correlation coefficient is key. In the present work, we propose a multiple testing approach to the choice of τ that uses the Bayes false discovery rate and a new estimator of the statistical power called average power function to balance the two types of statistical error. We show that the proposed average power function estimator behaves well both in case of independence and weak dependence of the tests and it is reliable under several simulated dependence conditions. Moreover, we propose a robust method for the choice of τ using the 5% and 95% percentiles of the average power function and False Discovery Rate bootstrap distributions, respectively, to improve stability. We applied our approach to functional magnetic resonance imaging and high density electroencephalogram data.

scholarly journals An Empirical Bayes Optimal Discovery Procedure Based on Semiparametric Hierarchical Mixture Models

2013 ◽  
Vol 2013 ◽  
pp. 1-9
Author(s):  
Hisashi Noma ◽  
Shigeyuki Matsui
Keyword(s):  
Mixture Model ◽  
Multiple Testing ◽  
Empirical Bayes ◽  
Gene Selection ◽  
Fixed Number ◽  
Test Statistic ◽  
Microarray Experiments ◽  
False Discovery ◽  
Genome Wide ◽  
Optimal Discovery Procedure

Multiple testing has been widely adopted for genome-wide studies such as microarray experiments. For effective gene selection in these genome-wide studies, the optimal discovery procedure (ODP), which maximizes the number of expected true positives for each fixed number of expected false positives, was developed as a multiple testing extension of the most powerful test for a single hypothesis by Storey (Journal of the Royal Statistical Society, Series B,vol. 69, no. 3, pp. 347–368, 2007). In this paper, we develop an empirical Bayes method for implementing the ODP based on a semiparametric hierarchical mixture model using the “smoothing-by-roughening" approach. Under the semiparametric hierarchical mixture model, (i) the prior distribution can be modeled flexibly, (ii) the ODP test statistic and the posterior distribution are analytically tractable, and (iii) computations are easy to implement. In addition, we provide a significance rule based on the false discovery rate (FDR) in the empirical Bayes framework. Applications to two clinical studies are presented.

scholarly journals A powerful and efficient two-stage method for detecting gene-to-gene interactions in GWAS

Biostatistics ◽  
2017 ◽  
Vol 18 (3) ◽  
pp. 477-494 ◽  
Author(s):  
Jakub Pecanka ◽  
Marianne A. Jonker ◽  
Zoltan Bochdanovits ◽  
Aad W. Van Der Vaart ◽  
Keyword(s):  
Complex Traits ◽  
Multiple Testing ◽  
Statistical Power ◽  
Genome Wide Association Study ◽  
Score Test ◽  
Interaction Model ◽  
Type I ◽  
Two Stage ◽  
Genome Wide ◽  
Strong Control

Summary For over a decade functional gene-to-gene interaction (epistasis) has been suspected to be a determinant in the “missing heritability” of complex traits. However, searching for epistasis on the genome-wide scale has been challenging due to the prohibitively large number of tests which result in a serious loss of statistical power as well as computational challenges. In this article, we propose a two-stage method applicable to existing case-control data sets, which aims to lessen both of these problems by pre-assessing whether a candidate pair of genetic loci is involved in epistasis before it is actually tested for interaction with respect to a complex phenotype. The pre-assessment is based on a two-locus genotype independence test performed in the sample of cases. Only the pairs of loci that exhibit non-equilibrium frequencies are analyzed via a logistic regression score test, thereby reducing the multiple testing burden. Since only the computationally simple independence tests are performed for all pairs of loci while the more demanding score tests are restricted to the most promising pairs, genome-wide association study (GWAS) for epistasis becomes feasible. By design our method provides strong control of the type I error. Its favourable power properties especially under the practically relevant misspecification of the interaction model are illustrated. Ready-to-use software is available. Using the method we analyzed Parkinson’s disease in four cohorts and identified possible interactions within several SNP pairs in multiple cohorts.

Application of the False Discovery Rate to Quantitative Trait Loci Interval Mapping With Multiple Traits

Genetics ◽  
2002 ◽  
Vol 161 (2) ◽  
pp. 905-914 ◽  
Author(s):  
Hakkyo Lee ◽  
Jack C M Dekkers ◽  
M Soller ◽  
Massoud Malek ◽  
Rohan L Fernando ◽  
...  
Keyword(s):  
Quantitative Trait Loci ◽  
False Discovery Rate ◽  
Error Rate ◽  
Quantitative Trait ◽  
Interval Mapping ◽  
Error Rates ◽  
Multiple Traits ◽  
Test Statistic ◽  
False Discovery ◽  
Trait Loci

Abstract Controlling the false discovery rate (FDR) has been proposed as an alternative to controlling the genomewise error rate (GWER) for detecting quantitative trait loci (QTL) in genome scans. The objective here was to implement FDR in the context of regression interval mapping for multiple traits. Data on five traits from an F2 swine breed cross were used. FDR was implemented using tests at every 1 cM (FDR1) and using tests with the highest test statistic for each marker interval (FDRm). For the latter, a method was developed to predict comparison-wise error rates. At low error rates, FDR1 behaved erratically; FDRm was more stable but gave similar significance thresholds and number of QTL detected. At the same error rate, methods to control FDR gave less stringent significance thresholds and more QTL detected than methods to control GWER. Although testing across traits had limited impact on FDR, single-trait testing was recommended because there is no theoretical reason to pool tests across traits for FDR. FDR based on FDRm was recommended for QTL detection in interval mapping because it provides significance tests that are meaningful, yet not overly stringent, such that a more complete picture of QTL is revealed.

scholarly journals Alpha Is Not the False Alarm Rate: An Activity to Dispel a Common Statistical Misconception

2018 ◽  
Vol 46 (1) ◽  
pp. 72-79 ◽  
Author(s):  
W. Burt Thompson
Keyword(s):  
False Alarm ◽  
False Alarm Rate ◽  
Research Finding ◽  
Statistical Power ◽  
Prior Probability ◽  
Statistical Significance ◽  
Statistical Test ◽  
Test Results ◽  
Web App ◽  
New Research

When a psychologist announces a new research finding, it is often based on a rejected null hypothesis. However, if that hypothesis is true, the claim is a false alarm. Many students mistakenly believe that the probability of committing a false alarm equals alpha, the criterion for statistical significance, which is typically set at 5%. Instructors should take specific steps to dispel this belief because it leads students to misinterpret statistical test results and it reinforces the more general misconception that results can be interpreted in isolation, without reference to theory or prior research. In the present study, students worked with a web app that shows how the false alarm rate is a function of the prior probability of an effect, statistical power, and alpha. Quiz scores suggest the activity helps correct the misconception, which can improve how students conduct and interpret research.

On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics

2000 ◽  
Vol 25 (1) ◽  
pp. 60-83 ◽  
Author(s):  
Yoav Benjamini ◽  
Yosef Hochberg
Keyword(s):  
False Discovery Rate ◽  
Multiple Testing ◽  
Meta Analysis ◽  
Test Statistics ◽  
Adaptive Procedure ◽  
New Approach ◽  
False Discovery ◽  
Behavioral Studies ◽  
Simultaneous Testing ◽  
Independent Test

A new approach to problems of multiple significance testing was presented in Benjamini and Hochberg (1995), which calls for controlling the expected ratio of the number of erroneous rejections to the number of rejections–the False Discovery Rate (FDR). The procedure given there was shown to control the FDR for independent test statistics. When some of the hypotheses are in fact false, that procedure is too conservative. We present here an adaptive procedure, where the number of true null hypotheses is estimated first as in Hochberg and Benjamini (1990), and this estimate is used in the procedure of Benjamini and Hochberg (1995). The result is still a simple stepwise procedure, to which we also give a graphical companion. The new procedure is used in several examples drawn from educational and behavioral studies, addressing problems in multi-center studies, subset analysis and meta-analysis. The examples vary in the number of hypotheses tested, and the implication of the new procedure on the conclusions. In a large simulation study of independent test statistics the adaptive procedure is shown to control the FDR and have substantially better power than the previously suggested FDR controlling method, which by itself is more powerful than the traditional family wise error-rate controlling methods. In cases where most of the tested hypotheses are far from being true there is hardly any penalty due to the simultaneous testing of many hypotheses.

Sign in / Sign up

Export Citation Format

Share Document