scholarly journals BayesFactorFMRI: Implementing Bayesian Second-Level fMRI Analysis with Multiple Comparison Correction and Bayesian Meta-Analysis of fMRI Images with Multiprocessing

2021 ◽  
Vol 9 ◽  
Author(s):  
Hyemin Han
Keyword(s):  
Meta Analysis ◽  
Multiple Comparison ◽  
Fmri Analysis ◽  
Multiple Comparison Correction

scholarly journals BayesFactorFMRI: Implementing Bayesian second-level fMRI analysis with multiple comparison correction and Bayesian meta-analysis of fMRI images with multiprocessing

2020 ◽  
Author(s):  
Hyemin Han
Keyword(s):  
Bayesian Analysis ◽  
Meta Analysis ◽  
Image Data ◽  
Fmri Data ◽  
Classical Analysis ◽  
Multiple Processors ◽  
Computationally Intensive ◽  
Fmri Analysis ◽  
Multiple Comparison Correction ◽  
Level Analysis

AbstractBayesFactorFMRI is a tool developed with R and Python to allow neuroimaging researchers to conduct Bayesian second-level analysis and Bayesian meta-analysis of fMRI image data with multiprocessing. This tool expedites computationally intensive Bayesian fMRI analysis through multiprocessing. Its GUI allows researchers who are not experts in computer programming to feasibly perform Bayesian fMRI analysis. BayesFactorFMRI is available via Zenodo and GitHub for download. It would be widely reused by neuroimaging researchers who intend to analyse their fMRI data with Bayesian analysis with better sensitivity compared with classical analysis while improving performance by distributing analysis tasks into multiple processors.

scholarly journals Small effect size leads to reproducibility failure in resting-state fMRI studies

2018 ◽  
Author(s):  
Xi-Ze Jia ◽  
Na Zhao ◽  
Barek Barton ◽  
Roxana Burciu ◽  
Nicolas Carrière ◽  
...  
Keyword(s):  
Effect Size ◽  
Resting State ◽  
Meta Analysis ◽  
False Negative ◽  
Autism Spectrum ◽  
Multiple Comparison ◽  
Brain Disorders ◽  
Group Design ◽  
Small Effect Size ◽  
Multiple Comparison Correction

AbstractThousands of papers using resting-state functional magnetic resonance imaging (RS-fMRI) have been published on brain disorders. Results in each paper may have survived correction for multiple comparison. However, since there have been no robust results from large scale meta-analysis, we do not know how many of published results are truly positives. The present meta-analytic work included 60 original studies, with 57 studies (4 datasets, 2266 participants) that used a between-group design and 3 studies (1 dataset, 107 participants) that employed a within-group design. To evaluate the effect size of brain disorders, a very large neuroimaging dataset ranging from neurological to psychiatric isorders together with healthy individuals have been analyzed. Parkinson’s disease off levodopa (PD-off) included 687 participants from 15 studies. PD on levodopa (PD-on) included 261 participants from 9 studies. Autism spectrum disorder (ASD) included 958 participants from 27 studies. The meta-analyses of a metric named amplitude of low frequency fluctuation (ALFF) showed that the effect size (Hedges’ g) was 0.19 - 0.39 for the 4 datasets using between-group design and 0.46 for the dataset using within-group design. The effect size of PD-off, PD-on and ASD were 0.23, 0.39, and 0.19, respectively. Using the meta-analysis results as the robust results, the between-group design results of each study showed high false negative rates (median 99%), high false discovery rates (median 86%), and low accuracy (median 1%), regardless of whether stringent or liberal multiple comparison correction was used. The findings were similar for 4 RS-fMRI metrics including ALFF, regional homogeneity, and degree centrality, as well as for another widely used RS-fMRI metric namely seed-based functional connectivity. These observations suggest that multiple comparison correction does not control for false discoveries across multiple studies when the effect sizes are relatively small. Meta-analysis on un-thresholded t-maps is critical for the recovery of ground truth. We recommend that to achieve high reproducibility through meta-analysis, the neuroimaging research field should share raw data or, at minimum, provide un-thresholded statistical images.

scholarly journals Implementation of Bayesian Multiple Comparison Correction in the Second-Level Analysis of fMRI Data: With Pilot Analyses of Simulation and Real fMRI Datasets Based on Voxelwise Inference

2019 ◽  
Author(s):  
Hyemin Han
Keyword(s):  
False Alarm ◽  
Correction Method ◽  
Multiple Comparison ◽  
Real Image ◽  
Classical Inference ◽  
Fmri Analysis ◽  
Simulated Images ◽  
Multiple Comparison Correction ◽  
Level Analysis ◽  
Image Datasets

AbstractWe developed and tested Bayesian multiple comparison correction method for Bayesian voxelwise second-level fMRI analysis with R. The performance of the developed method was tested with simulation and real image datasets. First, we compared false alarm and hit rates, which were used as proxies for selectivity and sensitivity, respectively, between Bayesian and classical inference were conducted. For the comparison, we created simulated images, added noise to the created images, and analyzed the noise-added images while applying Bayesian and classical multiple comparison correction methods. Second, we analyzed five real image datasets to examine how our Bayesian method worked in realistic settings. When the performance assessment was conducted, Bayesian correction method demonstrated good sensitivity (hit rate ≥ 75%) and acceptable selectivity (false alarm rate < 10%) when N ≤ 8. Furthermore, Bayesian correction method showed better sensitivity compared with classical correction method while maintaining the aforementioned acceptable selectivity.

scholarly journals Evaluation of multiple comparison correction procedures in drug assessment studies using LORETA maps

2015 ◽  
Vol 53 (10) ◽  
pp. 1011-1023 ◽  
Author(s):  
Joan Francesc Alonso ◽  
Sergio Romero ◽  
Miguel Ángel Mañanas ◽  
Mónica Rojas ◽  
Jordi Riba ◽  
...  
Keyword(s):  
Multiple Comparison ◽  
Multiple Comparison Correction

scholarly journals Reproducibility of R-fMRI Metrics on the Impact of Different Strategies for Multiple Comparison Correction and Sample Sizes

2017 ◽  
Author(s):  
Xiao Chen ◽  
Bin Lu ◽  
Chao-Gan Yan
Keyword(s):  
Sex Differences ◽  
Sample Size ◽  
Small Sample ◽  
Multiple Comparison ◽  
Sample Sizes ◽  
Retest Reliability ◽  
Test Retest Reliability ◽  
The Impact ◽  
Correction Strategies ◽  
Multiple Comparison Correction

ABSTRACTConcerns regarding reproducibility of resting-state functional magnetic resonance imaging (R-fMRI) findings have been raised. Little is known about how to operationally define R-fMRI reproducibility and to what extent it is affected by multiple comparison correction strategies and sample size. We comprehensively assessed two aspects of reproducibility, test-retest reliability and replicability, on widely used R-fMRI metrics in both between-subject contrasts of sex differences and within-subject comparisons of eyes-open and eyes-closed (EOEC) conditions. We noted permutation test with Threshold-Free Cluster Enhancement (TFCE), a strict multiple comparison correction strategy, reached the best balance between family-wise error rate (under 5%) and test-retest reliability / replicability (e.g., 0.68 for test-retest reliability and 0.25 for replicability of amplitude of low-frequency fluctuations (ALFF) for between-subject sex differences, 0.49 for replicability of ALFF for within-subject EOEC differences). Although R-fMRI indices attained moderate reliabilities, they replicated poorly in distinct datasets (replicability < 0.3 for between-subject sex differences, < 0.5 for within-subject EOEC differences). By randomly drawing different sample sizes from a single site, we found reliability, sensitivity and positive predictive value (PPV) rose as sample size increased. Small sample sizes (e.g., < 80 (40 per group)) not only minimized power (sensitivity < 2%), but also decreased the likelihood that significant results reflect “true” effects (PPV < 0.26) in sex differences. Our findings have implications for how to select multiple comparison correction strategies and highlight the importance of sufficiently large sample sizes in R-fMRI studies to enhance reproducibility.

scholarly journals The Garden of Forking Paths in Visualization: A Design Space for Reliable Exploratory Visual Analytics

2018 ◽  
Author(s):  
Xiaoying Pu ◽  
Matthew Kay
Keyword(s):  
Bias Correction ◽  
Visual Analytics ◽  
Design Space ◽  
Multiple Comparisons ◽  
Perceptual Bias ◽  
Multiple Comparison ◽  
Reliability Of Results ◽  
Nested Model ◽  
Paper Review ◽  
Multiple Comparison Correction

Tukey emphasized decades ago that taking exploratory findings as confirmatory is “destructively foolish”. We reframe recent conversations about the reliability of results from exploratory visual analytics—such as the multiple comparisons problem—in terms of Gelman and Loken’s garden of forking paths to lay out a design space for addressing the forking paths problem in visual analytics. This design space encompasses existing approaches to address the forking paths problem (multiple comparison correction) as well as solutions that have not been applied to exploratory visual analytics (regularization). We also discuss how perceptual bias correction techniques may be used to correct biases induced in analysts’ understanding of their data due to the forking paths problem, and outline how this problem can be cast as a threat to validity within Munzner’s Nested Model of visualization design. Finally, we suggest paper review guidelines to encourage reviewers to consider the forking paths problem when evaluating future designs of visual analytics tools.

scholarly journals A method to adjust a prior distribution in Bayesian second-level fMRI analysis

2020 ◽  
Author(s):  
Hyemin Han
Keyword(s):  
Prior Distribution ◽  
Bayes Factor ◽  
Meta Analysis ◽  
False Positive Rate ◽  
A Priori ◽  
Correction Method ◽  
Positive Rate ◽  
Improved Performance ◽  
Fmri Analysis ◽  
Priori Information

Previous research has shown the potential value of Bayesian methods in fMRI (functional magnetic resonance imaging) analysis. For instance, the results from Bayes factor-applied second-level fMRI analysis showed a higher hit rate compared with frequentist second-level fMRI analysis, suggesting greater sensitivity. Although the method reported more positives as a result of the higher sensitivity, it was able to maintain a reasonable level of selectivity in term of the false positive rate. Moreover, employment of the multiple comparison correction method to update the default prior distribution significantly improved the performance of Bayesian second-level fMRI analysis. However, previous studies have utilized the default prior distribution and did not consider the nature of each individual study. Thus, in the present study, a method to adjust the Cauchy prior distribution based on a priori information, which can be acquired from the results of relevant previous studies, was proposed and tested. A Cauchy prior distribution was adjusted based on the contrast, noise strength, and proportion of true positives that were estimated from a meta-analysis of relevant previous studies. In the present study, both the simulated images and real contrast images from two previous studies were used to evaluate the performance of the proposed method. The results showed that the employment of the prior adjustment method resulted in improved performance of Bayesian second-level fMRI analysis.

scholarly journals Evaluating Alternative Correction Methods for Multiple Comparison in Functional Neuroimaging Research

2019 ◽  
Vol 9 (8) ◽  
pp. 198 ◽  
Author(s):  
Hyemin Han ◽  
Andrea L. Glenn ◽  
Kelsie J. Dawson
Keyword(s):  
False Alarm ◽  
Functional Neuroimaging ◽  
Multiple Comparisons ◽  
Multiple Comparison ◽  
Alternative Methods ◽  
Parametric Mapping ◽  
Significant Challenge ◽  
Hit Rate ◽  
Autocorrelated Noise ◽  
Multiple Comparison Correction

A significant challenge for fMRI research is statistically controlling for false positives without omitting true effects. Although a number of traditional methods for multiple comparison correction exist, several alternative tools have been developed that do not rely on strict parametric assumptions, but instead implement alternative methods to correct for multiple comparisons. In this study, we evaluated three of these methods, Statistical non-Parametric Mapping (SnPM), 3DClustSim, and Threshold Free Cluster Enhancement (TFCE), by examining which method produced the most consistent outcomes even when spatially-autocorrelated noise was added to the original images. We assessed the false alarm rate and hit rate of each method after noise was applied to the original images.

scholarly journals A method to adjust a prior distribution in Bayesian second-level fMRI analysis

PeerJ ◽  
2021 ◽  
Vol 9 ◽  
pp. e10861
Author(s):  
Hyemin Han
Keyword(s):  
Prior Distribution ◽  
Bayes Factor ◽  
Meta Analysis ◽  
False Positive Rate ◽  
A Priori ◽  
Correction Method ◽  
Positive Rate ◽  
Improved Performance ◽  
Fmri Analysis ◽  
Priori Information

Previous research has shown the potential value of Bayesian methods in fMRI (functional magnetic resonance imaging) analysis. For instance, the results from Bayes factor-applied second-level fMRI analysis showed a higher hit rate compared with frequentist second-level fMRI analysis, suggesting greater sensitivity. Although the method reported more positives as a result of the higher sensitivity, it was able to maintain a reasonable level of selectivity in term of the false positive rate. Moreover, employment of the multiple comparison correction method to update the default prior distribution significantly improved the performance of Bayesian second-level fMRI analysis. However, previous studies have utilized the default prior distribution and did not consider the nature of each individual study. Thus, in the present study, a method to adjust the Cauchy prior distribution based on a priori information, which can be acquired from the results of relevant previous studies, was proposed and tested. A Cauchy prior distribution was adjusted based on the contrast, noise strength, and proportion of true positives that were estimated from a meta-analysis of relevant previous studies. In the present study, both the simulated images and real contrast images from two previous studies were used to evaluate the performance of the proposed method. The results showed that the employment of the prior adjustment method resulted in improved performance of Bayesian second-level fMRI analysis.

Sign in / Sign up

Export Citation Format

Share Document