scholarly journals Meta-Analysis of ERP Investigations of Pain Empathy underlines methodological issues in ERP research

2018 ◽  
Author(s):  
Michel-Pierre Coll
Keyword(s):  
Error Control ◽  
Meta Analysis ◽  
Event Related Potential ◽  
Selective Reporting ◽  
Validity And Reliability ◽  
Type 1 Error ◽  
Empathy For Pain ◽  
Potential Component ◽  
Clinical Populations

AbstractEmpathy has received considerable attention from the field of cognitive and social neuroscience. A significant portion of these studies used the event-related potential (ERP) technique to study the mechanisms of empathy for pain in others in different conditions and clinical populations. These show that specific ERP components measured during the observation of pain in others are modulated by several factors and altered in clinical populations. However, issues present in this literature such as analytical flexibility and lack of type 1 error control raise doubts regarding the validity and reliability of these conclusions. The current study compiled the results and methodological characteristics of 40 studies using ERP to study empathy of pain in others. The results of the meta-analysis suggest that the centro-parietal P3 and late positive potential component are sensitive to the observation of pain in others, while the early N1 and N2 components are not reliably associated with vicarious pain observation. The review of the methodological characteristics shows that the presence of selective reporting, analytical flexibility and lack of type 1 error control compromise the interpretation of these results. The implication of these results for the study of empathy and potential solutions to improve future investigations are discussed.

scholarly journals Coordinate Based Random Effect Size meta-analysis of neuroimaging studies

2016 ◽  
Author(s):  
CR Tench ◽  
Radu Tanasescu ◽  
WJ Cottam ◽  
CS Constantinescu ◽  
DP Auer
Keyword(s):  
Effect Size ◽  
Error Control ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Random Effect ◽  
Simulated Data ◽  
Real Data ◽  
Type 1 Error ◽  
Meta Analyses

1AbstractLow power in neuroimaging studies can make them difficult to interpret, and Coordinate based meta‐ analysis (CBMA) may go some way to mitigating this issue. CBMA has been used in many analyses to detect where published functional MRI or voxel-based morphometry studies testing similar hypotheses report significant summary results (coordinates) consistently. Only the reported coordinates and possibly t statistics are analysed, and statistical significance of clusters is determined by coordinate density.Here a method of performing coordinate based random effect size meta-analysis and meta-regression is introduced. The algorithm (ClusterZ) analyses both coordinates and reported t statistic or Z score, standardised by the number of subjects. Statistical significance is determined not by coordinate density, but by a random effects meta-analyses of reported effects performed cluster-wise using standard statistical methods and taking account of censoring inherent in the published summary results. Type 1 error control is achieved using the false cluster discovery rate (FCDR), which is based on the false discovery rate. This controls both the family wise error rate under the null hypothesis that coordinates are randomly drawn from a standard stereotaxic space, and the proportion of significant clusters that are expected under the null. Such control is vital to avoid propagating and even amplifying the very issues motivating the meta-analysis in the first place. ClusterZ is demonstrated on both numerically simulated data and on real data from reports of grey matter loss in multiple sclerosis (MS) and syndromes suggestive of MS, and of painful stimulus in healthy controls. The software implementation is available to download and use freely.

scholarly journals Accurate error control in high dimensional association testing using conditional false discovery rates

2018 ◽  
Author(s):  
James Liley ◽  
Chris Wallace
Keyword(s):  
Error Rate ◽  
Error Control ◽  
Association Studies ◽  
New Method ◽  
High Dimensional ◽  
Biomedical Sciences ◽  
Type 1 Error ◽  
False Discovery ◽  
Multiple Covariates

AbstractHigh-dimensional hypothesis testing is ubiquitous in the biomedical sciences, and informative covariates may be employed to improve power. The conditional false discovery rate (cFDR) is widely-used approach suited to the setting where the covariate is a set of p-values for the equivalent hypotheses for a second trait. Although related to the Benjamini-Hochberg procedure, it does not permit any easy control of type-1 error rate, and existing methods are over-conservative. We propose a new method for type-1 error rate control based on identifying mappings from the unit square to the unit interval defined by the estimated cFDR, and splitting observations so that each map is independent of the observations it is used to test. We also propose an adjustment to the existing cFDR estimator which further improves power. We show by simulation that the new method more than doubles potential improvement in power over unconditional analyses compared to existing methods. We demonstrate our method on transcriptome-wide association studies, and show that the method can be used in an iterative way, enabling the use of multiple covariates successively. Our methods substantially improve the power and applicability of cFDR analysis.

scholarly journals Magnitude Based Inference in Relation to One-sided Hypotheses Testing Procedures

2020 ◽  
Author(s):  
Janet Aisbett ◽  
Daniel Lakens ◽  
Kristin Sainani
Keyword(s):  
Error Control ◽  
Hypothesis Test ◽  
A Priori ◽  
Error Rates ◽  
Equivalence Testing ◽  
High Type ◽  
Testing Procedures ◽  
Type 1 Error ◽  
Sample Size Calculations

Magnitude based inference (MBI) was widely adopted by sport science researchers as an alternative to null hypothesis significance tests. It has been criticized for lacking a theoretical framework, mixing Bayesian and frequentist thinking, and encouraging researchers to run small studies with high Type 1 error rates. MBI terminology describes the position of confidence intervals in relation to smallest meaningful effect sizes. We show these positions correspond to combinations of one-sided tests of hypotheses about the presence or absence of meaningful effects, and formally describe MBI as a multiple decision procedure. MBI terminology operates as if tests are conducted at multiple alpha levels. We illustrate how error rates can be controlled by limiting each one-sided hypothesis test to a single alpha level. To provide transparent error control in a Neyman-Pearson framework and encourage the use of standard statistical software, we recommend replacing MBI with one-sided tests against smallest meaningful effects, or pairs of such tests as in equivalence testing. Researchers should pre-specify their hypotheses and alpha levels, perform a priori sample size calculations, and justify all assumptions. Our recommendations show researchers what tests to use and how to design and report their statistical analyses to accord with standard frequentist practice.

scholarly journals Permutation-based significance analysis reduces the type 1 error rate in bisulfite sequencing data analysis of human umbilical cord blood samples

2021 ◽  
Author(s):  
Essi Laajala ◽  
Viivi Halla-aho ◽  
Toni Grönroos ◽  
Ubaid Ullah ◽  
Mari Vähä-Mäkilä ◽  
...  
Keyword(s):  
Cord Blood ◽  
Umbilical Cord ◽  
Umbilical Cord Blood ◽  
Error Control ◽  
Bisulfite Sequencing ◽  
Differential Methylation ◽  
Gestational Weight ◽  
Type 1 Error ◽  
Analysis Workflow

Background: The aim of this study was to detect differential methylation in umbilical cord blood that is associated with maternal and pregnancy-related variables, such as maternal age and gestational weight gain. These have been studied earlier with 450K microarrays but not with bisulfite sequencing. Methods: Reduced representation bisulfite sequencing (RRBS) analysis was performed on 200 umbilical cord blood samples. Altogether 24 clinical and technical covariates were included in a binomial mixed effects model, which was fit separately for each high-coverage CpG site, followed by spatial and multiple testing adjustment of P values. Inflation of spatially adjusted P values was discovered in a permutation analysis, which was then applied for empirical type 1 error control. Results: Empirical type 1 error control decreased the number of findings associated with each covariate to zero or a small fraction of the number that would have been discovered with standard cutoffs. In this collection of samples, some differential methylation was associated with sex, the usage of epidural anesthetic during delivery, 1 minute Apgar points, maternal age and height, gestational weight gain, maternal smoking, and maternal insulin-treated diabetes, but not with the birth weight of the newborn infant, maternal pre-pregnancy BMI, the number of earlier miscarriages, the mode of delivery, labor induction, or the cosine transformed month of birth. Conclusions: The autocorrelation-adjusted Z-test is a convenient tool for detecting differentially methylated regions, but the significance should either be determined empirically or before the spatial adjustment. With appropriate significance thresholds, the detected differentially methylated regions were reproducible across studies, technologies, and statistical models. Our RRBS data analysis workflow is available in https://github.com/EssiLaajala/RRBS_workflow. Keywords: DNA methylation, bisulfite sequencing, RRBS, umbilical cord blood, pregnancy, sex, spatial correlation, type 1 error, differential methylation, analysis workflow

scholarly journals Rare variants association testing for a binary outcome when pooling individual level data from heterogeneous studies

2020 ◽  
Author(s):  
Tamar Sofer ◽  
Na Guo
Keyword(s):  
Genetic Variants ◽  
Error Control ◽  
Rare Variants ◽  
Score Test ◽  
Saddlepoint Approximation ◽  
Rare Allele ◽  
Association Testing ◽  
Type 1 Error ◽  
Rare Genetic Variants

AbstractWhole genome and exome sequencing studies are used to test the association of rare genetic variants with health traits. Many existing WGS efforts now aggregate data from heterogeneous groups, e.g. combining sets of individuals of European and African ancestries. We here investigate the statistical implications on rare variant association testing with a binary trait when combining together heterogeneous studies, defined as studies with potentially different disease proportion and different frequency of variant carriers. We study and compare in simulations the type 1 error control and power of the naïve Score test, the saddlepoint approximation to the score test (SPA test), and the BinomiRare test in a range of settings, focusing on low numbers of variant carriers. We show that type 1 error control and power patterns depend on both the number of carriers of the rare allele and on disease prevalence in each of the studies. We develop recommendations for association analysis of rare genetic variants. (1) The Score test is preferred when the case proportion in the sample is 50%. (2) Do not down-sample controls to balance case-control ratio, because it reduces power. Rather, use a test that controls the type 1 error. (3) Conduct stratified analysis in parallel with combined analysis. Aggregated testing may have lower power when the variant effect size differs between strata.

scholarly journals Too True to be Bad: When Sets of Studies with Significant and Non-Significant Findings Are Probably True

2017 ◽  
Author(s):  
Daniel Lakens ◽  
Alexander Etz
Keyword(s):  
Publication Bias ◽  
Error Rate ◽  
Statistical Power ◽  
Scientific Literature ◽  
Alternative Hypothesis ◽  
Meta Analysis ◽  
Likelihood Ratios ◽  
Type 1 Error

Psychology journals rarely publish non-significant results. At the same time, it is often very unlikely (or ‘too good to be true’) that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and non-significant results are likely to be true, or ‘too true to be bad’. As we show, mixed results are not only likely to be observed in lines of research, but when observed, mixed results often provide evidence for the alternative hypothesis, given reasonable levels of statistical power and an adequately controlled low Type 1 error rate. Researchers should feel comfortable submitting such lines of research with an internal meta-analysis for publication. A better understanding of probabilities, accompanied by more realistic expectations of what real lines of studies look like, might be an important step in mitigating publication bias in the scientific literature.

scholarly journals Too True to be Bad

2017 ◽  
Vol 8 (8) ◽  
pp. 875-881 ◽  
Author(s):  
Daniël Lakens ◽  
Alexander J. Etz
Keyword(s):  
Publication Bias ◽  
Error Rate ◽  
Statistical Power ◽  
Scientific Literature ◽  
Alternative Hypothesis ◽  
Meta Analysis ◽  
Likelihood Ratios ◽  
Type 1 Error

Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and nonsignificant results are likely to be true or “too true to be bad.” As we show, mixed results are not only likely to be observed in lines of research but also, when observed, often provide evidence for the alternative hypothesis, given reasonable levels of statistical power and an adequately controlled low Type 1 error rate. Researchers should feel comfortable submitting such lines of research with an internal meta-analysis for publication. A better understanding of probabilities, accompanied by more realistic expectations of what real sets of studies look like, might be an important step in mitigating publication bias in the scientific literature.

scholarly journals Multi-Ancestry Meta-Analysis yields novel genetic discoveries and ancestry-specific associations

2021 ◽  
Author(s):  
Patrick Turley ◽  
Alicia R. Martin ◽  
Grant Goldman ◽  
Hui Li ◽  
Masahiro Kanai ◽  
...  
Keyword(s):  
Error Rate ◽  
Genome Wide Association Study ◽  
Meta Analysis ◽  
East Asian ◽  
Genome Wide Association ◽  
European Ancestry ◽  
Summary Statistics ◽  
Type 1 Error ◽  
Genome Wide

ABSTRACTWe present a new method, Multi-Ancestry Meta-Analysis (MAMA), which combines genome-wide association study (GWAS) summary statistics from multiple populations to produce new summary statistics for each population, identifying novel loci that would not have been discovered in either set of GWAS summary statistics alone. In simulations, MAMA increases power with less bias and generally lower type-1 error rate than other multi-ancestry meta-analysis approaches. We apply MAMA to 23 phenotypes in East-Asian- and European-ancestry populations and find substantial gains in power. In an independent sample, novel genetic discoveries from MAMA replicate strongly.

Sign in / Sign up

Export Citation Format

Share Document