Detecting Publication Bias in Meta‐analyses: A Case Study of Fluctuating Asymmetry and Sexual Selection

1999 ◽  
Vol 154 (2) ◽  
pp. 220-233 ◽  
Author(s):  
A. Richard Palmer
Keyword(s):  
Sexual Selection ◽  
Publication Bias ◽  
Fluctuating Asymmetry ◽  
Meta Analyses

How to Detect Publication Bias in Psychological Research

2019 ◽  
Vol 227 (4) ◽  
pp. 261-279 ◽  
Author(s):  
Frank Renkewitz ◽  
Melanie Keiner
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Type I Error ◽  
Psychological Research ◽  
Type I ◽  
True Effect Size ◽  
Questionable Research Practices ◽  
True Effect ◽  
Meta Analyses

Abstract. Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates. Both of these problems lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect such bias in meta-analytic results. We present an evaluation of the performance of six of these tools. To assess the Type I error rate and the statistical power of these methods, we simulated a large variety of literatures that differed with regard to true effect size, heterogeneity, number of available primary studies, and sample sizes of these primary studies; furthermore, simulated studies were subjected to different degrees of publication bias. Our results show that across all simulated conditions, no method consistently outperformed the others. Additionally, all methods performed poorly when true effect sizes were heterogeneous or primary studies had a small chance of being published, irrespective of their results. This suggests that in many actual meta-analyses in psychology, bias will remain undiscovered no matter which detection method is used.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

Network vs. Pairwise Meta-Analyses: A Case Study of the Impact of an Evidence-Synthesis Paradigm on Value of Information Outcomes

2014 ◽  
Vol 32 (10) ◽  
pp. 995-1004 ◽  
Author(s):  
Zafar Zafari ◽  
Kristian Thorlund ◽  
J. Mark FitzGerald ◽  
Carlo A. Marra ◽  
Mohsen Sadatsafavi
Keyword(s):  
Value Of Information ◽  
Evidence Synthesis ◽  
Meta Analyses ◽  
The Impact

scholarly journals Leishmanicidal effects of resveratrol and its derivatives: a systematic review and meta-Analysis

2020 ◽  
Author(s):  
Nasrin Amiri Dashatan ◽  
Marzieh Ashrafmansouri ◽  
Mehdi Koushki ◽  
Nayebali Ahmadi
Keyword(s):  
Systematic Review ◽  
Publication Bias ◽  
Data Extraction ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Protective Effects ◽  
Random Effects Models ◽  
Important Health ◽  
Mean Differences ◽  
Meta Analyses

Abstract Background Leishmaniasis is one of the most important health problems worldwide. The evidence has suggested that resveratrol and its derivatives have anti-leishmanial effects; however, the results are inconsistent and inconclusive. The aim of this study was to assess the effect of resveratrol and its derivatives on the Leishmania viability through a systematic review and meta-analysis of available relevant studies. Methods The electronic databases PubMed, ScienceDirect, Embase, Web of Science and Scopus were queried between October 2000 and April 2020 using a comprehensive search strategy. The eligible articles selected and data extraction conducted by two reviewers. Mean differences of IC50 (concentration leading to reduction of 50% of Leishmania) for each outcome was calculated using random-effects models. Sensitivity analyses and prespecified subgroup were conducted to evaluate potential heterogeneity and the stability of the pooled results. Publication bias was evaluated using the Egger’s and Begg’s tests. We also followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines for this review. Results Ten studies were included in the meta-analysis. We observed that RSV and its derivatives had significant reducing effects on Leishmania viability in promastigote [24.02 µg/ml; (95% CI 17.1, 30.8); P < 0.05; I2 = 99.8%; P heterogeneity = 0.00] and amastigote [18.3 µg/ml; (95% CI 13.5, 23.2); P < 0.05; I2 = 99.6%; P heterogeneity = 0.00] stages of Leishmania. A significant publication bias was observed in the meta-analysis. Sensitivity analyses showed a similar effect size while reducing the heterogeneity. Subgroup analysis indicated that the pooled effects of leishmanicidal of resveratrol and its derivatives were affected by type of stilbenes and Leishmania species. Conclusions Our findings clearly suggest that the strategies for the treatment of leishmaniasis should be focused on natural products such as RSV and its derivatives. Further study is needed to identify the mechanisms mediating this protective effects of RSV and its derivatives in leishmaniasis.

scholarly journals Is Replication Possible without Fidelity?

2021 ◽  
Author(s):  
Michelle Renee Ellefson ◽  
Daniel Oppenheimer
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Experimental Psychology ◽  
Intervention Science ◽  
Experimental Protocols ◽  
Meta Analyses

Failure of replication attempts in experimental psychology might extend beyond p-hacking, publication bias or hidden moderators; reductions in experimental power can be caused by violations of fidelity to a set of experimental protocols. In this paper, we run a series of simulations to systematically explore how manipulating fidelity influences effect size. We find statistical patterns that mimic those found in ManyLabs style replications and meta-analyses, suggesting that fidelity violations are present in many replication attempts in psychology. Scholars in intervention science, medicine, and education have developed methods of improving and measuring fidelity, and as replication becomes more mainstream in psychology, the field would benefit from adopting such approaches as well.

Sign in / Sign up

Export Citation Format

Share Document