Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses of Atopic Dermatitis Treatments and Interventions

Dermatology ◽  
2021 ◽  
pp. 1-10
Author(s):  
Vanessa Lin ◽  
Raahi Patel ◽  
Alexis Wirtz ◽  
Deepika Mannem ◽  
Ryan Ottwell ◽  
...  
Keyword(s):  
Atopic Dermatitis ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Data Extraction ◽  
Included Study ◽  
Primary Objective ◽  
Quality Of Reporting ◽  
Study Characteristics ◽  
Meta Analyses

<b><i>Background:</i></b> Spin – the misrepresentation of a study’s results – has been identified in abstracts of studies focused on a variety of disorders from multiple fields of medicine. <b><i>Objectives:</i></b> This study’s primary objective was to evaluate the abstracts of systematic reviews and meta-analyses focused on the treatment of atopic dermatitis for the nine most severe forms of spin. <b><i>Methods:</i></b> We systematically searched Embase and MEDLINE for systematic reviews of atopic dermatitis therapies. Screening and data extraction occurred in a masked, duplicate fashion. Each included study was evaluated for the nine most severe types of spin and other study characteristics. <b><i>Results:</i></b> Our searches retrieved 2,456 studies, of which 113 were included for data extraction. Spin was found in 74.3% of our included studies (84/113). Spin type 6 occurred most frequently (68/113, 60.2%). Spin types 1, 2, and 9 were not identified. All industry-funded systematic reviews contained spin in their abstract. The presence of spin was not associated with any specific study characteristics, including the methodological quality of the study. <b><i>Conclusions:</i></b> Severe forms of spin were found in the majority of abstracts for systematic reviews of atopic dermatitis treatments. Steps should be taken to prevent spin to improve the quality of reporting in abstracts.

Evaluating the quality of systematic reviews and meta-analyses published on direct oral anticoagulants in the past 5 years

2020 ◽  
pp. bmjebm-2019-111326
Author(s):  
Ali Eshaghpour ◽  
Allen Li ◽  
Arshia Pedram Javidan ◽  
Natalie Chen ◽  
Sarah Yang ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Data Extraction ◽  
Oral Anticoagulants ◽  
Direct Oral Anticoagulants ◽  
Reporting Quality ◽  
Treatment And Prevention ◽  
Publication Number ◽  
Meta Analyses

Systematic reviews (SRs) have been reported with increasing frequency as a means of collating studies which may have been performed over different period of times, in different geographical areas and by different groups of investigators. As SRs have become more common, quality metrics such as Assessing the Methodological Quality of Systematic Reviews (AMSTAR) have become available for these reviews. AMSTAR is an 11-point checklist that assesses the methodological and reporting quality of a SR. In clinical practice, direct oral anticoagulants (DOACs) have been increasingly used for the treatment and prevention of both venous and arterial thromboembolism. We sought to evaluate the quality of SRs published on DOACs using the AMSTAR criteria. A comprehensive search of Medline, EMBASE and the Cochrane Database of Systematic Reviews from January 2013 to February 2019 was performed. Two reviewers independently screened titles and abstracts and subsequently full texts for eligibility. Data extraction was also completed in duplicate. Categories of extracted data included AMSTAR rating, journal of publication, year of publication, number of studies included in the SR, reporting adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, number of times the paper was cited and journal impact factor. A total of 3729 articles were identified, of which 250 were eligible for analysis. SR quality was highly variable with mean (SD) AMSTAR score of 5.68/11 (2.21). Reporting adherence to PRISMA guideline correlated with a moderate (5–8) or high quality (9–11) (OR=4.19, p<0.01) AMSTAR score. The methodological quality of DOACs was generally rated to be low-moderate, and improved adherence to AMSTAR methodological practices are strongly recommended.

scholarly journals Factors underpinning discrepant findings from reviews on the same topic: A systematic analysis of Cochrane and non-Cochrane reviews

2019 ◽  
Author(s):  
Claudia Hacke ◽  
David Nunan
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Data Extraction ◽  
Meta Analysis ◽  
List Type ◽  
Eligibility Criteria ◽  
Cochrane Reviews ◽  
And Performance ◽  
Meta Analyses

AbstractObjectiveTo explore factors underpinning discrepancies in reported pooled effect estimates from Cochrane and non-Cochrane systematic reviews answering the same question.Study Design and SettingWe observed discrepant pooled effects in 23 out of 24 pairs of meta-analyses from Cochrane and non-Cochrane systematic reviews answering the same question. Here we present the results of a systematic assessment of methodological quality and factors that explain the observed quantitative discrepancies. Methodological quality of each review was assessed using AMSTAR (Assessing the Methodological Quality of Systematic Reviews). Matched pairs were contrasted at the macro- (review methodology), meso- (application of methodology) and micro- (data extraction) level and reasons for differences were derived.ResultsAll Cochrane reviews had high methodological quality (AMSTAR 8-11), whereas the majority (87.5%) of non-Cochrane reviews were classified as moderate (AMSTAR 4-7). Only one pair included exactly the same studies for their respective meta-analyses but there was still a discrepancy in the pooled estimate due to differences in data extraction. One pair did not include any study of its match and for one pair the same effect estimates were reported despite inclusion of different studies. The remaining pairs included at least one study in their match. Due to insufficient reporting (predominantly affecting non-Cochrane reviews) we were only able to completely ascertain the reasons for discrepancies in all included studies for 9/24 (37.5%) pairs. Across all pairs, differences in pre-defined methods (macro-level) including search strategy, eligibility criteria and performance of dual screening could possibly explain mismatches in included studies. Study selection procedures (meso-level) including disagreements in the interpretation of pre-defined eligibility criteria (14 matches) were identified as reasons underpinning discrepant review findings. Comparison of data extraction from primary studies (micro-level) was not possible in 13/24 pairs as a result of the non-Cochrane review providing insufficient details of the studies included in their meta-analyses. Two out of 24 pairs completely agreed on the numerical data presented for the same studies in their respective meta-analysis. Both review types provided sufficient information to check the accuracy of data extraction for 8 pairs (45 studies) where there were discrepancies. An assessment of 50% (22 studies) of these showed that reasons for differences in extracted data could be identified in 15 studies. We found examples for both types of review where data presented were discrepant from that given in the source study without a plausible explanation.ConclusionMethodological and author judgements and performance are key aspects underpinning poor overlap of included studies and discrepancies in reported pooled effect estimates between topic-matched reviews. Though caution must be taken when extrapolating, our findings raise the question as to what extent the entire meta-analysis evidence-base accurately reflects the available primary research both in terms of volume and data. Reinforcing awareness of the application of guidelines for systematic reviews and meta-analyses may help mitigate some of the key issues identified in our analysis.What is new?Key findings Non-Cochrane reviews were of a lower overall methodological quality compared with Cochrane reviews. Discrepant results of meta-analyses on the same topic can be attributed to differences in included studies based on review author decision, judgements and performance at different stages of the review process.What this adds to what was known?This study provides the most robust analysis to date of the potential methodological factors underpinning discrepant review findings between matched meta-analyses answering the same question. Assessing differences between reviews at the macro-, meso-, and micro-levels is a useful method to identify reasons for discrepant meta-analyses at key stages of the review process.What is the implication and what should change now?There is a need for a standardised approach to performing matched-pair analysis of meta-analyses and systematic reviews answering the same question. Our paper provides a base for this that can be refined by replication and expert consensus.

scholarly journals Quality of the Evidence Supporting the Role of Oral Nutritional Supplements in the Management of Malnutrition: An Overview of Systematic Reviews and Meta-Analyses

2020 ◽  
Author(s):  
Christine Baldwin ◽  
Rosemary Smith ◽  
Michelle Gibbs ◽  
C Elizabeth Weekes ◽  
Peter W Emery
Keyword(s):  
At Risk ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Hospital Readmissions ◽  
Nutritional Supplement ◽  
Measurement Tool ◽  
High Quality ◽  
Oral Nutritional Supplements ◽  
Meta Analyses

ABSTRACT There is considerable heterogeneity across the findings of systematic reviews of oral nutritional supplement (ONS) interventions, presenting difficulties for healthcare decision-makers and patients alike. It is not known whether heterogeneity arises from differences in patient populations or relates to methodological rigor. This overview aimed to collate and compare findings from systematic reviews of ONSs compared with routine care in adult patients who were malnourished or at risk of malnutrition with any clinical condition and to examine their methodological quality. Three electronic databases were searched to July 2019, supplemented with hand-searching. Data on all outcomes were extracted and review methodological quality assessed using A MeaSurement Tool for Assessment of systematic Reviews (AMSTAR). Twenty-two reviews were included, 11 in groups from mixed clinical backgrounds and 11 in specific clinical conditions. Ninety-one meta-analyses were identified for 12 different outcomes but there was discordance between results. Significant benefits of ONSs were reported in 4 of 4 analyses of energy intake, 7 of 11 analyses of body weight, 7 of 22 analyses of mortality, 10 of 17 analyses of complications (total and infectious), 1 of 3 analyses of muscle strength, 4 of 9 analyses of body composition/nutritional status, 2 of 14 analyses of length of stay, and 2 of 5 analyses of hospital readmissions. Ten reviews were high quality (AMSTAR scores 8–11), 9 moderate (AMSTAR scores 3–8), and 3 poor (AMSTAR scores 0–3). Methodological deficiencies were limitations to searches, poor reporting of heterogeneity, and failure to incorporate quality of evidence into any recommendations. Discordance between reviews was not markedly reduced when only high-quality reviews were considered. Evidence for the effects of ONS in malnourished patients or those who are at risk of malnutrition is uncertain, and discordance in results can arise from differences in clinical background of patients or the etiological basis of malnutrition.

scholarly journals A Critical Overview of Systematic Reviews and Meta-Analyses on Acupuncture for Poststroke Insomnia

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Jinke Huang ◽  
Manli Wu ◽  
Simin Liang ◽  
Xiaohui Qin ◽  
Min Shen ◽  
...  
Keyword(s):  
Quality Assessment ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Acupuncture Treatment ◽  
Descriptive Analysis ◽  
Control Group ◽  
Evidence Quality ◽  
Meta Analyses ◽  
Quality Evidence

Objectives. Acupuncture has increasingly been used for insomnia relief after stroke. We aimed to evaluate the methodological quality and summarize the evidence regarding the effectiveness of acupuncture for poststroke insomnia (PSI) from systematic reviews/meta-analyses (SRs/MAs). Methods. Eight databases were searched from inception through August 23, 2020. SRs/MAs on acupuncture treatment for PSI were included. Methodological quality assessment was performed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), and evidence quality assessment was performed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE). Results. Six SRs/MAs on acupuncture treatment for PSI were included. The AMSTAR-2 showed that the methodological quality of all included SRs/MAs was rated as critically low. According to the evaluation results of GRADE, 38.9% (7/18) of outcomes were rated as very low-quality evidence, 22.2% (4/18) were low-quality evidence, and 8.9% (7/18) were moderate-quality evidence. Descriptive analysis results showed that acupuncture was an effective treatment modality for PSI. Conclusions. All included reviews indicated that acupuncture was more effective than the control group for the treatment of PSI, but the credibility of the results is limited owing to the generally low methodological and evidence quality of the included SRs/MAs. More high-quality evidence is needed to determine whether acupuncture is more effective than other treatments.

scholarly journals Quality assessment of systematic reviews and meta-analyses that examine preventive antibiotic uses and management practices designed to prevent disease in livestock

2019 ◽  
Vol 20 (2) ◽  
pp. 305-318
Author(s):  
Rachael Vriezen ◽  
Jan M. Sargeant ◽  
Ellen Vriezen ◽  
Charlotte B. Winder ◽  
Annette M. O'Connor
Keyword(s):  
Quality Assessment ◽  
Systematic Reviews ◽  
Management Practices ◽  
Disease Risk ◽  
Data Extraction ◽  
A Priori ◽  
Antibiotic Use ◽  
Study Selection ◽  
Meta Analyses

AbstractTo implement effective stewardship in food animal production, it is essential that producers and veterinarians are aware of preventive interventions to reduce illness in livestock. Systematic reviews and meta-analyses (SR/MA) provide transparent, replicable, and quality-assessed overviews. At present, it is unknown how many SR/MA evaluate preventive antibiotic use or management practices aimed at reducing disease risk in animal agriculture. Further, the quality of existing reviews is unknown. Our aim was to identify reviews investigating these topics and to provide an assessment of their quality. Thirty-eight relevant reviews were identified. Quality assessment was based on the AMSTAR 2 framework for the critical appraisal of systematic reviews. The quality of most of the reviews captured was classified as critically low (84.2%, n = 32/38), and only a small percentage of the evaluated reviews did not contain critical weaknesses (7.9%, n = 3/38). Particularly, a small number of reviews reported the development of an a priori protocol (15.8%, n = 6/38), and few reviews stated that key review steps were conducted in duplicate (study selection/screening: 26.3%, n = 10/38; data extraction: 15.8%, n = 6/38). The development of high-quality reviews summarizing evidence on approaches to antibiotic reduction is essential, and thus greater adherence to quality conduct guidelines for synthesis research is crucial.

scholarly journals Systematic Reviews and Meta-Analysis in Spine Surgery—How Good Are They in Methodological Quality? A Systematic Review

2020 ◽  
pp. 219256822090681 ◽  
Author(s):  
Muthu Sathish ◽  
Ramakrishnan Eswar
Keyword(s):  
Systematic Review ◽  
Spine Surgery ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Meta Analysis ◽  
Eligibility Criteria ◽  
Cochrane Database ◽  
Critical Flaw ◽  
Meta Analyses

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

scholarly journals Quality of the Evidence Supporting the Role of Acupuncture for Stable Angina Pectoris: An Umbrella Review of Systematic Reviews

2021 ◽  
Vol 8 ◽  
Author(s):  
Min Shen ◽  
Jinke Huang ◽  
Tao Qiu
Keyword(s):  
Angina Pectoris ◽  
Systematic Reviews ◽  
Stable Angina ◽  
Methodological Quality ◽  
Reporting Quality ◽  
Stable Angina Pectoris ◽  
Quality Reporting ◽  
Evidence Quality ◽  
Meta Analyses

Background: To systematically appraise and synthesize evidence, we conducted an overview of systematic reviews/meta-analyses (SRs/MAs) on acupuncture for stable angina pectoris (SAP).Methods: Eight databases were searched for SRs/MAs of acupuncture on SAP. The methodological quality, reporting quality, and evidence quality were evaluated by Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system, respectively.Results: A total of seven published SRs/MAs met the inclusion criteria. According to the evaluation results of AMSTAR-2, two studies were considered as of moderate quality; the remaining five were considered as of very low quality. According to the evaluation results of the PRISMA checklist, only one study reported the checklist in its entirety, while others had reporting deficiencies. According to GRADE, a total of 18 outcome indicators extracted from the included studies were evaluated. The evidence quality was very low in three, low in three, moderate in eight, and high in four.Conclusion: Acupuncture may be beneficial for SAP from the currently published evidence. However, this conclusion must be interpreted cautiously due to the generally low methodological quality, reporting quality, and evidence quality of the included studies. More rigorous, more standardized and comprehensive SRs/MAs are needed to provide strong evidence for convincing conclusions.

Sign in / Sign up

Export Citation Format

Share Document