Reporting of the methodological quality of search strategies in orthodontic quantitative systematic reviews

2020 ◽  
Author(s):  
Danah AlMubarak ◽  
Nikolaos Pandis ◽  
Martyn T Cobourne ◽  
Jadbinder Seehra
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Search Strategy ◽  
Grey Literature ◽  
Reporting Quality ◽  
Search Strategies ◽  
Cochrane Library ◽  
Eligibility Criteria ◽  
Cochrane Reviews

Summary Background This study aimed to assess the reporting of the methodological quality of search strategies undertaken in orthodontic quantitative systematic reviews (SRs) and hence their reproducibility. Materials and methods A search of a single electronic database (Medline via PubMed) was undertaken to identify interventional orthodontic SRs with meta-analysis published within a 10-year period. The Cochrane Library of Systematic Reviews was also sourced. Full articles were reviewed by two assessors against the eligibility criteria. The reporting quality of each search strategy was assessed using a previously validated checklist with a score of 1 or 2 given for each of the eight items. Cumulative totals were calculated. Guided by previous research, the authors agreed the following cut-offs to categorize the overall level of quality: 8–10 (poor), 10–12 (fair), and greater than 13 (good). Results A total of 127 SRs were analysed. The overall median quality score for the reporting of the search strategy was 14 [interquartile range (IQR): 13–15]. Cochrane SRs and those originating in Europe received higher aggregate scores, whereas no difference was evident based on Prospero registration. The continent of the corresponding author predicated the overall score. Non-Cochrane reviews achieved lower overall scores compared to Cochrane reviews (−1.0, 95% confidence interval: −1.65, −0.34, P = 0.003). The most frequently searched database was EMBASE (N = 93) and the median number of authors was 5 (IQR 4–6). Authors of 26.8% of SRs searched the grey literature. Language restrictions were applied to the search strategies of 88 (69.3%) SRs. Conclusions The reporting quality of search strategies undertaken in orthodontic SRs is at a good level but differences between Cochrane and non-Cochrane reviews currently exist. The reporting of searching of the grey literature and application of no language restrictions can be improved.

scholarly journals Reporting of the Role of the Expert Searcher in Cochrane Reviews

2006 ◽  
Vol 1 (4) ◽  
pp. 3 ◽  
Author(s):  
Li Zhang ◽  
Margaret Sampson ◽  
Jessie McGowan
Keyword(s):  
Systematic Reviews ◽  
Search Strategy ◽  
Data Extraction ◽  
Cochrane Library ◽  
Eligibility Criteria ◽  
Statistical Decision ◽  
Cochrane Reviews ◽  
Statistical Decisions ◽  
Search Quality

Introduction - This study applied the principles of evidence based information practice to clarify the role of information specialists and librarians in the preparation of Cochrane systematic reviews and to determine whether information specialists impact the quality of searching in Cochrane systematic reviews. Objectives - This research project sought to determine how the contribution of the person responsible for searching in the preparation of Cochrane systematic reviews was reported; whether the contribution was recognized through authorship or acknowledgement; the qualifications of the searcher; and the association between the type of contributorship and characteristics of the search strategy, assessability, and the presence of certain types of errors. Methods - Data sources: The Cochrane Database of Systematic Reviews, The Cochrane Library 3 (2002). Inclusion criteria: The study included systematic reviews that met the following criteria: one or more sections of the Cochrane Highly Sensitive Search Strategy were utilised, primary studies were either randomised controlled trials (RCTs) or quasi-RCTs, and included and excluded studies were clearly identified. Data extraction: Two librarians assessed the searches for errors, establishing consensus on discordant ratings. Results - Of the 169 reviews screened for this project, 105 met all eligibility criteria. Authors fulfilled the searching role in 41.9% of reviews studied, acknowledged persons or groups in 13.3%, a combination in 9.5%, and the role was not reported in 35.2% of reviews. For the 78 reviews in which meta-analyses were performed, the positions of those responsible for statistical decisions were examined for comparative purposes. The statistical role was performed by an author in 47.4% of cases and unreported in the same number of cases. Insufficient analyzable data was obtained regarding professional qualifications (3/105 for searching, 2/78 for statistical decisions). Search quality was assessed for 66 searches across 74 reviews. In general, it was more possible to assess the search quality when the searcher role was reported. An association was found between the reporting of searcher role and the presence of a consequential error. There was no association between the number of consequential errors and how the contribution of the searcher was reported. Conclusions - Qualifications of the persons responsible for searching and statistical decision-making were poorly reported in Cochrane reviews, but more complete role reporting is associated with greater assessability of searches and fewer substantive errors in search strategies.

scholarly journals Search for unpublished data by systematic reviewers: an audit

BMJ Open ◽  
2017 ◽  
Vol 7 (10) ◽  
pp. e017737 ◽  
Author(s):  
Hedyeh Ziai ◽  
Rujun Zhang ◽  
An-Wen Chan ◽  
Nav Persaud
Keyword(s):  
Publication Bias ◽  
Systematic Reviews ◽  
Data Extraction ◽  
Grey Literature ◽  
Cochrane Library ◽  
Eligibility Criteria ◽  
Ovid Medline ◽  
Cochrane Reviews ◽  
Unpublished Studies ◽  
Cochrane Database

ObjectivesWe audited a selection of systematic reviews published in 2013 and reported on the proportion of reviews that researched for unpublished data, included unpublished data in analysis and assessed for publication bias.DesignAudit of systematic reviews.Data sourcesWe searched PubMed and Ovid MEDLINE In-Process & Other Non-Indexed Citations between 1 January 2013 and 31 December 2013 for the following journals:Journal of the American Medical Association,The British Medical Journal,Lancet,Annals of Internal Medicineand theCochrane Database of Systematic Reviews. We also searched the Cochrane Library and included 100 randomly selected Cochrane reviews.Eligibility criteriaSystematic reviews published in 2013 in the selected journals were included. Methodological reviews were excluded.Data extraction and synthesisTwo reviewers independently reviewed each included systematic review. The following data were extracted: whether the review searched for grey literature or unpublished data, the sources searched, whether unpublished data were included in analysis, whether publication bias was assessed and whether there was evidence of publication bias.Main findings203 reviews were included for analysis. 36% (73/203) of studies did not describe any attempt to obtain unpublished studies or to search grey literature. 89% (116/130) of studies that sought unpublished data found them. 33% (68/203) of studies included an assessment of publication bias, and 40% (27/68) of these found evidence of publication bias.ConclusionA significant fraction of systematic reviews included in our study did not search for unpublished data. Publication bias may be present in almost half the published systematic reviews that assessed for it. Exclusion of unpublished data may lead to biased estimates of efficacy or safety in systematic reviews.

scholarly journals Reporting quality of systematic review abstracts in articles hand and wrist pathology: a review

2019 ◽  
Vol 2 (2) ◽  
pp. 50-57
Author(s):  
Amanda Yang Shen ◽  
Robert S Ware ◽  
Tom J O'Donohoe ◽  
Jason Wasiak
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
English Language ◽  
Meta Analysis ◽  
Reporting Quality ◽  
Cochrane Library ◽  
Annual Basis ◽  
Cochrane Reviews ◽  
Abstract Reporting

Background: An increasing number of systematic reviews are published on an annual basis. Although perusal of the full text of articles is preferable, abstracts are sometimes relied upon to guide clinical decisions. Despite this, the abstracts of systematic reviews have historically been poorly reported. We evaluated the reporting quality of systematic review abstracts within hand and wrist pathology literature. Methods: We searched MEDLINE®, EMBASE and Cochrane Library from inception to December 2017 for systematic reviews in hand and wrist pathology using the 12-item PRISMA-A checklist to assess abstract reporting quality. Results: A total of 114 abstracts were included. Most related to fracture (38%) or arthritis (17%) management. Forty-seven systematic reviews (41%) included meta-analysis. Mean PRISMA-A score was 3.6/12 with Cochrane reviews having the highest mean score and hand-specific journals having the lowest. Abstracts longer than 300 words (mean difference [MD]: 1.43, 95% CI [0.74, 2.13]; p <0.001) and systematic reviews with meta-analysis (MD: 0.64, 95% CI [0.05, 1.22]; p = 0.034) were associated with higher scores. Unstructured abstracts were associated with lower scores (MD: –0.65, 95% CI [–1.28, –0.02]; p = 0.044). A limitation of this study is the possible exclusion of relevant studies that were not published in the English language. Conclusion: Abstracts of systematic reviews pertaining to hand and wrist pathology have been suboptimally reported as assessed by the PRISMA-A checklist. Improvements in reporting quality could be achieved by endorsement of PRISMA-A guidelines by authors and journals, and reducing constraints on abstract length.

scholarly journals Interventions for Treating Tennis Elbow: An Overview of Systematic Reviews in the Past Ten Years

2020 ◽  
Author(s):  
Xue Wang ◽  
Jun Xiong ◽  
Jun Yang ◽  
Ting Yuan
Keyword(s):  
Systematic Reviews ◽  
Comparative Studies ◽  
Methodological Quality ◽  
Tennis Elbow ◽  
Meta Analysis ◽  
Reporting Quality ◽  
Research Process ◽  
Cochrane Library ◽  
Different Types

Abstract purpose: Tennis elbow is a common orthopedic disease, and there are many ways to treat it. This overview aimed to summarize the evidence of different treatments for tennis elbows, so as to provide the best guidance for clinical treatment.Methods: Use computer to search CNKI, WanFang database, WeiPu database, CBM database, PubMed, Cochrane Library and Embase from the time of establishment to May 31, 2019.Te Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and latest Assessment of Multiple Systematic Reviews 2 (AMSTAR2) checklists were used to assess reporting characteristics and methodological quality, respectively.Results: A total of 37 references were included. Methodological quality and reporting quality were unsatisfactory. Methodological quality was generally low and many key items were not reported. Some research reports are of high quality, but there is no trial registration and protocol written in advance, which may lead to some bias in the research process. The most frequent problems included non-registration of study protocol, absence of a list of excluded studies, and unclear acknowledgment of conflicts of interests. The different types of interventions included have been shown to relieve pain, improve quality of life, and restore elbow function, but there has been a lack of comparative studies.Conclusion: The reporting and methodological quality of systematic reviews and meta-analysis studies were sub-optimal, which demands further improvement. Comparative studies of different types of interventions are needed to determine unclear.PROSPERO registration number: CRD42015017071

Methodological quality and reporting of systematic reviews in hand and wrist pathology

2017 ◽  
Vol 42 (8) ◽  
pp. 852-856 ◽  
Author(s):  
J. Wasiak ◽  
A. Y. Shen ◽  
R. Ware ◽  
T. J. O’Donohoe ◽  
C. M. Faggion
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Reporting Quality ◽  
Assessment Tools ◽  
Cochrane Library ◽  
Measurement Tool ◽  
Level Of Evidence ◽  
Methodological Assessment ◽  
Meta Analyses

The objective of this study was to assess methodological and reporting quality of systematic reviews in hand and wrist pathology. MEDLINE, EMBASE and Cochrane Library were searched from inception to November 2016 for relevant studies. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using a measurement tool to assess systematic reviews, the Assessment of Multiple Systematic Reviews (AMSTAR). Descriptive statistics and linear regression were used to identify features associated with improved methodological quality. A total of 91 studies were included in the analysis. Most reviews inadequately reported PRISMA items regarding study protocol, search strategy and bias and AMSTAR items regarding protocol, publication bias and funding. Systematic reviews published in a plastics journal, or which included more authors, were associated with higher AMSTAR scores. A large proportion of systematic reviews within hand and wrist pathology literature score poorly with validated methodological assessment tools, which may affect the reliability of their conclusions. Level of evidence: I

scholarly journals A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer

BMJ Open ◽  
2018 ◽  
Vol 8 (3) ◽  
pp. e020869 ◽  
Author(s):  
Marius Goldkuhle ◽  
Vikram M Narayan ◽  
Aaron Weigl ◽  
Philipp Dahm ◽  
Nicole Skoetz
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Grey Literature ◽  
High Impact ◽  
Impact Factors ◽  
High Impact Journal ◽  
Eligibility Criteria ◽  
Medical Journals ◽  
Cochrane Reviews ◽  
Meta Analyses

ObjectiveTo compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.DesignMethodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.Data sourcesCancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.Eligibility criteria for selecting studiesWe included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.ResultsWe included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95% CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95% CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95% CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95% CI 0.02 to 0.09), use individual patient data (RR 0.03; 95% CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95% CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0–143); high-impact journal SRs: 74.45 (0–652)).ConclusionsWhen comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.

scholarly journals Factors underpinning discrepant findings from reviews on the same topic: A systematic analysis of Cochrane and non-Cochrane reviews

2019 ◽  
Author(s):  
Claudia Hacke ◽  
David Nunan
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Data Extraction ◽  
Meta Analysis ◽  
List Type ◽  
Eligibility Criteria ◽  
Cochrane Reviews ◽  
And Performance ◽  
Meta Analyses

AbstractObjectiveTo explore factors underpinning discrepancies in reported pooled effect estimates from Cochrane and non-Cochrane systematic reviews answering the same question.Study Design and SettingWe observed discrepant pooled effects in 23 out of 24 pairs of meta-analyses from Cochrane and non-Cochrane systematic reviews answering the same question. Here we present the results of a systematic assessment of methodological quality and factors that explain the observed quantitative discrepancies. Methodological quality of each review was assessed using AMSTAR (Assessing the Methodological Quality of Systematic Reviews). Matched pairs were contrasted at the macro- (review methodology), meso- (application of methodology) and micro- (data extraction) level and reasons for differences were derived.ResultsAll Cochrane reviews had high methodological quality (AMSTAR 8-11), whereas the majority (87.5%) of non-Cochrane reviews were classified as moderate (AMSTAR 4-7). Only one pair included exactly the same studies for their respective meta-analyses but there was still a discrepancy in the pooled estimate due to differences in data extraction. One pair did not include any study of its match and for one pair the same effect estimates were reported despite inclusion of different studies. The remaining pairs included at least one study in their match. Due to insufficient reporting (predominantly affecting non-Cochrane reviews) we were only able to completely ascertain the reasons for discrepancies in all included studies for 9/24 (37.5%) pairs. Across all pairs, differences in pre-defined methods (macro-level) including search strategy, eligibility criteria and performance of dual screening could possibly explain mismatches in included studies. Study selection procedures (meso-level) including disagreements in the interpretation of pre-defined eligibility criteria (14 matches) were identified as reasons underpinning discrepant review findings. Comparison of data extraction from primary studies (micro-level) was not possible in 13/24 pairs as a result of the non-Cochrane review providing insufficient details of the studies included in their meta-analyses. Two out of 24 pairs completely agreed on the numerical data presented for the same studies in their respective meta-analysis. Both review types provided sufficient information to check the accuracy of data extraction for 8 pairs (45 studies) where there were discrepancies. An assessment of 50% (22 studies) of these showed that reasons for differences in extracted data could be identified in 15 studies. We found examples for both types of review where data presented were discrepant from that given in the source study without a plausible explanation.ConclusionMethodological and author judgements and performance are key aspects underpinning poor overlap of included studies and discrepancies in reported pooled effect estimates between topic-matched reviews. Though caution must be taken when extrapolating, our findings raise the question as to what extent the entire meta-analysis evidence-base accurately reflects the available primary research both in terms of volume and data. Reinforcing awareness of the application of guidelines for systematic reviews and meta-analyses may help mitigate some of the key issues identified in our analysis.What is new?Key findings Non-Cochrane reviews were of a lower overall methodological quality compared with Cochrane reviews. Discrepant results of meta-analyses on the same topic can be attributed to differences in included studies based on review author decision, judgements and performance at different stages of the review process.What this adds to what was known?This study provides the most robust analysis to date of the potential methodological factors underpinning discrepant review findings between matched meta-analyses answering the same question. Assessing differences between reviews at the macro-, meso-, and micro-levels is a useful method to identify reasons for discrepant meta-analyses at key stages of the review process.What is the implication and what should change now?There is a need for a standardised approach to performing matched-pair analysis of meta-analyses and systematic reviews answering the same question. Our paper provides a base for this that can be refined by replication and expert consensus.

PP109 Use Of Speech Recognition In Medical Reports: A Systematic Review

2018 ◽  
Vol 34 (S1) ◽  
pp. 107-107
Author(s):  
Thomas Poder ◽  
Véronique Déry ◽  
Jean-Francois Fisette
Keyword(s):  
Systematic Review ◽  
Speech Recognition ◽  
Systematic Reviews ◽  
Grey Literature ◽  
Medical Doctor ◽  
Cochrane Library ◽  
Medical Specialties ◽  
Medical Reports ◽  
End Use

Introduction:Speech recognition is increasingly used in medical reporting. The aim of this article is to identify in the literature the advantages and weaknesses of this technology, as well as barriers and facilitators to its implementation.Methods:A systematic review of systematic reviews has been conducted in PubMed, Scopus, Cochrane Library and Center for Reviews and Dissemination up to August 2017. The grey literature has also been consulted. The quality of systematic reviews has been assessed with the AMSTAR checklist. Inclusion criteria were to use speech recognition for medical reporting (front or back-end). A Survey has also been conducted in Quebec, Canada, to identify the dissemination of this technology in this province, as well as the factors of success or failure in its implementation.Results:Five systematic reviews were identified. These reviews indicated a high level of heterogeneity across studies. The quality of the studies reported was generally poor. Speech recognition is not as accurate as human transcription but can dramatically reduce the turnaround times for reporting. In front-end use, medical doctors need to spend more time for dictation and correction than with human transcription. With speech recognition, major errors can be up to three times more frequent. In back-end use, a potential increase in the productivity of transcriptionist is noted.Conclusions:Speech recognition offers some advantages for medical reporting, the main one being a reduction in turnaround times. However, these advantages are challenged by an increased burden for medical doctor and risks of additional errors in medical reports. It is also hard to identify for which medical specialties and which clinical activities the use of speech recognition will be the most beneficial.

scholarly journals Telemonitoring Interventions in COPD Patients: Overview of Systematic Reviews

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Xuanlin Li ◽  
Yang Xie ◽  
Hulei Zhao ◽  
Hailong Zhang ◽  
Xueqing Yu ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Exercise Capacity ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Grade System ◽  
Cochrane Library ◽  
Exacerbation Rate ◽  
Copd Patients

Objective. The role of telemonitoring interventions (TIs) for chronic obstructive pulmonary disease (COPD) has been studied in many systematic reviews (SRs) and meta-analyses (MAs), but robust conclusions have not been reached due to wide variations in scopes, qualities, and outcomes. The aim of this overview was to determine the effectiveness of TIs on COPD patients. Methods. PubMed, EMBASE, Web of Science, and Cochrane Library were searched for all reviews on the topic of TI in treating COPD from inception to July 8, 2019, without restrictions on language. According to the inclusion and exclusion criteria, the retrieved literature studies were screened to select SRs and MAs of randomized control trials (RCTs) that evaluated the effects of TIs in COPD patients. The methodological quality of SRs and MAs was assessed with the AMSTAR-2 tool, and the strength of evidence was assessed with the grades of recommendations, assessment, development, and evaluation (GRADE) system for concerned outcomes in terms of mortality, quality of life (SGRQ total scores), exercise capacity (6MWD), and exacerbation-related outcomes (hospitalizations, exacerbation rate, and emergency room visits). Results. Our overview included eight SRs and MAs published in 2011 to 2019, from 95 RCTs involving 10632 participants. After strict evaluation by the AMSTAR-2 tool, 75% of the SRs and MAs in this overview had either low or critically low methodological quality. The effects of TIs for COPD on mortality, quality of life, exercise capacity, and exacerbation-related outcomes are limited, and all of these outcomes scored either low or very low quality of evidence on the GRADE system. Conclusions. There might be insufficient evidence to support the effectiveness of TIs for COPD currently, but the results of this overview should be interpreted dialectically and prudently, and the role of TIs in COPD needs further exploration.

scholarly journals Systematic Reviews and Meta-Analysis in Spine Surgery—How Good Are They in Methodological Quality? A Systematic Review

2020 ◽  
pp. 219256822090681 ◽  
Author(s):  
Muthu Sathish ◽  
Ramakrishnan Eswar
Keyword(s):  
Systematic Review ◽  
Spine Surgery ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Meta Analysis ◽  
Eligibility Criteria ◽  
Cochrane Database ◽  
Critical Flaw ◽  
Meta Analyses

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

Sign in / Sign up

Export Citation Format

Share Document