scholarly journals Risk of bias in Cochrane systematic reviews: assessments of risk related to attrition bias are highly inconsistent

2018 ◽  
Author(s):  
Andrija Babic ◽  
Ruzica Tokalic ◽  
João Amílcar Silva Cunha ◽  
Ivana Novak ◽  
Jelena Suto ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Risk Of Bias ◽  
Systematic Review Methodology ◽  
Attrition Bias ◽  
Included Trial ◽  
Cochrane Reviews ◽  
Cochrane Database ◽  
Golden Standard ◽  
Improve Reliability

AbstractBackgroundAn important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews (CSRs) are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in CSRs of interventions published in the Cochrane Database of Systematic Reviews (CDSR).MethodsWe analyzed CSRs published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many CSRs had different judgments for the same supporting explanations.ResultsIn the main analysis we included 10292 judgments and supporting explanations for attrition bias from 729 CSRs. We categorized supporting explanations for those judgments into four categories and we found that most of the supporting explanations were unclear. Numerical indicators for percent of attrition, as well as statistics related to attrition were judged very differently. One third of CSR authors had more than one category of supporting explanation; some had up to four different categories. Inconsistencies were found even with the number of judgments, names of risk of bias domains and different judgments for the same supporting explanations in the same CSR.ConclusionWe found very high inconsistency in methods of appraising risk of attrition bias in recent Cochrane reviews. Systematic review authors need clear guidance about different categories they should assess and judgments for those explanations. Clear instructions about appraising risk of attrition bias will improve reliability of the Cochrane’s risk of bias tool, help authors in making decisions about risk of bias and help in making reliable decisions in healthcare.

Updated Method Guidelines for Cochrane Musculoskeletal Group Systematic Reviews and Metaanalyses

2013 ◽  
Vol 41 (2) ◽  
pp. 194-205 ◽  
Author(s):  
Elizabeth A.T. Ghogomu ◽  
Lara J. Maxwell ◽  
Rachelle Buchbinder ◽  
Tamara Rader ◽  
Jordi Pardo Pardo ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Musculoskeletal Diseases ◽  
Cochrane Collaboration ◽  
Previous Method ◽  
Systematic Review Methodology ◽  
Quality Of Reporting ◽  
For Profit ◽  
Cochrane Reviews

The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit, international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. It is important that authors conducting CMSG reviews and the readers of our reviews be aware of and use updated, state-of-the-art systematic review methodology. One hundred sixty reviews have been published. Previous method guidelines for systematic reviews of interventions in the musculoskeletal field published in 2006 have been substantially updated to incorporate methodological advances that are mandatory or highly desirable in Cochrane reviews and knowledge translation advances. The methodological advances include new guidance on searching, new risk-of-bias assessment, grading the quality of the evidence, the new Summary of Findings table, and comparative effectiveness using network metaanalysis. Method guidelines specific to musculoskeletal disorders are provided by CMSG editors for various aspects of undertaking a systematic review. These method guidelines will help improve the quality of reporting and ensure high standards of conduct as well as consistency across CMSG reviews.

scholarly journals Introducing a special issue with a focus on systematic reviews

2016 ◽  
Vol 17 (1) ◽  
pp. 1-2 ◽  
Author(s):  
J. M. Sargeant ◽  
A. M. O'Connor
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Health Research ◽  
Animal Health ◽  
Risk Of Bias ◽  
Special Issue ◽  
Systematic Review Methodology ◽  
Review Section

AbstractSystematic reviews answer specific review questions by following structured steps and employing specific methods to reduce the risk of bias and to maximize transparency in the process of the review, and systematic review methodology differs from traditional narrative reviews in many ways. As a journal devoted to reviews, it is appropriate that Animal Health Research Reviews (AHRR) includes this approach to reviews of the literature. The aim of this special issue of AHRR was to illustrate the scope of articles that can be considered for submission to the systematic review section of this journal for prospective authors and readers.

THE TYPICAL COCHRANE REVIEW

2002 ◽  
Vol 18 (4) ◽  
pp. 820-823 ◽  
Author(s):  
Susan Mallett ◽  
Mike Clarke
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Cochrane Review ◽  
Descriptive Study ◽  
Median Number ◽  
Interquartile Range ◽  
Cochrane Library ◽  
Cochrane Reviews ◽  
Cochrane Database ◽  
The Cochrane Library

Objectives: To describe the number of trials and participants in a typical systematic review from The Cochrane Database of Systematic Reviews.Methods: The number of trials in 1,000 Cochrane systematic reviews in issue 1, 2001 of The Cochrane Database of Systematic Reviews was counted for three categories of trial: included trials, ongoing trials, and trials awaiting assessment for inclusion. (The term trial is used in this paper, although a small number of Cochrane reviews include studies that are not trials.) The total number of participants in included trials was extracted from a sample of reviews.Results: A total of 9,778 trials were included in the Cochrane reviews. There were a further 356 ongoing trials and 1,138 trials awaiting assessment for inclusion. A typical review contained six included trials. Forty percent of the reviews listed ongoing trials and/or trials awaiting assessment for inclusion. Based on a sample of 258 reviews, the median number of participants per review was 945 (interquartile range, 313 to 2,511) per review and 118 (interquartile range, 60 to 241) per trial.Conclusion: This report is a descriptive study of the number of trials and participants in a typical Cochrane review from The Cochrane Library, issue 1, 2001.

scholarly journals Interventions to improve healthcare workers’ hand hygiene compliance: A systematic review of systematic reviews

2018 ◽  
Vol 39 (12) ◽  
pp. 1449-1456 ◽  
Author(s):  
Lesley Price ◽  
Jennifer MacDonald ◽  
Lucyna Gozdzielewska ◽  
Tracey Howe ◽  
Paul Flowers ◽  
...  
Keyword(s):  
Systematic Review ◽  
Hand Hygiene ◽  
Systematic Reviews ◽  
Hand Hygiene Compliance ◽  
Educational Interventions ◽  
Risk Of Bias ◽  
Future Research ◽  
Monitoring Technology ◽  
Positive Effects ◽  
Hygiene Compliance

AbstractObjectiveTo synthesize the existing evidence base of systematic reviews of interventions to improve healthcare worker (HCW) hand hygiene compliance (HHC).MethodsPRISMA guidelines were followed, and 10 information sources were searched in September 2017, with no limits to language or date of publication, and papers were screened against inclusion criteria for relevance. Data were extracted and risk of bias was assessed.ResultsOverall, 19 systematic reviews (n=20 articles) were included. Only 1 article had a low risk of bias. Moreover, 15 systematic reviews showed positive effects of interventions on HCW HHC, whereas 3 reviews evaluating monitoring technology did not. Findings regarding whether multimodal rather than single interventions are preferable were inconclusive. Targeting social influence, attitude, self-efficacy, and intention were associated with greater effectiveness. No clear link emerged between how educational interventions were delivered and effectiveness.ConclusionsThis is the first systematic review of systematic reviews of interventions to improve HCW HHC. The evidence is sufficient to recommend the implementation of interventions to improve HCW HHC (except for monitoring technology), but it is insufficient to make specific recommendations regarding the content or how the content should be delivered. Future research should rigorously apply behavior change theory, and recommendations should be clearly described with respect to intervention content and how it is delivered. Such recommendations should be tested for longer terms using stronger study designs with clearly defined outcomes.

scholarly journals Assessments of risk of bias in systematic reviews of observational nutritional epidemiologic studies are often not appropriate or comprehensive: a methodological study

2021 ◽  
pp. e000248
Author(s):  
Dena Zeraatkar ◽  
Alana Kohut ◽  
Arrti Bhasin ◽  
Rita E Morassut ◽  
Isabella Churchill ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Epidemiologic Studies ◽  
Risk Of Bias ◽  
Selective Reporting ◽  
Methodological Study ◽  
Essential Component ◽  
Cochrane Database

BackgroundAn essential component of systematic reviews is the assessment of risk of bias. To date, there has been no investigation of how reviews of non-randomised studies of nutritional exposures (called ‘nutritional epidemiologic studies’) assess risk of bias.ObjectiveTo describe methods for the assessment of risk of bias in reviews of nutritional epidemiologic studies.MethodsWe searched MEDLINE, EMBASE and the Cochrane Database of Systematic Reviews (Jan 2018–Aug 2019) and sampled 150 systematic reviews of nutritional epidemiologic studies.ResultsMost reviews (n=131/150; 87.3%) attempted to assess risk of bias. Commonly used tools neglected to address all important sources of bias, such as selective reporting (n=25/28; 89.3%), and frequently included constructs unrelated to risk of bias, such as reporting (n=14/28; 50.0%). Most reviews (n=66/101; 65.3%) did not incorporate risk of bias in the synthesis. While more than half of reviews considered biases due to confounding and misclassification of the exposure in their interpretation of findings, other biases, such as selective reporting, were rarely considered (n=1/150; 0.7%).ConclusionReviews of nutritional epidemiologic studies have important limitations in their assessment of risk of bias.

scholarly journals Development of an efficient search filter to retrieve systematic reviews from PubMed

2021 ◽  
Vol 109 (4) ◽  
Author(s):  
José Antonio Salvador-Oliván ◽  
Gonzalo Marco-Cuenca ◽  
Rosario Arquero-Avilés
Keyword(s):  
Systematic Review ◽  
Decision Making ◽  
Systematic Reviews ◽  
Search Strategy ◽  
Average Precision ◽  
Search Filter ◽  
Search Terms ◽  
Mesh Terms ◽  
Cochrane Database ◽  
Relevant Publication

Objective: Locating systematic reviews is essential for clinicians and researchers when creating or updating reviews and for decision-making in health care. This study aimed to develop a search filter for retrieving systematic reviews that improves upon the performance of the PubMed systematic review search filter.Methods: Search terms were identified from abstracts of reviews published in Cochrane Database of Systematic Reviews and the titles of articles indexed as systematic reviews in PubMed. Both the precision of the candidate terms and the number of systematic reviews retrieved from PubMed were evaluated after excluding the subset of articles retrieved by the PubMed systematic review filter. Terms that achieved a precision greater than 70% and relevant publication types indexed with MeSH terms were included in the filter search strategy.Results: The search strategy used in our filter added specific terms not included in PubMed’s systematic review filter and achieved a 61.3% increase in the number of retrieved articles that are potential systematic reviews. Moreover, it achieved an average precision that is likely greater than 80%.Conclusions: The developed search filter will enable users to identify more systematic reviews from PubMed than the PubMed systematic review filter with high precision.

scholarly journals Pharmacogenetics of Type 2 diabetes mellitus: A Systematic Review Protocol (Preprint)

2019 ◽  
Author(s):  
Hamid Reza Aghaei Meybodi ◽  
Negar Sarhangi ◽  
Anoosh Naghavi ◽  
Marzieh Rahbaran ◽  
Maryam Hassani Doabsari ◽  
...  
Keyword(s):  
Systematic Review ◽  
Type 2 Diabetes ◽  
Data Extraction ◽  
Relevant Information ◽  
Risk Of Bias ◽  
Cochrane Database ◽  
Registration Number ◽  
Newcastle Ottawa Scale ◽  
Review Protocol

UNSTRUCTURED The objective of this systematic review is to determine the effect of genetic variants that associate with antidiabetic medications and their efficacy and toxicity in T2DM patients. The understanding may allow interventions for improving management of T2DM and later systematically evaluated in more in-depth studies. We will have performed a comprehensive search using PubMed, Scopus, EMBASE, Web of Sciences and Cochrane database from 1990 to 2018. Relevant journals and references of all included studies will be hand searched to find the additional studied. Eligible studies such as pharmacogenetics studies in terms of drug response and toxicity in the type 2 diabetes patients and performed just on human will be included. Data extraction and quality assessment will be carried out by two independent reviewers and disagreements will be resolved through third expert reviewer. Risk of bias will be assessed with the Cochrane Risk of Bias tool for randomized studies and Newcastle-Ottawa Scale (NOS) for observational Studies. Narrative synthesis will be conducted by the combination of key findings. The results of this study will be submitted to a peer-reviewed journal for publication and also presented at PROSPERO. We expect this review will provide highly relevant information for clinicians, pharmaceutical industry that will benefit from the summary of the best available data regarding the efficacy of antidiabetic medication in the aspect of pharmacogenetics. PROSPERO Registration number (CRD42018104843)

scholarly journals Methods of conduct and reporting of living systematic reviews: a protocol for a living methodological survey

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 221 ◽  
Author(s):  
Assem M. Khamis ◽  
Lara A. Kahale ◽  
Hector Pardo-Hernandez ◽  
Holger J. Schünemann ◽  
Elie A. Akl
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Evidence Synthesis ◽  
Cochrane Library ◽  
Cross Sectional Survey ◽  
Systematic Review Methodology ◽  
Cross Sectional ◽  
Prospective Longitudinal ◽  
Over Time

Background: The living systematic review (LSR) is an emerging approach for improved evidence synthesis that uses continual updating to include relevant new evidence as soon as it is published. The objectives of this study are to: 1) assess the methods of conduct and reporting of living systematic reviews using a living study approach; and 2) describe the life cycle of living systematic reviews, i.e., describe the changes over time to their methods and findings. Methods: For objective 1, we will begin by conducting a cross-sectional survey and then update its findings every 6 months by including newly published LSRs. For objective 2, we will conduct a prospective longitudinal follow-up of the cohort of included LSRs. To identify LSRs, we will continually search the following electronic databases: Medline, EMBASE and the Cochrane library. We will also contact groups conducting LSRs to identify eligible studies that we might have missed. We will follow the standard systematic review methodology for study selection and data abstraction. For each LSR update, we will abstract information on the following: 1) general characteristics, 2) systematic review methodology, 3) living approach methodology, 4) results, and 5) editorial and publication processes. We will update the findings of both the surveys and the longitudinal follow-up of included LSRs every 6 months. In addition, we will identify articles addressing LSR methods to be included in an ‘LSR methods repository’. Conclusion: The proposed living methodological survey will allow us to monitor how the methods of conduct, and reporting as well as the findings of LSRs change over time. Ultimately this should help with ensuring the quality and transparency of LSRs.

scholarly journals Additional Search Strategies May Not Be Necessary for a Rapid Systematic Review

2015 ◽  
Vol 10 (2) ◽  
pp. 150
Author(s):  
Joanne L. Jordan
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Reference Lists ◽  
Meta Analysis ◽  
Search Strategies ◽  
Risk Of Bias ◽  
Chronic Depression ◽  
Randomized Controlled ◽  
Central Database

A Review of: Westphal, A., Kriston, L., Hölzel, L.P., Härter, M., & von Wolff, A. (2014). Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of Public Health Research, 3(2), 177. doi: 10.4081/jphr.2014.177 Abstract Objective – To evaluate the efficiency and contribution of additional searching strategies for finding randomized controlled trials (RCTs) in a systematic review. Design – A methodological case study. Setting – Biomedical literature. Methods – A sensitive search (defined as “the ratio of the number of relevant reports identified to the total number of relevant reports in existence”) was conducted of electronic databases, Cochrane CENTRAL database, MEDLINE, EMBASE, PsycInfo, CINAHL, BIOSIS, and Web of Science databases (Science and Social Science Citation Indexes). The following additional searching strategies were conducted: hand-searching contents of relevant journals (Archives of General Psychiatry, Journal of Consulting and Clinical Psychology, and Journal of Affective Disorders), citation tracking (forwards tracking using Social Science and Science Citation Index and backwards tracking by looking through reference lists of included studies), screening reference lists of relevant systematic reviews, searching clinical trials registers (ClinicalTrials.gov and ICTRP registers), and contacting first authors of included studies to find any similar unpublished studies. The number of articles identified by each of these methods was recorded and screened for inclusion in the systematic review. The authors calculated what they labelled as the ‘efficiency’ of each searching strategy (the number of included studies identified by the search method as a proportion of the full text articles screened) and the ‘contribution’ of the search strategies (the ratio of included studies identified by that method to the final number of included studies in the systematic review). The methodological quality of each included study was assessed using the Cochrane Risk of Bias Tool, which is a critical appraisal checklist used to judge the study’s value in the systematic review. The meta-analysis in the systematic review was conducted with and without the studies identified by the additional searching strategies to assess their impact on the review’s findings. Main Results – In total 50 studies were identified, 42 from electronic database searches and 8 from additional search strategies. As illustrated by the results in Table 1, the most useful additional search strategy was screening reference lists of relevant systematic reviews. Journal hand-searching and contacting authors also contributed to the review. Of the eight studies identified by the additional search strategies none were judged to have a low risk of bias (four had high risk of bias and four were unclear). Of the 42 included studies from electronic searches only 11 were judged to have a low risk of bias, whereas 9 studies had a high risk of bias and 22 were unclear. Excluding the eight studies retrieved from additional search strategies in the systematic review meta-analysis did not influence the results on the effectiveness of the different interventions for chronic depression. These studies were found to be indexed correctly on the electronic databases, but were not identified in the initial search. Conclusion – Additional search strategies, especially screening reference lists of systematic reviews and hand-searching relevant journals, retrieved a substantial number of relevant studies for a systematic review of interventions for treating chronic depression. However, results of the review’s meta-analysis did not differ when these additional studies (rated as either high or unclear risk of bias) were not included and search methods were time consuming. It might be reasonable to rely on electronic searching strategies when resources for conducting a systematic review are limited or when doing a “rapid review.” The benefits and limitations of additional search strategies should be considered particularly when resources or time for conducting a systematic review are limited. If the electronic database search is sensitive and includes the Cochrane CENTRAL database additional search strategies may not be necessary, but these findings should be tested in other research areas.

Sign in / Sign up

Export Citation Format

Share Document