scholarly journals Reliability and External Validity of AMSTAR in Assessing Quality of TCM Systematic Reviews

2012 ◽  
Vol 2012 ◽  
pp. 1-7 ◽  
Author(s):  
Deying Kang ◽  
Yuxia Wu ◽  
Dan Hu ◽  
Qi Hong ◽  
Jialiang Wang ◽  
...  
Keyword(s):  
Construct Validity ◽  
Quality Assessment ◽  
Study Design ◽  
Systematic Reviews ◽  
External Validity ◽  
Methodological Quality ◽  
Interrater Agreement ◽  
The Individual ◽  
Design And Methods

Objective. The aim of this study is to measure the reliability and external validity of AMSTAR by applying it to a sample of TCM systematic reviews.Study Design and Methods. We tested the agreement, reliability, construct validity, and feasibility of AMSTAR through comparisons with OQAQ. Statistical analyses were performed by using SPSS 13.0.Results. A random of sample with 41 TCM systematic reviews was selected from a database. The interrater agreement of the individual items of AMSTAR was moderate with a mean kappa of 0.50 (95% CI: 0.26, 0.73). The ICC for AMSTAR against OQAQ (total score of 9 items, excluding item 10) was 0.87 (95% CI: 0.76, 0.93).Conclusions. Although there is room for improvement on few items, the new tool is reliable, valid, and easy to use for methodological quality assessment of systematic reviews on TCM.

scholarly journals Impact of Adjustment for Quality on Results of Metaanalyses of Diagnostic Accuracy

2007 ◽  
Vol 53 (2) ◽  
pp. 164-172 ◽  
Author(s):  
Mariska Leeflang ◽  
Johannes Reitsma ◽  
Rob Scholten ◽  
Anne Rutjes ◽  
Marcello Di Nisio ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Quality Assessment ◽  
Confidence Intervals ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Patient Characteristics ◽  
Clinical Decision ◽  
High Quality ◽  
Poor Reporting

Abstract Background: We examined whether and to what extent different strategies of defining and incorporating quality of included studies affect the results of metaanalyses of diagnostic accuracy. Methods: We evaluated the methodological quality of 487 diagnostic-accuracy studies in 30 systematic reviews with the QUADAS (Quality Assessment of Diagnostic-Accuracy Studies) checklist. We applied 3 strategies that varied both in the definition of quality and in the statistical approach to incorporate the quality-assessment results into metaanalyses. We compared magnitudes of diagnostic odds ratios, widths of their confidence intervals, and changes in a hypothetical clinical decision between strategies. Results: Following 2 definitions of quality, we concluded that only 70 or 72 of 487 studies were of “high quality”. This small number was partly due to poor reporting of quality items. None of the strategies for accounting for differences in quality led systematically to accuracy estimates that were less optimistic than ignoring quality in metaanalyses. Limiting the review to high-quality studies considerably reduced the number of studies in all reviews, with wider confidence intervals as a result. In 18 reviews, the quality adjustment would have resulted in a different decision about the usefulness of the test. Conclusions: Although reporting the results of quality assessment of individual studies is necessary in systematic reviews, reader wariness is warranted regarding claims that differences in methodological quality have been accounted for. Obstacles for adjusting for quality in metaanalyses are poor reporting of design features and patient characteristics and the relatively low number of studies in most diagnostic reviews.

scholarly journals Development and Validation of a Nutrition-Specific Set of Tools to Assess the Quality of Nutrition Studies

2020 ◽  
Vol 4 (Supplement_2) ◽  
pp. 1377-1377
Author(s):  
Karima Benkhedda ◽  
Stephen Brooks ◽  
Linda Greene-Finestone ◽  
Shannon Kelly ◽  
Amanda MacFarlane ◽  
...  
Keyword(s):  
Quality Assessment ◽  
Study Design ◽  
Systematic Reviews ◽  
A Priori ◽  
Reliability And Validity ◽  
Quality Appraisal ◽  
Case Control Studies ◽  
Perfect Agreement ◽  
Development And Validation

Abstract Objectives To develop and validate a set of 3 quality assessment instruments (QAls) for evaluating the quality of nutrition studies, for each of the commonly used study designs: (1) randomized controlled trials (RCTs), (2) prospective cohort, and (3) case-control studies. Methods The QAI development and validation process included 8 steps: 1) identify and evaluate existing general QAls for adaptation with nutrition-specific quality appraisal items; 2) scan the literature to identify nutrition-specific quality appraisal issues; 3) generate nutrition-specific items to be added to each of the general QAIs, adapt existing guidance for general items for nutrition applications and develop guidance for added nutrition items; 4) review, by two experts in clinical and population nutrition, of the modified general QAIs with added nutrition-specific items and guidance; 5) assess reliability and validity of the QAI for each study design; 6) improve the usability and feasibility, of the QAIs by considering feedback from the validation exercise to refine the wording of the guidance; 7) develop a worksheet to help evaluate, a priori, topic-specific methodology to address risk of bias; and  8) validate the final QAIs using five peer-reviewed studies identified from published systematic reviews with reported quality assessment. Agreement and reliability were determined for each QAI. Results Results of the validation show good to perfect agreement among evaluators for the overall study rating and across domains. When compared to the study quality assessment reported in the systematic review, nutrition- specific items had the greatest impact on study ratings, generally resulting in a downgrade of the overall rating. Conclusions A set of nutrition-specific QAls were developed to assess the quality and robustness of nutrition studies. These tools incorporate general quality issues of study design and conduct, as well as address recognised nutrition study-specific issues. They will improve consistency in how nutrition studies are assessed particularly in nutrition-related systematic reviews. This will contribute to the overall quality of assessment of diet and Funding Sources This work was supported by Health Canada.

scholarly journals A Critical Overview of Systematic Reviews and Meta-Analyses on Acupuncture for Poststroke Insomnia

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Jinke Huang ◽  
Manli Wu ◽  
Simin Liang ◽  
Xiaohui Qin ◽  
Min Shen ◽  
...  
Keyword(s):  
Quality Assessment ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Acupuncture Treatment ◽  
Descriptive Analysis ◽  
Control Group ◽  
Evidence Quality ◽  
Meta Analyses ◽  
Quality Evidence

Objectives. Acupuncture has increasingly been used for insomnia relief after stroke. We aimed to evaluate the methodological quality and summarize the evidence regarding the effectiveness of acupuncture for poststroke insomnia (PSI) from systematic reviews/meta-analyses (SRs/MAs). Methods. Eight databases were searched from inception through August 23, 2020. SRs/MAs on acupuncture treatment for PSI were included. Methodological quality assessment was performed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), and evidence quality assessment was performed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE). Results. Six SRs/MAs on acupuncture treatment for PSI were included. The AMSTAR-2 showed that the methodological quality of all included SRs/MAs was rated as critically low. According to the evaluation results of GRADE, 38.9% (7/18) of outcomes were rated as very low-quality evidence, 22.2% (4/18) were low-quality evidence, and 8.9% (7/18) were moderate-quality evidence. Descriptive analysis results showed that acupuncture was an effective treatment modality for PSI. Conclusions. All included reviews indicated that acupuncture was more effective than the control group for the treatment of PSI, but the credibility of the results is limited owing to the generally low methodological and evidence quality of the included SRs/MAs. More high-quality evidence is needed to determine whether acupuncture is more effective than other treatments.

scholarly journals Methodological quality of meta-analyses indexed in PsycINFO: leads for enhancements: a meta-epidemiological study

BMJ Open ◽  
2020 ◽  
Vol 10 (8) ◽  
pp. e036349
Author(s):  
Victoria Leclercq ◽  
Charlotte Beaudart ◽  
Sara Ajamieh ◽  
Ezio Tirelli ◽  
Olivier Bruyère
Keyword(s):  
Epidemiological Study ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Positive Influence ◽  
Measurement Tool ◽  
Data Collection And Analysis ◽  
Literature Search Strategy ◽  
The Individual ◽  
Meta Analyses

ObjectivesMeta-analyses (MAs) are often used because they are lauded to provide robust evidence that synthesises information from multiple studies. However, the validity of MA conclusions relies on the procedural rigour applied by the authors. Therefore, this meta-research study aims to characterise the methodological quality and meta-analytic practices of MAs indexed in PsycINFO.DesignA meta-epidemiological study.ParticipantsWe evaluated a random sample of 206 MAs indexed in the PsycINFO database in 2016.Primary and secondary outcomesTwo authors independently extracted the methodological characteristics of all MAs and checked their quality according to the 16 items of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR2) tool for MA critical appraisal. Moreover, we investigated the effect of mentioning Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) on the methodological quality of MAs.ResultsAccording to AMSTAR2 criteria, 95% of the 206 MAs were rated as critically low quality. Statistical methods were appropriate and publication bias was well evaluated in 87% and 70% of the MAs, respectively. However, much improvement is needed in data collection and analysis: only 11% of MAs published a research protocol, 44% had a comprehensive literature search strategy, 37% assessed and 29% interpreted the risk of bias in the individual included studies, and 11% presented a list of excluded studies. Interestingly, the explicit mentioning of PRISMA suggested a positive influence on the methodological quality of MAs.ConclusionThe methodological quality of MAs in our sample was critically low according to the AMSTAR2 criteria. Some efforts to tremendously improve the methodological quality of MAs could increase their robustness and reliability.

scholarly journals Overview of “Systematic Reviews” of the Built Environment’s Effects on Mental Health

2020 ◽  
Vol 2020 ◽  
pp. 1-10 ◽  
Author(s):  
Solange Núñez-González ◽  
J. Andrés Delgado-Ron ◽  
Christopher Gault ◽  
Adriana Lara-Vinueza ◽  
Denisse Calle-Celi ◽  
...  
Keyword(s):  
Mental Health ◽  
Psychological Distress ◽  
Built Environment ◽  
Quality Assessment ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Well Being ◽  
Evidence Base ◽  
Health And Well Being

Good mental health is related to mental and psychological well-being, and there is growing interest in the potential role of the built environment on mental health, yet the evidence base underpinning the direct or indirect effects of the built environment is not fully clear. The aim of this overview is to assess the effect of the built environment on mental health-related outcomes. Methods. This study provides an overview of published systematic reviews (SRs) that assess the effect of the built environment on mental health. We reported the overview according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Databases searched until November 2019 included the Cochrane Database of Systematic Reviews, EMBASE, MEDLINE (OVID 1946 to present), LILACS, and PsycINFO. Two authors independently selected reviews, extracted data, and assessed the methodological quality of included reviews using the Assessing Methodological Quality of Systematic Reviews-2 (AMSTAR-2). Results. In total, 357 records were identified from a structured search of five databases combined with the references of the included studies, and eleven SRs were included in the narrative synthesis. Outcomes included mental health and well-being, depression and stress, and psychological distress. According to AMSTAR-2 scores, the quality assessment of the included SRs was categorized as “high” in two SRs and as “critically low” in nine SRs. According to the conclusions of the SRs reported by the authors, only one SR reported a “beneficial” effect on mental health and well-being outcomes. Conclusion. There was insufficient evidence to make firm conclusions on the effects of built environment interventions on mental health outcomes (well-being, depression and stress, and psychological distress). The evidence collected reported high heterogeneity (outcomes and measures) and a moderate- to low-quality assessment among the included SRs.

scholarly journals An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics

PLoS ONE ◽  
2021 ◽  
Vol 16 (2) ◽  
pp. e0246080
Author(s):  
Chen Min ◽  
Mi Xue ◽  
Fei Haotian ◽  
Li Jialian ◽  
Zhang Lingli
Keyword(s):  
Systematic Review ◽  
Health Economic ◽  
Economic Evaluation ◽  
Quality Assessment ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Biomedical Literature ◽  
Assessment Criteria ◽  
Economic Research

Background The systematic review of economic evaluations plays a critical role in making well-informed decisions about competing healthcare interventions. The quality of these systematic reviews varies due to the lack of internationally recognized methodological evaluation standards. Methods Nine English and Chinese databases including the Cochrane Library, PubMed, EMbase (Ovid), NHS economic evaluation database (NHSEED) (Ovid), Health Technology Assessment (HTA) database, Chinese National Knowledge Infrastructure (CNKI), WangFang, VIP Chinese Science & Technology Periodicals (VIP) and Chinese Biomedical Literature Database (CBM) were searched. Two reviewers independently screened studies and extracted data. The methodological quality of the literature was measured with modified AMSTAR. Data were narrative synthesized. Results 165 systematic reviews were included. The overall methodological quality of the literature was moderate according to the AMSTAR scale. In these articles, thirteen quality assessment tools and 32 author self-defined criteria were used. The three most widely used tools were the Drummond checklist (19.4%), the BMJ checklist (15.8%), the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement (12.7%). Others included the Quality of Health Economic Studies (QHES), the Consensus on Health Economic Criteria (CHEC), the checklist of Center for Reviews and Dissemination (CRD), the Philips checklist, the World Health Organization (WHO) checklist, the checklist of Critical Appraisal Skills Program (CASP), the Pediatric Quality Appraisal Questionnaire (PQAQ), the Joanna Briggs Institute (JBI) checklist, Spanish and Chinese guidelines. The quantitative scales used in these literature were the QHES and PQAQ. Conclusions Evidence showed that pharmacoeconomic systematic reviews’ methodology remained to be improved, and the quality assessment criteria were gradually unified. Multiple scales can be used in combination to evaluate the quality of economic research in different settings and types.

Non-Vitamin K Antagonist Oral Anticoagulants in Pulmonary Embolism: An Overview of Systematic Reviews

2020 ◽  
Vol 26 (23) ◽  
pp. 2686-2691 ◽  
Author(s):  
Ioannis Doundoulakis ◽  
Christina Antza ◽  
Haralambos Karvounis ◽  
George Giannakoulas
Keyword(s):  
Pulmonary Embolism ◽  
Vitamin K ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Randomized Clinical Trials ◽  
Oral Anticoagulants ◽  
Vitamin K Antagonist ◽  
Language Restriction ◽  
Overview Of Systematic Reviews

Background: Anticoagulation in patients with pulmonary embolism. Objective: To identify how non-vitamin K antagonist oral anticoagulants are associated with multiple outcomes in patients with pulmonary embolism. Methods: We performed a systematic search of systematic reviews via multiple electronic databases from inception to August 19th, 2019, without language restriction. Two authors independently extracted data and assessed the methodological quality of the included systematic reviews using the ROBIS tool. Results: We found twelve systematic reviews. Eleven SRs collected their data from randomized clinical trials and one from observational studies. All the included studies were published between 2014 and 2019 in English. The methodological quality of the 12 systematic reviews was low to high. None of the systematic reviews, which are included in our overview of systematic reviews, has evaluated the overall quality of evidence outcome using the Grading of Recommendations Assessments, Development and Evaluation (GRADE) approach. Conclusion: This is the first effort to summarize evidence about non-vitamin K antagonist oral anticoagulants in an overview of systematic reviews focusing exclusively on patients with pulmonary embolism. The evidence suggests that the non-vitamin K antagonist oral anticoagulants seem to be more effective and safer than a dualdrug approach with LMWH- VKA.

Sign in / Sign up

Export Citation Format

Share Document