A Systematic Review Of The Methodological Quality Of Network Meta-Analyses

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

Download Full-text

An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review

PLoS ONE ◽

10.1371/journal.pone.0121715 ◽

2015 ◽

Vol 10 (4) ◽

pp. e0121715 ◽

Cited By ~ 16

Author(s):

James D. Chambers ◽

Huseyin Naci ◽

Olivier J. Wouters ◽

Junhee Pyo ◽

Shalak Gunjal ◽

...

Keyword(s):

Systematic Review ◽

Methodological Quality ◽

Meta Analyses

Download Full-text

Efficacy of Oral Ginger (Zingiber officinale) for Dysmenorrhea: A Systematic Review and Meta-Analysis

Evidence-based Complementary and Alternative Medicine ◽

10.1155/2016/6295737 ◽

2016 ◽

Vol 2016 ◽

pp. 1-10 ◽

Cited By ~ 6

Author(s):

Chen X. Chen ◽

Bruce Barrett ◽

Kristine L. Kwekkeboom

Keyword(s):

Systematic Review ◽

Methodological Quality ◽

Weighted Mean Difference ◽

Meta Analysis ◽

Grey Literature ◽

Zingiber Officinale ◽

Mean Difference ◽

Significant Difference ◽

Meta Analyses

This systematic review examines the efficacy of oral ginger for dysmenorrhea. Key biomedical databases and grey literature were searched. We included randomized controlled trials comparing oral ginger against placebo or active treatment in women with dysmenorrhea. Six trials were identified. Two authors independently reviewed the articles, extracted data, and assessed risk of bias. Discrepancies were resolved by consensus with a third reviewer. We completed a narrative synthesis of all six studies and exploratory meta-analyses of three studies comparing ginger with placebo and two studies comparing ginger with a nonsteroidal anti-inflammatory drug (NSAID). Ginger appeared more effective for reducing pain severity than placebo. The weighted mean difference on a 10 cm visual analogue scale was 1.55 cm (favoring ginger) (95% CI 0.68 to 2.43). No significant difference was found between ginger and mefenamic acid (an NSAID). The standardized mean difference was 0 (95% CI −0.40 to 0.41). Available data suggest that oral ginger could be an effective treatment for menstrual pain in dysmenorrhea. Findings, however, need to be interpreted with caution because of the small number of studies, poor methodological quality of the studies, and high heterogeneity across trials. The review highlights the need for future trials with high methodological quality.

Download Full-text

P14.71 An assessment of the reporting and methodological quality of meningioma systematic reviews and meta-analyses

Neuro-Oncology ◽

10.1093/neuonc/noab180.178 ◽

2021 ◽

Vol 23 (Supplement_2) ◽

pp. ii51-ii52

Author(s):

A M George ◽

S Gupta ◽

S M Keshwara ◽

M A Mustafa ◽

C S Gillespie ◽

...

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Neurofibromatosis Type ◽

Risk Of Bias ◽

Measurement Tool ◽

High Quality ◽

Meta Analyses

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.

Download Full-text

Correction: An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review

PLoS ONE ◽

10.1371/journal.pone.0131953 ◽

2015 ◽

Vol 10 (7) ◽

pp. e0131953 ◽

Cited By ~ 1

Author(s):

James D. Chambers ◽

Huseyin Naci ◽

Olivier J. Wouters ◽

Junhee Pyo ◽

Shalak Gunjal ◽

...

Keyword(s):

Systematic Review ◽

Methodological Quality ◽

Meta Analyses

Download Full-text

Evaluating the Quality of the Iranian Systematic Reviews, Meta-Analyses, and Economic Evaluations in Healthcare from 2005 to 2015

Health Technology Assessment in Action ◽

10.18502/htaa.v3i1.5852 ◽

2021 ◽

Author(s):

Morteza Arab-Zozani ◽

Zahra Heidarifard ◽

Efat Jabarpour

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Methodological Quality ◽

Time Horizon ◽

Meta Analysis ◽

Economic Evaluations ◽

Evidence Based ◽

Economic Perspective ◽

Meta Analyses

Context: The number of studies on health is increasing rapidly worldwide and in Iran. Systematic review studies, meta-analyses, and economic evaluation are of great importance in evidence-based decision making because of their standing in the evidence-based pyramid. The purpose of this study was to evaluate the reporting and methodological quality of Iranian systematic reviews, meta-analysis studies and economic evaluations on healthcare. Evidence Acquisition: PubMed and Scopus databases were searched to find considered studies, including systematic reviews, meta analyses and economic evaluations published from 2005 to 2015. Because of the high volume of review studies, 10% of all systematic reviews and meta-analyses were selected as a random sample. Also, all economic evaluations were included. Articles were evaluated using checklists, including PRISMA, AMSTAR and QHES with a maximum score of 27, 11 and 100, respectively. The quality score for each criterion as well as the epidemiological and descriptive characteristics of all articles was determined. Data were analyzed using SPSS V. 16 software. Results: After searching the databases, 1084 systematic reviews and meta-analyses were obtained, 10% of which were included in the study. A total of 41 economic evaluations were also included. The mean scores of systematic reviews and meta-analyses based on PRISMA and AMSTAR checklists were 17.04 (5.35) and 5.42 (1.97), respectively, and 68.21 (12.44) for economic evaluations based on QHES. Only three systematic reviews and meta-analysis articles had recorded protocols and 85% of the studies included the terms “systematic review” and “meta-analysis” in their titles. Only one study had been updated. In addition, 81% of the systematic reviews and meta-analyses were published in specialized journals and 47% in Iranian journals. Financial resources and conflict of interests had been mentioned in 33% and 66% of the studies, respectively. Of the selected studies, 60% had evaluated the quality of the articles and 35% of the studies had assessed publication bias. In economic evaluations, 56% had used CEA analysis, 22% CUA analysis, 12% CBA analysis, and one study had used CMA analysis. Of these studies, 54% were model-based health economic studies and 12% were trial-based. The economic perspective was the health care system in most studies. Forty-four percent of the studies had a short time horizon of one year or less, whereas 33% had a lifetime horizon. Moreover, 68% of the studies showed sensitivity analysis and only 5 included the magnitude and direction of the bias. Conclusions: Overall, the reporting and methodological quality of the selected studies were estimated at a moderate level. Based on these results, it is recommended to adopt strategies to reduce preventable errors in studies. Having a primary plan and protocol and registering it as a systematic review can be an important factor in improving the quality of studies. Economic evaluations should also focus on issues, such as economic perspective, time horizon, available bias, and sensitivity analysis.

Download Full-text

Epidemiological characteristics and methodological quality of meta-analyses on diabetes mellitus treatment: a systematic review

Acta Endocrinologica ◽

10.1530/eje-16-0172 ◽

2016 ◽

Vol 175 (5) ◽

pp. 353-360 ◽

Cited By ~ 6

Author(s):

Xin Yin Wu ◽

Victor C K Lam ◽

Yue Feng Yu ◽

Robin S T Ho ◽

Ye Feng ◽

...

Keyword(s):

Diabetes Mellitus ◽

Systematic Review ◽

Publication Bias ◽

Methodological Quality ◽

Treatment Decision ◽

Poor Performance ◽

Meta Analyses ◽

Diabetes Mellitus Treatment ◽

Epidemiological Characteristics

Objective Well-conducted meta-analyses (MAs) are considered as one of the best sources of clinical evidence for treatment decision. MA with methodological flaws may introduce bias and mislead evidence users. The aim of this study is to investigate the characteristics and methodological quality of MAs on diabetes mellitus (DM) treatments. Design Systematic review. Methods Cochrane Database of Systematic Review and Database of Abstract of Reviews of Effects were searched for relevant MAs. Assessing methodological quality of systematic reviews (AMSTAR) tool was used to evaluate the methodological quality of included MAs. Logistic regression analysis was used to identify association between characteristics of MA and AMSTAR results. Results A total of 252 MAs including 4999 primary studies and 13,577,025 patients were included. Over half of the MAs (65.1%) only included type 2 DM patients and 160 MAs (63.5%) focused on pharmacological treatments. About 89.7% MAs performed comprehensive literature search and 89.3% provided characteristics of included studies. Included MAs generally had poor performance on the remaining AMSTAR items, especially in assessing publication bias (39.3%), providing lists of studies (19.0%) and declaring source of support comprehensively (7.5%). Only 62.7% MAs mentioned about harm of interventions. MAs with corresponding author from Asia performed less well in providing MA protocol than those from Europe. Conclusions Methodological quality of MA on DM treatments was unsatisfactory. There is considerable room for improvement, especially in assessing publication bias, providing lists of studies and declaring source of support comprehensively. Also, there is an urgent need for MA authors to report treatment harm comprehensively.

Download Full-text

A systematic review of network meta-analyses for pharmacological treatment of common mental disorders

Evidence-Based Mental Health ◽

10.1136/eb-2017-102718 ◽

2018 ◽

Vol 21 (1) ◽

pp. 7-11 ◽

Cited By ~ 7

Author(s):

Taryn Williams ◽

Dan J Stein ◽

Jonathan Ipser

Keyword(s):

Systematic Review ◽

Decision Making ◽

Anxiety Disorder ◽

Pharmacological Treatment ◽

Methodological Quality ◽

Clinical Decision Making ◽

Common Mental Disorders ◽

Clinical Decision ◽

Meta Analyses

QuestionNetwork meta-analyses (NMAs) of treatment efficacy across different pharmacological treatments help inform clinical decision-making, but their methodological quality may vary a lot depending also on the quality of the included primary studies. We therefore conducted a systematic review of NMAs of pharmacological treatment for common mental disorders in order to assess the methodological quality of these NMAs, and to relate study characteristics to the rankings of efficacy and tolerability.Study selection and analysisWe searched three databases for NMAs of pharmacological treatment used in major depression, generalised anxiety disorder (GAD), social anxiety disorder (SAD), post-traumatic stress disorder (PTSD), obsessive-compulsive disorder (OCD) and specific phobia.Studies were appraised using the International Society for Pharmacoeconomics and Outcomes Research checklist of good research practices for indirect-treatment-comparison and network-meta-analysis studies.FindingsTwenty NMAs were eligible for inclusion. The number of randomised controlled trials per NMA ranged from 11 to 234, and included between 801 to more than 26 000 participants. Overall, antidepressants were found to be efficacious and tolerable agents for several disorders based on rankings (45%) or statistical significance (55%). The majority of NMAs in this review adhered to guidelines by including a network diagram (70%), assessing consistency (75%), making use of a random effects model (75%), providing information on the model used to fit the data (75%) and adjusting for covariates (75%).ConclusionsThe 20 NMAs of depression and anxiety disorders, PTSD and/or OCD included in this review demonstrate some methodological strengths in comparison with the larger body of published NMAs for medical disorders, support current treatment guidelines and help inform clinical decision-making.

Download Full-text

Major Mistakes and Errors in the Use of Trial Sequential Analysis in Systematic Reviews or Meta-analyses – Protocol for a Systematic Review

10.21203/rs.3.rs-900530/v1 ◽

2021 ◽

Author(s):

Christian Gunge Riberholt ◽

Markus Harboe Olsen ◽

Joachim Birch Milan ◽

Christian Gluud

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Clinical Guidelines ◽

Methodological Quality ◽

Sequential Analysis ◽

Trial Sequential Analysis ◽

Type I ◽

Research Waste ◽

Meta Analyses

Abstract Background: Adequately conducted systematic reviews with meta-analyses are considered the highest level of evidence and thus directly defines many clinical guidelines. However, the risk of type I and II errors in meta-analyses are substantial. Trial Sequential Analysis is a method for controlling these risks. Erroneous use of the method might lead to research waste or misleading conclusions. Methods: The current protocol describes a systematic review aimed to identify common and major mistakes and errors in the use of Trial Sequential Analysis by evaluating published systematic reviews and meta-analyses that include this method. We plan to include all studies using Trial Sequential Analysis published from 2018 to 2021, an estimated 400 to 600 publications. We will search Medical Literature Analysis and Retrieval System Online (MEDLINE) and the Cochrane Database of Systematic Reviews (CDSR), including studies with all types of participants, interventions, and outcomes. The search will begin in July 2021. Two independent reviewers will screen titles and abstracts, include relevant full text articles, extract data from the studies into a predefined checklist, and evaluate the methodological quality of the study using the AMSTAR 2 (Assessing the methodological quality of systematic reviews). Discussion: This protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P). The identified mistakes and errors will form the basis of a reviewed guideline for the use of Trial Sequential Analysis. Appropriately controlling for type I and II errors might reduce research waste and improve quality and precision of the evidence that clinical guidelines are based upon.

Download Full-text