scholarly journals Methodological Quality Assessment of Systematic Review or Meta-Analysis Using AMSTAR-2: the Long-Term Effectiveness or Efficacy of Opioids for Chronic Non-Cancer Pain

2019 ◽  
Author(s):  
Qian Li ◽  
Ke Deng ◽  
Xiaoyuan Jiang ◽  
Huan Tao ◽  
Hui Liu ◽  
...  

Abstract Background:Systematic review or meta-analysis, the strong study design of high quality evidence, give inconsistent conclusion of long-term effectiveness or efficacy of opioids for chronic non-cancer pain. We appraised the methodological quality of systematic reviews or meta-analyses. Methods: We found the relevant systematic reviews or meta-analyses by searching Medline, EMBASE, the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, the International prospective register of systematic reviews, Psyc ARTICLES/OVID, the Chinese Bio-Medical Literature Database, the China National Knowledge Infrastructure, and the Wan Fang Data and VIP Database on March 1st, 2019. The methodological quality was assessed by A Measurement Tool to Assess Systematic Reviews-2(AMSTAR-2). Spearman correlation analysis and non-parametric tests were used to assess the association between quality and factors. Results: Twenty-one systematic reviews or meta-analyses were included in our study. One has no individual study. In terms of methodological quality, twelve reviews were critically low in overall confidence, four reviews were low, two reviews were moderate, two reviews were high. When referring to the systematic reviews or meta-analyses of relatively better methodological quality with more credible results and conclusions, the effectiveness or efficacy of opioids was small to questionable. Cochrane reviews performed better than non-Cochrane reviews in establishing prior protocol (100% vs 17%, P<0.05), providing an excluded studies list (100% vs 50%, P<0.05) and taking risk of bias into account when interpreting the results of the review (100% vs 75%, P<0.05). There was a strong correlation (ρ=0.526, P<0.05) between the impact factor of systematic reviews or meta-analyses in published journals and methodological quality. Conclusion The methodological quality of the included systematic reviews or meta-analyses is far from satisfactory and needs improvement, especially in establishing prior protocol and justifying significant deviations from the protocol, providing an excluded primary studies list, reporting the funding information of primary studies, and assessing the potential impact of risk of bias on individual studies.

2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.


2021 ◽  
Vol 8 ◽  
Author(s):  
Jinke Huang ◽  
Xiaohui Qin ◽  
Min Shen ◽  
Yanjuan Xu ◽  
Yong Huang

Background: Tai chi (TC) is a popular form of exercise among adults with chronic heart failure (CHF), yet services are greatly underutilized. The aim of the current study was to identify and summarize the existing evidence and to systematically determine the clinical effectiveness of Tai Chi in the management of CHF using a systematic overview.Methods: Both English and Chinese databases were searched for systematic reviews (SRs)/meta-analyses (MAs) on TC for CHF from their inception to June 2020. The methodological quality, reporting quality, and risk of bias of SRs/MAs were assessed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, and Risk of Bias in Systematic reviews (ROBIS), respectively. The evidence quality of outcome measures was assessed by the Grades of Recommendations, Assessment, Development and Evaluation (GRADE).Results: Six SRs/MAs using a quantitative synthesis to assess various outcomes of TC in CHF were included in this overview. The methodological quality, reporting quality and risk of bias of the SRs/MAs and the evidence quality of the outcome measures are generally unsatisfactory. The limitations of the past SRs/MAs included the lack of either the protocol or registration, the list of excluded studies, and the computational details of meta-analysis were inadequately reported. The critical problems were that qualitative data synthesis relied on trials with small sample sizes and critical low quality.Conclusions: TC may be a promising complementary treatment for CHF. However, further rigorous and comprehensive SRs/MAs and RCTs are required to provide robust evidence for definitive conclusions.


2019 ◽  
Author(s):  
Claudia Hacke ◽  
David Nunan

AbstractObjectiveTo explore factors underpinning discrepancies in reported pooled effect estimates from Cochrane and non-Cochrane systematic reviews answering the same question.Study Design and SettingWe observed discrepant pooled effects in 23 out of 24 pairs of meta-analyses from Cochrane and non-Cochrane systematic reviews answering the same question. Here we present the results of a systematic assessment of methodological quality and factors that explain the observed quantitative discrepancies. Methodological quality of each review was assessed using AMSTAR (Assessing the Methodological Quality of Systematic Reviews). Matched pairs were contrasted at the macro- (review methodology), meso- (application of methodology) and micro- (data extraction) level and reasons for differences were derived.ResultsAll Cochrane reviews had high methodological quality (AMSTAR 8-11), whereas the majority (87.5%) of non-Cochrane reviews were classified as moderate (AMSTAR 4-7). Only one pair included exactly the same studies for their respective meta-analyses but there was still a discrepancy in the pooled estimate due to differences in data extraction. One pair did not include any study of its match and for one pair the same effect estimates were reported despite inclusion of different studies. The remaining pairs included at least one study in their match. Due to insufficient reporting (predominantly affecting non-Cochrane reviews) we were only able to completely ascertain the reasons for discrepancies in all included studies for 9/24 (37.5%) pairs. Across all pairs, differences in pre-defined methods (macro-level) including search strategy, eligibility criteria and performance of dual screening could possibly explain mismatches in included studies. Study selection procedures (meso-level) including disagreements in the interpretation of pre-defined eligibility criteria (14 matches) were identified as reasons underpinning discrepant review findings. Comparison of data extraction from primary studies (micro-level) was not possible in 13/24 pairs as a result of the non-Cochrane review providing insufficient details of the studies included in their meta-analyses. Two out of 24 pairs completely agreed on the numerical data presented for the same studies in their respective meta-analysis. Both review types provided sufficient information to check the accuracy of data extraction for 8 pairs (45 studies) where there were discrepancies. An assessment of 50% (22 studies) of these showed that reasons for differences in extracted data could be identified in 15 studies. We found examples for both types of review where data presented were discrepant from that given in the source study without a plausible explanation.ConclusionMethodological and author judgements and performance are key aspects underpinning poor overlap of included studies and discrepancies in reported pooled effect estimates between topic-matched reviews. Though caution must be taken when extrapolating, our findings raise the question as to what extent the entire meta-analysis evidence-base accurately reflects the available primary research both in terms of volume and data. Reinforcing awareness of the application of guidelines for systematic reviews and meta-analyses may help mitigate some of the key issues identified in our analysis.What is new?Key findings Non-Cochrane reviews were of a lower overall methodological quality compared with Cochrane reviews. Discrepant results of meta-analyses on the same topic can be attributed to differences in included studies based on review author decision, judgements and performance at different stages of the review process.What this adds to what was known?This study provides the most robust analysis to date of the potential methodological factors underpinning discrepant review findings between matched meta-analyses answering the same question. Assessing differences between reviews at the macro-, meso-, and micro-levels is a useful method to identify reasons for discrepant meta-analyses at key stages of the review process.What is the implication and what should change now?There is a need for a standardised approach to performing matched-pair analysis of meta-analyses and systematic reviews answering the same question. Our paper provides a base for this that can be refined by replication and expert consensus.


2020 ◽  
pp. 219256822090681 ◽  
Author(s):  
Muthu Sathish ◽  
Ramakrishnan Eswar

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.


2014 ◽  
Vol 133 (3) ◽  
pp. 206-217 ◽  
Author(s):  
Valter Silva ◽  
Antonio Jose Grande ◽  
Alan Pedrosa Viegas de Carvalho ◽  
Ana Luiza Cabrera Martimbianco ◽  
Rachel Riera

CONTEXT AND OBJECTIVE: Overviews of Systematic Reviews (OoRs) are a new type of study in which multiple evidence from systematic reviews (SRs) is compiled into an accessible and useful document. The aim here was to describe the state of the art and critically assess Cochrane OoRs that have been published.DESIGN AND SETTING: Descriptive study conducted at a research center.METHODS: The OoRs identified through the filter developed in Part I of this study were evaluated in five domains: methodological quality; quality of evidence; implications for practice; general profile of OoRs; and length of work.RESULTS: All 13 OoRs included had high methodological quality. Some OoRs did not present sufficient data to judge the quality of evidence; using sensitivity analysis, the quality of evidence of the OoRs increased. Regarding implications for practice, 64% of the interventions were judged as beneficial or harmful, while 36% of them showed insufficient evidence for judgment. It is expected (with 95% confidence interval) that one OoR will include 9,462 to 64,469 patients, 9 to 29 systematic reviews and 80 to 344 primary studies, and assess 6 to 21 interventions; and that 50 to 92% of OoRs will produce meta-analysis. The OoRs generated 2 to 26 meta-analyses over a period of 18 to 31 months.CONCLUSION: The OoRs presented high methodological quality; the quality of evidence tended to be moderate/high; most interventions were judged to be beneficial/harmful; the mean length of work was 24 months. The OoR profile adds power to decision-making.


BMJ Open ◽  
2021 ◽  
Vol 11 (10) ◽  
pp. e049213
Author(s):  
Karla Morganna Pereira Pinto de Mendonça ◽  
Sean Collins ◽  
Tácito ZM Santos ◽  
Gabriela Chaves ◽  
Sarah Leite ◽  
...  

IntroductionButeyko method is recommended as a non-pharmacological treatment for people with asthma. Although the worldwide interest in the Buteyko method, there is a paucity of studies gathering evidence to support its use. Therefore, we aim to conduct a systematic review and meta-analysis to assess the effects of the Buteyko method in children and adults with asthma.Methods and analysisWe will search on Cochrane Central Register of Controlled Trials, MEDLINE, Embase, US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov and WHO International Clinical Trials Registry Platform for studies focusing on the Buteyko method for children and adults with asthma. The searches will be carried out in September 2021 from database’s inception to the present. We will include randomised controlled trials comparing Buteyko method alone with asthma education or inactive control intervention. There will be no restriction on language. Primary outcomes include quality of life, asthma symptoms and adverse events/side effects. Two review authors will independently screen the studies for inclusion and extract data. We will assess the quality of the included studies using the ‘Risk of Bias’ tool. The certainty of the evidence will be assessed using the GRADE approach. Data synthesis will be conducted using Review Manager software. Reporting of the review will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidance and the Cochrane Handbook for Systematic Reviews of Interventions.Ethics and disseminationThis study will assess and provide evidence for the use of the Buteyko method in people with asthma. We will analyse secondary data and this does not require ethics approval. The findings will be published in peer-reviewed journals, at relevant conferences and will be shared in plain language in social media. Moreover, the findings of this review could guide the direction of healthcare practice and research.PROSPERO registration numberCRD42020193132.


Author(s):  
Morteza Arab-Zozani ◽  
Zahra Heidarifard ◽  
Efat Jabarpour

Context: The number of studies on health is increasing rapidly worldwide and in Iran. Systematic review studies, meta-analyses, and economic evaluation are of great importance in evidence-based decision making because of their standing in the evidence-based pyramid. The purpose of this study was to evaluate the reporting and methodological quality of Iranian systematic reviews, meta-analysis studies and economic evaluations on healthcare. Evidence Acquisition: PubMed and Scopus databases were searched to find considered studies, including systematic reviews, meta analyses and economic evaluations published from 2005 to 2015. Because of the high volume of review studies, 10% of all systematic reviews and meta-analyses were selected as a random sample. Also, all economic evaluations were included. Articles were evaluated using checklists, including PRISMA, AMSTAR and QHES with a maximum score of 27, 11 and 100, respectively. The quality score for each criterion as well as the epidemiological and descriptive characteristics of all articles was determined. Data were analyzed using SPSS V. 16 software. Results: After searching the databases, 1084 systematic reviews and meta-analyses were obtained, 10% of which were included in the study. A total of 41 economic evaluations were also included. The mean scores of systematic reviews and meta-analyses based on PRISMA and AMSTAR checklists were 17.04 (5.35) and 5.42 (1.97), respectively, and 68.21 (12.44) for economic evaluations based on QHES. Only three systematic reviews and meta-analysis articles had recorded protocols and 85% of the studies included the terms “systematic review” and “meta-analysis” in their titles. Only one study had been updated. In addition, 81% of the systematic reviews and meta-analyses were published in specialized journals and 47% in Iranian journals. Financial resources and conflict of interests had been mentioned in 33% and 66% of the studies, respectively. Of the selected studies, 60% had evaluated the quality of the articles and 35% of the studies had assessed publication bias. In economic evaluations, 56% had used CEA analysis, 22% CUA analysis, 12% CBA analysis, and one study had used CMA analysis. Of these studies, 54% were model-based health economic studies and 12% were trial-based. The economic perspective was the health care system in most studies. Forty-four percent of the studies had a short time horizon of one year or less, whereas 33% had a lifetime horizon. Moreover, 68% of the studies showed sensitivity analysis and only 5 included the magnitude and direction of the bias. Conclusions: Overall, the reporting and methodological quality of the selected studies were estimated at a moderate level. Based on these results, it is recommended to adopt strategies to reduce preventable errors in studies. Having a primary plan and protocol and registering it as a systematic review can be an important factor in improving the quality of studies. Economic evaluations should also focus on issues, such as economic perspective, time horizon, available bias, and sensitivity analysis.


2021 ◽  
Author(s):  
Leonard TF Ho ◽  
Fiona YT Ke ◽  
Charlene HL Wong ◽  
Irene XY Wu ◽  
Andy KL Cheung ◽  
...  

Abstract Background: While well-conducted systematic reviews (SRs) can provide best evidence on the potential effectiveness of acupuncture, limitations on methodological rigour of SRs may impact trustworthiness of their conclusions. This cross-sectional study aimed to evaluate the methodological quality of a representative sample of SRs on acupuncture effectiveness.Methods: CDSR, MEDLINE, and EMBASE were searched for SRs on acupuncture. AMSTAR2 was applied for assessing methodological quality. Associations between bibliographical characteristics and methodological quality ratings were examined. Results: A total of 106 SRs were appraised. Only one (0.9%) SR was of high overall methodological quality, zero (0%) was of moderate-quality, six (5.7%) and 99 (93.4%) were of low-quality and critically low-quality respectively. Among appraised SRs, only ten (9.4%) provided an a priori protocol, four (3.8%) conducted a comprehensive literature search, five (4.7%) provided a list of excluded study, and six (5.7%) performed meta-analysis appropriately. Cochrane reviews, update reviews, reviews with corresponding authors from the America, and reviews that searched non-English databases had relatively higher overall quality. Conclusions: Methodological quality of SRs on acupuncture is unsatisfactory. Future reviewers should improve critical areas of publishing protocols, performing comprehensive search, providing a list of excluded studies with justifications for exclusion, and conducting meta-analysis appropriately.


2015 ◽  
Vol 16 (5) ◽  
pp. 781-791 ◽  
Author(s):  
Youngshin Song ◽  
Minhye Oh ◽  
Seyeon Park ◽  
Myouyun Park ◽  
Kyoungok Kim ◽  
...  

2020 ◽  
Vol 2020 ◽  
pp. 1-19
Author(s):  
Zipan Lyu ◽  
Zhongyu Huang ◽  
Fengbin Liu ◽  
Zhengkun Hou

Objective. To access the methodological and reporting quality of systematic reviews (SRs)/meta-analyses (MAs) about Chinese medical treatment for gastroesophageal reflux disease (GERD). Methods. The PubMed, Wanfang Data, China National Knowledge Infrastructure (CNKI), Chinese Science and Technology Periodical Database (VIP), Chinese Biomedical (CBM), Web of Science, and Cochrane Library databases were searched from inception to June 2020. Two researchers independently screened the literature considering the eligibility criteria. Overview Quality Assessment Questionnaire (OQAQ), Assessment of Multiple Systematic Reviews 2 (AMSTAR 2), and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used to assess the methodological and reporting quality of the included reports. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was used to evaluate the level of evidence in each report. Results. Thirty-three SRs/MAs met the inclusion criteria. The OQAQ results showed that defects in the methodological quality of 17/32 reports were major, with scores of 3 points. Analyzing a single item as the object, search strategies (item 2), and risk of bias in individual studies (item 4) was considered poor. The AMSTAR 2 results showed that 25.4% of the items were not reported, and 7.8% of the items were only partially reported. The overall assessment of AMSTAR 2 showed the majority of systematic reviews and meta-analyses were of low/very low (31/33, 93.9%) methodological quality, with a lack of protocol registration and excluded study list. The PRISMA results showed that 19.9% of items were not reported, and 15.2% of items were only partially reported, due to a lack of protocol registration and study selection methods. The methodological and reporting quality of the included studies was generally poor. Evidence evaluation with GRADE showed that most (31/33) of the included studies had low or very low levels of evidence. Conclusion. The methodological and reporting quality of SRs/MAs about Chinese medical treatment for GERD is generally poor. The main problems included incomplete search strategies, risk of bias in individual studies, the lack of protocol registration and excluded study list, and incorrect study selection methods.


Sign in / Sign up

Export Citation Format

Share Document