scholarly journals Low Methodological Quality of Systematic Reviews on Acupuncture: A Cross- Sectional Study

Author(s):  
Leonard TF Ho ◽  
Fiona YT Ke ◽  
Charlene HL Wong ◽  
Irene XY Wu ◽  
Andy KL Cheung ◽  
...  

Abstract Background: While well-conducted systematic reviews (SRs) can provide best evidence on the potential effectiveness of acupuncture, limitations on methodological rigour of SRs may impact trustworthiness of their conclusions. This cross-sectional study aimed to evaluate the methodological quality of a representative sample of SRs on acupuncture effectiveness.Methods: CDSR, MEDLINE, and EMBASE were searched for SRs on acupuncture. AMSTAR2 was applied for assessing methodological quality. Associations between bibliographical characteristics and methodological quality ratings were examined. Results: A total of 106 SRs were appraised. Only one (0.9%) SR was of high overall methodological quality, zero (0%) was of moderate-quality, six (5.7%) and 99 (93.4%) were of low-quality and critically low-quality respectively. Among appraised SRs, only ten (9.4%) provided an a priori protocol, four (3.8%) conducted a comprehensive literature search, five (4.7%) provided a list of excluded study, and six (5.7%) performed meta-analysis appropriately. Cochrane reviews, update reviews, reviews with corresponding authors from the America, and reviews that searched non-English databases had relatively higher overall quality. Conclusions: Methodological quality of SRs on acupuncture is unsatisfactory. Future reviewers should improve critical areas of publishing protocols, performing comprehensive search, providing a list of excluded studies with justifications for exclusion, and conducting meta-analysis appropriately.

2017 ◽  
Vol 27 (6) ◽  
pp. 619-627 ◽  
Author(s):  
V. C. H. Chung ◽  
X. Y. Wu ◽  
Y. Feng ◽  
R. S. T. Ho ◽  
S. Y. S. Wong ◽  
...  

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.


2020 ◽  
Vol 17 (8) ◽  
pp. 949-957
Author(s):  
Irene X. Y. Wu ◽  
Yihong Deng ◽  
Huan Wang ◽  
Yancong Chen ◽  
Charlene H. L. Wong ◽  
...  

2020 ◽  
Vol 12 ◽  
pp. 1759720X2095996
Author(s):  
Irene XY Wu ◽  
Huan Wang ◽  
Lin Zhu ◽  
Yancong Chen ◽  
Charlene HL Wong ◽  
...  

Background: Healthcare providers need reliable evidence for supporting the adoption of new interventions, of which the source of evidence often originates from systematic reviews (SRs). However, little assessment on the rigor of SRs related to osteoarthritis interventions has been conducted. This cross-sectional study aimed to evaluate the methodological quality and predictors among SRs on osteoarthritis interventions. Methods: Four electronic databases (Cochrane Database of Systematic Reviews, MEDLINE, Embase, and PsycINFO) were searched, from 1 January 2008 to 10 October 2019. An SR was eligible if it focused on osteoarthritis interventions, and we performed at least one meta-analysis. Methodological quality was assessed using the validated AMSTAR 2 instrument. Multivariate regression analyses were conducted to assess predictors of methodological quality. Results: In total, 167 SRs were included. The most SRs were non-Cochrane reviews (88.6%), and 54.5% investigated non-pharmacological interventions. Only seven (4.2%) had high methodological quality. Respectively, eight (4.8%), 25 (15.0%), and 127 (76.0%) SRs had moderate, low, and critically low quality. Main methodological weaknesses were as follows: only 16.8% registered protocol a priori, 4.2% searched literature comprehensively, 25.7% included lists of excluded studies with justifications, and 30.5% assessed risk of bias appropriately by considering allocation concealment, blinding of patients and assessors, random sequence generation and selective reported outcomes. Cochrane reviews [adjusted odds ratio (AOR) 251.5, 95% confidence interval (CI) 35.5–1782.6], being updates of previous SRs (AOR 3.9, 95% CI 1.1–13.7), and SRs published after 2017 (AOR 7.7, 95% CI 2.8–21.5) were positively related to higher methodological quality. Conclusion: Despite signs of improvement in recent years, most of the SRs on osteoarthritis interventions have critically low methodological quality, especially among non-Cochrane reviews. Future SRs should be improved by conducting comprehensive literature search, justifying excluded studies, publishing a protocol, and assessing the risk of bias of included studies appropriately.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Leonard Ho ◽  
Fiona Y. T. Ke ◽  
Charlene H. L. Wong ◽  
Irene X. Y. Wu ◽  
Andy K. L. Cheung ◽  
...  

Abstract Background While well-conducted systematic reviews (SRs) can provide the best evidence on the potential effectiveness of acupuncture, limitations on the methodological rigour of SRs may impact the trustworthiness of their conclusions. This cross-sectional study aimed to evaluate the methodological quality of a representative sample of SRs on acupuncture effectiveness. Methods Cochrane Database of Systematic Reviews, MEDLINE, and EMBASE were searched for SRs focusing on the treatment effect of manual acupuncture or electro-acupuncture published during January 2018 and March 2020. Eligible SRs must contain at least one meta-analysis and be published in English language. Two independent reviewers extracted the bibliographical characteristics of the included SRs with a pre-designed questionnaire and appraised the methodological quality of the studies with the validated AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2). The associations between bibliographical characteristics and methodological quality ratings were explored using Kruskal-Wallis rank tests and Spearman’s rank correlation coefficients. Results A total of 106 SRs were appraised. Only one (0.9%) SR was of high overall methodological quality, zero (0%) was of moderate-quality, six (5.7%) and 99 (93.4%) were of low-quality and critically low-quality respectively. Among appraised SRs, only ten (9.4%) provided an a priori protocol, four (3.8%) conducted a comprehensive literature search, five (4.7%) provided a list of excluded studies, and six (5.7%) performed meta-analysis appropriately. Cochrane SRs, updated SRs, and SRs that did not search non-English databases had relatively higher overall quality. Conclusions Methodological quality of SRs on acupuncture is unsatisfactory. Future reviewers should improve critical methodological aspects of publishing protocols, performing comprehensive search, providing a list of excluded studies with justifications for exclusion, and conducting appropriate meta-analyses. These recommendations can be implemented via enhancing the technical competency of reviewers in SR methodology through established education approaches as well as quality gatekeeping by journal editors and reviewers. Finally, for evidence users, skills in SR critical appraisal remain to be essential as relevant evidence may not be available in pre-appraised formats.


Heliyon ◽  
2020 ◽  
Vol 6 (9) ◽  
pp. e04776
Author(s):  
Katja Matthias ◽  
Olesja Rissling ◽  
Dawid Pieper ◽  
Johannes Morche ◽  
Marc Nocon ◽  
...  

Bone ◽  
2020 ◽  
Vol 139 ◽  
pp. 115541
Author(s):  
Anna K.N. Tsoi ◽  
Leonard T.F. Ho ◽  
Irene X.Y. Wu ◽  
Charlene H.L. Wong ◽  
Robin S.T. Ho ◽  
...  

BMJ ◽  
2005 ◽  
Vol 330 (7499) ◽  
pp. 1053 ◽  
Author(s):  
Lorenzo P Moja ◽  
Elena Telaro ◽  
Roberto D'Amico ◽  
Ivan Moschetti ◽  
Laura Coe ◽  
...  

2020 ◽  
Vol 10 (5) ◽  
pp. 667-673 ◽  
Author(s):  
Joseph R. Dettori ◽  
Andrea C. Skelly ◽  
Erika D. Brodt

Study Design: A systematic cross-sectional survey of systematic reviews (SRs). Objective: To evaluate the methodological quality of spine surgery SRs published in 2018 using the updated AMSTAR 2 critical appraisal instrument. Methods: We identified the PubMed indexed journals devoted to spine surgery research in 2018. All SRs of spine surgical interventions from those journals were critically appraised for quality independently by 2 reviewers using the AMSTAR 2 instrument. We calculated the percentage of SRs achieving a positive response for each AMSTAR 2 domain item and assessed the levels of confidence in the results of each SR. Results: We identified 28 SRs from 4 journals that met our criteria for inclusion. Only 49.5% of the AMSTAR 2 domain items satisfied the AMSTAR 2 criteria. Critical domain items were satisfied less often (39.1%) compared with noncritical domain items (57.3%). Domain items most poorly reported include accounting for individual study risk of bias when interpreting results (14%), list and justification of excluded articles (18%), and an a priori establishment of methods prior to the review or registered protocol (18%). The overall confidence in the results was rated “low” in 2 SRs and “critically low” in 26. Conclusions: The credibility of a SR and its value to clinicians and policy makers are dependent on its methodological quality. This appraisal found significant methodological limitations in several critical domains, such that the confidence in the findings of these reviews is “critically low.”


Sign in / Sign up

Export Citation Format

Share Document