Methodological Quality of Drug Safety Systematic Reviews: A Cross-Sectional Study

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.

Download Full-text

The methodological quality of systematic reviews on the treatment of adult major depression needs improvement according to AMSTAR 2: A cross-sectional study

Heliyon ◽

10.1016/j.heliyon.2020.e04776 ◽

2020 ◽

Vol 6 (9) ◽

pp. e04776

Author(s):

Katja Matthias ◽

Olesja Rissling ◽

Dawid Pieper ◽

Johannes Morche ◽

Marc Nocon ◽

...

Keyword(s):

Major Depression ◽

Systematic Reviews ◽

Methodological Quality ◽

Cross Sectional Study ◽

Sectional Study ◽

Cross Sectional

Download Full-text

Methodological quality of systematic reviews on treatments for osteoporosis: A cross-sectional study

Bone ◽

10.1016/j.bone.2020.115541 ◽

2020 ◽

Vol 139 ◽

pp. 115541

Author(s):

Anna K.N. Tsoi ◽

Leonard T.F. Ho ◽

Irene X.Y. Wu ◽

Charlene H.L. Wong ◽

Robin S.T. Ho ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Cross Sectional Study ◽

Sectional Study ◽

Cross Sectional

Download Full-text

Reporting Quality of Drug Safety Systematic Reviews: A Cross-Sectional Study

Value in Health ◽

10.1016/j.jval.2018.07.639 ◽

2018 ◽

Vol 21 ◽

pp. S85

Author(s):

L Li ◽

C Xu ◽

K Deng ◽

X Zhou ◽

X Sun

Keyword(s):

Systematic Reviews ◽

Drug Safety ◽

Reporting Quality ◽

Cross Sectional Study ◽

Sectional Study ◽

Cross Sectional

Download Full-text

Methodological Quality of Systematic Reviews and Meta-Analysis on Asthma Treatments. A Cross-Sectional Study

Annals of the American Thoracic Society ◽

10.1513/annalsats.202003-187oc ◽

2020 ◽

Vol 17 (8) ◽

pp. 949-957

Author(s):

Irene X. Y. Wu ◽

Yihong Deng ◽

Huan Wang ◽

Yancong Chen ◽

Charlene H. L. Wong ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Cross Sectional Study ◽

Sectional Study ◽

Cross Sectional

Download Full-text

Low Methodological Quality of Systematic Reviews on Acupuncture: A Cross- Sectional Study

10.21203/rs.3.rs-778930/v1 ◽

2021 ◽

Author(s):

Leonard TF Ho ◽

Fiona YT Ke ◽

Charlene HL Wong ◽

Irene XY Wu ◽

Andy KL Cheung ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

A Priori ◽

Cross Sectional Study ◽

Sectional Study ◽

Cross Sectional ◽

Quality Ratings ◽

Cochrane Reviews

Abstract Background: While well-conducted systematic reviews (SRs) can provide best evidence on the potential effectiveness of acupuncture, limitations on methodological rigour of SRs may impact trustworthiness of their conclusions. This cross-sectional study aimed to evaluate the methodological quality of a representative sample of SRs on acupuncture effectiveness.Methods: CDSR, MEDLINE, and EMBASE were searched for SRs on acupuncture. AMSTAR2 was applied for assessing methodological quality. Associations between bibliographical characteristics and methodological quality ratings were examined. Results: A total of 106 SRs were appraised. Only one (0.9%) SR was of high overall methodological quality, zero (0%) was of moderate-quality, six (5.7%) and 99 (93.4%) were of low-quality and critically low-quality respectively. Among appraised SRs, only ten (9.4%) provided an a priori protocol, four (3.8%) conducted a comprehensive literature search, five (4.7%) provided a list of excluded study, and six (5.7%) performed meta-analysis appropriately. Cochrane reviews, update reviews, reviews with corresponding authors from the America, and reviews that searched non-English databases had relatively higher overall quality. Conclusions: Methodological quality of SRs on acupuncture is unsatisfactory. Future reviewers should improve critical areas of publishing protocols, performing comprehensive search, providing a list of excluded studies with justifications for exclusion, and conducting meta-analysis appropriately.

Download Full-text

Low methodological quality of systematic reviews on acupuncture: a cross-sectional study

BMC Medical Research Methodology ◽

10.1186/s12874-021-01437-0 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Leonard Ho ◽

Fiona Y. T. Ke ◽

Charlene H. L. Wong ◽

Irene X. Y. Wu ◽

Andy K. L. Cheung ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

English Language ◽

Meta Analysis ◽

Rank Correlation ◽

Cross Sectional Study ◽

Measurement Tool ◽

Sectional Study ◽

Cross Sectional

Abstract Background While well-conducted systematic reviews (SRs) can provide the best evidence on the potential effectiveness of acupuncture, limitations on the methodological rigour of SRs may impact the trustworthiness of their conclusions. This cross-sectional study aimed to evaluate the methodological quality of a representative sample of SRs on acupuncture effectiveness. Methods Cochrane Database of Systematic Reviews, MEDLINE, and EMBASE were searched for SRs focusing on the treatment effect of manual acupuncture or electro-acupuncture published during January 2018 and March 2020. Eligible SRs must contain at least one meta-analysis and be published in English language. Two independent reviewers extracted the bibliographical characteristics of the included SRs with a pre-designed questionnaire and appraised the methodological quality of the studies with the validated AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2). The associations between bibliographical characteristics and methodological quality ratings were explored using Kruskal-Wallis rank tests and Spearman’s rank correlation coefficients. Results A total of 106 SRs were appraised. Only one (0.9%) SR was of high overall methodological quality, zero (0%) was of moderate-quality, six (5.7%) and 99 (93.4%) were of low-quality and critically low-quality respectively. Among appraised SRs, only ten (9.4%) provided an a priori protocol, four (3.8%) conducted a comprehensive literature search, five (4.7%) provided a list of excluded studies, and six (5.7%) performed meta-analysis appropriately. Cochrane SRs, updated SRs, and SRs that did not search non-English databases had relatively higher overall quality. Conclusions Methodological quality of SRs on acupuncture is unsatisfactory. Future reviewers should improve critical methodological aspects of publishing protocols, performing comprehensive search, providing a list of excluded studies with justifications for exclusion, and conducting appropriate meta-analyses. These recommendations can be implemented via enhancing the technical competency of reviewers in SR methodology through established education approaches as well as quality gatekeeping by journal editors and reviewers. Finally, for evidence users, skills in SR critical appraisal remain to be essential as relevant evidence may not be available in pre-appraised formats.

Download Full-text