PP434 Quality Of Studies Submitted To Support Requests For Medical Devices And Equipment Incorporation In A Teaching Hospital

2020 ◽  
Vol 36 (S1) ◽  
pp. 35-35
Author(s):  
Katiuce Tomazi Kny ◽  
Maria Angélica Pires Ferreira

IntroductionIn hospitals with standardized processes of health technology assessment (HTA), clinical studies are usually required to support incorporation requests. Factors such as conflicts of interest and confirmation bias affect the quality of the evidence. The objective of this study was to assess the quality of studies submitted to the university hospital to support the incorporation of non-drugs technologies.MethodsAll submissions received from 2017 to 2019 were reviewed. Data about methodological quality of studies sent by requesters and their utilization of final recommendation were collected, as well as about studies retrieved during additional search performed by HTA. The Jadad and Assessing the Methodological Quality of Systematic Reviews (AMSTAR) scales were used for assessing the quality of randomized clinical trials (RCTs) and systematic reviews (SRs), respectively.ResultsTwenty-one requests for incorporation of equipment were analyzed. The average number of studies attached was 4.5. In eight requests (53.3%), both SRs and RCTs were attached. In seven (46.7%) only low quality studies were included were included; additional search identified RCTs and SRs in four of these cases. According to the Jadad and AMSTAR scales, 60 percent of the submitted RCTs showed a high risk of bias, while 57 percent of the SRs showed moderate quality, respectively.ConclusionsThe best evidence is not always submitted during the incorporation request process. Requirements for studies of moderate-to-high quality to accept the incorporation demand could be helpful to avoid inefficiencies in hospital-based HTA.

2020 ◽  
Vol 26 (23) ◽  
pp. 2686-2691 ◽  
Author(s):  
Ioannis Doundoulakis ◽  
Christina Antza ◽  
Haralambos Karvounis ◽  
George Giannakoulas

Background: Anticoagulation in patients with pulmonary embolism. Objective: To identify how non-vitamin K antagonist oral anticoagulants are associated with multiple outcomes in patients with pulmonary embolism. Methods: We performed a systematic search of systematic reviews via multiple electronic databases from inception to August 19th, 2019, without language restriction. Two authors independently extracted data and assessed the methodological quality of the included systematic reviews using the ROBIS tool. Results: We found twelve systematic reviews. Eleven SRs collected their data from randomized clinical trials and one from observational studies. All the included studies were published between 2014 and 2019 in English. The methodological quality of the 12 systematic reviews was low to high. None of the systematic reviews, which are included in our overview of systematic reviews, has evaluated the overall quality of evidence outcome using the Grading of Recommendations Assessments, Development and Evaluation (GRADE) approach. Conclusion: This is the first effort to summarize evidence about non-vitamin K antagonist oral anticoagulants in an overview of systematic reviews focusing exclusively on patients with pulmonary embolism. The evidence suggests that the non-vitamin K antagonist oral anticoagulants seem to be more effective and safer than a dualdrug approach with LMWH- VKA.


2017 ◽  
Vol 27 (6) ◽  
pp. 619-627 ◽  
Author(s):  
V. C. H. Chung ◽  
X. Y. Wu ◽  
Y. Feng ◽  
R. S. T. Ho ◽  
S. Y. S. Wong ◽  
...  

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.


2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.


2020 ◽  
Vol 2020 ◽  
pp. 1-19
Author(s):  
Zipan Lyu ◽  
Zhongyu Huang ◽  
Fengbin Liu ◽  
Zhengkun Hou

Objective. To access the methodological and reporting quality of systematic reviews (SRs)/meta-analyses (MAs) about Chinese medical treatment for gastroesophageal reflux disease (GERD). Methods. The PubMed, Wanfang Data, China National Knowledge Infrastructure (CNKI), Chinese Science and Technology Periodical Database (VIP), Chinese Biomedical (CBM), Web of Science, and Cochrane Library databases were searched from inception to June 2020. Two researchers independently screened the literature considering the eligibility criteria. Overview Quality Assessment Questionnaire (OQAQ), Assessment of Multiple Systematic Reviews 2 (AMSTAR 2), and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used to assess the methodological and reporting quality of the included reports. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was used to evaluate the level of evidence in each report. Results. Thirty-three SRs/MAs met the inclusion criteria. The OQAQ results showed that defects in the methodological quality of 17/32 reports were major, with scores of 3 points. Analyzing a single item as the object, search strategies (item 2), and risk of bias in individual studies (item 4) was considered poor. The AMSTAR 2 results showed that 25.4% of the items were not reported, and 7.8% of the items were only partially reported. The overall assessment of AMSTAR 2 showed the majority of systematic reviews and meta-analyses were of low/very low (31/33, 93.9%) methodological quality, with a lack of protocol registration and excluded study list. The PRISMA results showed that 19.9% of items were not reported, and 15.2% of items were only partially reported, due to a lack of protocol registration and study selection methods. The methodological and reporting quality of the included studies was generally poor. Evidence evaluation with GRADE showed that most (31/33) of the included studies had low or very low levels of evidence. Conclusion. The methodological and reporting quality of SRs/MAs about Chinese medical treatment for GERD is generally poor. The main problems included incomplete search strategies, risk of bias in individual studies, the lack of protocol registration and excluded study list, and incorrect study selection methods.


2019 ◽  
Vol 8 (1) ◽  
Author(s):  
Ana Penedones ◽  
Carlos Alves ◽  
Francisco Batel Marques

Abstract Background Several recommendations are available to conduct and report a systematic review of adverse drug reactions. This study is aimed at identifying and comparing the methodologies of the two most commonly used recommendations to conduct and report systematic reviews on drug’s safety. Methods Two systematic reviews were conducted following the recommendations “Cochrane Handbook for Systematic Reviews of Interventions” and “Systematic Reviews’ Centre for Reviews and Dissemination guidance for undertaking reviews in healthcare.” The methods of each recommendation were characterized, and the results and the discussion of each systematic review were also evaluated. Results The methodologies of both recommendations are similar. The review question was structured. Both recommendations suggest to include pre- and post-marketing data. The recommended data sources differed and, consequently, the results of the systematic reviews (37 vs. 35 studies). Other aspects of search literature were identical. Different tools are suggested to evaluate the methodological quality of the included studies. For case reports, both recommendations only report some questions that may be helpful to assess risk of bias. The reporting of the results and discussion is also identical for both recommendations. Conclusions Few methodological differences were observed between the analyzed recommendations to conduct a systematic review on drug’s safety. Combining their methods into a single and recognized recommendation could be of great value.


2019 ◽  
Author(s):  
Qian Li ◽  
Ke Deng ◽  
Xiaoyuan Jiang ◽  
Huan Tao ◽  
Hui Liu ◽  
...  

Abstract Background:Systematic review or meta-analysis, the strong study design of high quality evidence, give inconsistent conclusion of long-term effectiveness or efficacy of opioids for chronic non-cancer pain. We appraised the methodological quality of systematic reviews or meta-analyses. Methods: We found the relevant systematic reviews or meta-analyses by searching Medline, EMBASE, the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, the International prospective register of systematic reviews, Psyc ARTICLES/OVID, the Chinese Bio-Medical Literature Database, the China National Knowledge Infrastructure, and the Wan Fang Data and VIP Database on March 1st, 2019. The methodological quality was assessed by A Measurement Tool to Assess Systematic Reviews-2(AMSTAR-2). Spearman correlation analysis and non-parametric tests were used to assess the association between quality and factors. Results: Twenty-one systematic reviews or meta-analyses were included in our study. One has no individual study. In terms of methodological quality, twelve reviews were critically low in overall confidence, four reviews were low, two reviews were moderate, two reviews were high. When referring to the systematic reviews or meta-analyses of relatively better methodological quality with more credible results and conclusions, the effectiveness or efficacy of opioids was small to questionable. Cochrane reviews performed better than non-Cochrane reviews in establishing prior protocol (100% vs 17%, P<0.05), providing an excluded studies list (100% vs 50%, P<0.05) and taking risk of bias into account when interpreting the results of the review (100% vs 75%, P<0.05). There was a strong correlation (ρ=0.526, P<0.05) between the impact factor of systematic reviews or meta-analyses in published journals and methodological quality. Conclusion The methodological quality of the included systematic reviews or meta-analyses is far from satisfactory and needs improvement, especially in establishing prior protocol and justifying significant deviations from the protocol, providing an excluded primary studies list, reporting the funding information of primary studies, and assessing the potential impact of risk of bias on individual studies.


2021 ◽  
Vol 8 ◽  
Author(s):  
Jinke Huang ◽  
Xiaohui Qin ◽  
Min Shen ◽  
Yanjuan Xu ◽  
Yong Huang

Background: Tai chi (TC) is a popular form of exercise among adults with chronic heart failure (CHF), yet services are greatly underutilized. The aim of the current study was to identify and summarize the existing evidence and to systematically determine the clinical effectiveness of Tai Chi in the management of CHF using a systematic overview.Methods: Both English and Chinese databases were searched for systematic reviews (SRs)/meta-analyses (MAs) on TC for CHF from their inception to June 2020. The methodological quality, reporting quality, and risk of bias of SRs/MAs were assessed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, and Risk of Bias in Systematic reviews (ROBIS), respectively. The evidence quality of outcome measures was assessed by the Grades of Recommendations, Assessment, Development and Evaluation (GRADE).Results: Six SRs/MAs using a quantitative synthesis to assess various outcomes of TC in CHF were included in this overview. The methodological quality, reporting quality and risk of bias of the SRs/MAs and the evidence quality of the outcome measures are generally unsatisfactory. The limitations of the past SRs/MAs included the lack of either the protocol or registration, the list of excluded studies, and the computational details of meta-analysis were inadequately reported. The critical problems were that qualitative data synthesis relied on trials with small sample sizes and critical low quality.Conclusions: TC may be a promising complementary treatment for CHF. However, further rigorous and comprehensive SRs/MAs and RCTs are required to provide robust evidence for definitive conclusions.


2017 ◽  
Vol 33 (S1) ◽  
pp. 74-75
Author(s):  
Elke Hausner ◽  
Marco Knelangen ◽  
Laura Sanders ◽  
Siw Waffenschmidt

INTRODUCTION:Health Technology Assessment (HTA) reports may have a major impact on the health care provided in a country. Hence, one would assume that these reports have a high methodological quality and thus represent a potentially important source of information, for instance, for identifying primary studies for inclusion in the evidence syntheses (for example, systematic reviews, Cochrane reviews, HTA reports). The aim of the present analysis is to evaluate the methodological quality of HTA reports used as a literature source for HTA reports produced by the German Institute for Quality and Efficiency in Health Care (IQWiG).METHODS:Eligible IQWiG reports were assessments of drug or non-drug interventions considering HTA reports as the literature source for primary studies and published up to October 2016. An HTA report included in the IQWIG report was considered in the analysis if it was a complete report published in English or German and indexed in the Health Technology Assessment Database (Wiley) or MEDLINE. Only the most current HTA report in an IQWiG report was considered; if more than one current HTA report was available, the one for inclusion in the analysis was randomly selected. The methodological quality of the HTA reports identified was evaluated with the AMSTAR (“Assessment of Multiple Systematic Reviews”) tool (1), which comprises 11 items on methodological quality (meaning a maximum achievable score of 11).RESULTS:A total of fifty eligible IQWiG reports using fourty-one eligible HTA reports as literature sources were identified. The mean AMSTAR score of these HTA reports was 5.3 (95 percent Confidence Interval, CI: 4.3, 6.2). None of the HTA reports achieved a score of 11, nineteen (46 percent) had a score between 6 and 10, and twenty-two had a score below 6.CONCLUSIONS:HTA reports included in IQWiG reports only have an average methodological quality.


2021 ◽  
Vol 12 ◽  
Author(s):  
Jie Zhang ◽  
Qianying Yu ◽  
Li Peng ◽  
Yuesi Qin ◽  
Mingyi Jing ◽  
...  

Background: In recent years, systematic reviews/meta-analyses (SRs/MAs) of Chinese herbal medicine (CHM) for psoriasis have continuously emerged. Their methods and evidence quality, however, are yet to be evaluated, and whether their conclusions can provide clinicians with reliable evidence is still debatable.Objectives: This overview aims to evaluate the methodological quality, risk of bias, and reporting quality of relevant SRs/MAs, as well as the current evidence of CHM for treating psoriasis.Methods: We searched nine electronic databases from their respective time of establishment to January 20, 2021, as well as the reference lists of the included SRs/MAs, protocol registries, and gray literature. Two reviewers independently used the following: A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2, Risk of Bias in Systematic Reviews (ROBIS), the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), and Grades of Recommendations, Assessment, Development and Evaluation (GRADE) to evaluate the methodological quality, risk of bias, reporting quality, and evidence quality of the included SRs/MAs.Results: This review included 14 SRs/MAs involving 45 outcomes, of which 12 (85.71%) SRs/MAs had a very low quality evaluated by AMSTAR 2 and 7 (50.00%) SRs/MAs had a high risk of bias assessed by ROBIS. The protocol and registration and funding statements were the major reporting flaws according to the PRISMA checklist. The evaluation with the GRADE system demonstrated no outcome of high-quality evidence, and inconsistent efficacy evaluations were found in this overview. Only 15 (33.33%) outcomes were moderate-quality evidence, supporting the claim that CHM plus Western medicine (WM) was superior to WM. Generally low quality of evidence showed no difference in the incidence of adverse events between the combined therapy and WM. However, the conclusion that CHM was superior to WM cannot be drawn due to the inconsistent results.Conclusion: Despite that CHM has the potential benefit and safety in the adjuvant treatment of psoriasis, the conclusion should be treated with caution because of the generally low quality of methodology and evidence. In the future, high-quality randomized controlled trials (RCTs) should be carried out, and the quality of relevant SRs should also be improved to promote their clinical application.


2020 ◽  
Vol 12 ◽  
Author(s):  
Jinke Huang ◽  
Min Shen ◽  
Xiaohui Qin ◽  
Manli Wu ◽  
Simin Liang ◽  
...  

Background: Acupuncture may be an effective complementary treatment for Alzheimer's disease (AD). The aim of this study was to summarize the evidence provided by systematic reviews (SRs)/meta-analyses (MAs) on the effect of acupuncture on AD.Methods: Eight electronic databases were searched from their inception until October 19, 2020. The methodological quality, reporting quality, and risk of bias of the included SRs were assessed by the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the Risk of Bias in Systematic Reviews (ROBIS) tool, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Moreover, the evidence quality of the outcome measures was assessed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE).Results: Eleven SRs/MAs met all inclusion criteria. According to the results of the AMSTAR-2, all included reviews were rated critically as being of low quality. With PRISMA, the reporting checklist was relatively complete, but some reporting weaknesses remained in the topics of the protocol and registration, search strategy, risk of bias, additional analyses, and funding. Based on the ROBIS tool, only two SRs/MAs had a low risk of bias. With the GRADE system, no high-quality evidence was found, and only seven outcomes provided moderate-quality evidence. Among the downgraded factors, the risk of bias within the original trials was ranked first, followed by inconsistency, imprecision, and publication bias.Conclusions: Acupuncture is a promising complementary treatment for AD. However, due to the low quality of the SRs/MAs supporting these results, high-quality studies with rigorous study designs and larger samples are needed before widespread recommendations can be made.


Sign in / Sign up

Export Citation Format

Share Document