GRADEing the quality of evidence on safety: an adapted GRADE approach for preparing lists of potentially inappropriate medication for older adults

Abstract BackgroundSystematic reviews that synthesize safety outcomes pose challenges (e.g. rare events), which poses questions for grading the strength of the body of evidence. In this contribution, we suggest adaption of the GRADE system for grading the quality of evidence on safety outcomes for developing a potentially inappropriate medication list (PRISCUS).MethodsWe systematically assessed each of the five GRADE domains for rating-down (study limitations, imprecision, inconsistency, indirectness, publication bias) and the criteria for rating-up, considering if special considerations or revisions of the original approach were indicated. The result was gathered in a written document and discussed in a group-meeting. Subsequently, we performed a proof-of-concept application using a convenience sample of systematic reviews. Results We adapted aspects of the criteria study limitations, imprecision, publication bias and rating-up for large effect. In addition, we suggest a new criterion to account for data from subgroup-analyses. The proof-of-concept application did not reveal a need for further revision and thus we used the approach for the systematic reviews that were prepared for the PRISCUS-list. We assessed 51 outcomes for 19 clinical questions. Each of the proposed adaptions was applied. There were neither an excessive number of low and very low ratings, nor an excessive number of high ratings, but the different methodological quality of the safety outcomes appeared to be well reflected.ConclusionThe adaptions appear to have the potential to overcome some of the challenges when grading the methodological quality of harms and thus may be helpful for producers of evidence syntheses considering safety.

Download Full-text

An Overview of Systematic Reviews of Chinese Herbal Medicine for Alzheimer’s Disease

Frontiers in Pharmacology ◽

10.3389/fphar.2021.761661 ◽

2021 ◽

Vol 12 ◽

Author(s):

Nanyang Liu ◽

Tingting Zhang ◽

Jiahui Sun ◽

Jiuxiu Yao ◽

Lina Ma ◽

...

Keyword(s):

Alzheimer’S Disease ◽

Alzheimer's Disease ◽

Herbal Medicine ◽

Systematic Reviews ◽

Chinese Herbal Medicine ◽

Methodological Quality ◽

Efficacy And Safety ◽

Quality Of Evidence ◽

Chinese Herbal

Background: Multiple systematic reviews (SRs) have been conducted to evaluate the efficacy and safety of Chinese herbal medicine (CHM) in patients with Alzheimer’s disease (AD). Here, we aim to perform an overview to assess the methodological quality and quality of evidence of the SRs to provide convincing data on the treatment of AD with CHM.Method: Six electronic databases including Chinese and English were searched, until April 31, 2021. Two researchers independently screen documents and extract data according to the predesigned rules. A Measure Tool to Assessment System Reviews 2 (AMSTAR-2) was used to investigate the methodological quality, and the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) was used to determine the quality of evidence for outcomes.Results: Twelve qualified SRs including 163 randomized controlled trials were reviewed. The methodological quality of the included SRs was considered extremely low assessed through AMSTAR-2. Compared with western medicines (WM) alone, CHM as an adjuvant treatment has shown significant effects in improving Mini-mental State Examination, Alzheimer’s Disease Assessment Scale-Cognitive, and Clinical Dementia Rating scores. The same is true for CHM alone. Regarding the effect on Activities Daily Living, neither the single CHM nor the combination with WM has an obvious effect. For the total effective rate, both single CHM and the combination with WM shown significant effects. Nine SRs suggested that CHM as adjuvant therapy or single-use had fewer adverse events than WM. Additionally, the quality of evidence for the main outcome was reviewed as low or extremely low according to GRADE profiler data.Conclusion: Current evidence suggests that CHM may be beneficial in improving the cognitive function of AD patients. However, we should be cautious about the evidence due to methodological flaws and low quality. High-quality RCTs are further needed to confirm the efficacy and safety of CHM for AD.

Download Full-text

Overview of systematic reviews - a new type of study. Part II

Sao Paulo Medical Journal ◽

10.1590/1516-3180.2013.8150015 ◽

2014 ◽

Vol 133 (3) ◽

pp. 206-217 ◽

Cited By ~ 16

Author(s):

Valter Silva ◽

Antonio Jose Grande ◽

Alan Pedrosa Viegas de Carvalho ◽

Ana Luiza Cabrera Martimbianco ◽

Rachel Riera

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Quality Of Evidence ◽

Overview Of Systematic Reviews ◽

New Type ◽

The Mean ◽

Meta Analyses ◽

General Profile

CONTEXT AND OBJECTIVE: Overviews of Systematic Reviews (OoRs) are a new type of study in which multiple evidence from systematic reviews (SRs) is compiled into an accessible and useful document. The aim here was to describe the state of the art and critically assess Cochrane OoRs that have been published.DESIGN AND SETTING: Descriptive study conducted at a research center.METHODS: The OoRs identified through the filter developed in Part I of this study were evaluated in five domains: methodological quality; quality of evidence; implications for practice; general profile of OoRs; and length of work.RESULTS: All 13 OoRs included had high methodological quality. Some OoRs did not present sufficient data to judge the quality of evidence; using sensitivity analysis, the quality of evidence of the OoRs increased. Regarding implications for practice, 64% of the interventions were judged as beneficial or harmful, while 36% of them showed insufficient evidence for judgment. It is expected (with 95% confidence interval) that one OoR will include 9,462 to 64,469 patients, 9 to 29 systematic reviews and 80 to 344 primary studies, and assess 6 to 21 interventions; and that 50 to 92% of OoRs will produce meta-analysis. The OoRs generated 2 to 26 meta-analyses over a period of 18 to 31 months.CONCLUSION: The OoRs presented high methodological quality; the quality of evidence tended to be moderate/high; most interventions were judged to be beneficial/harmful; the mean length of work was 24 months. The OoR profile adds power to decision-making.

Download Full-text

Oriental Herbal Medicine for Neurological Disorders in Children: An Overview of Systematic Reviews

The American Journal of Chinese Medicine ◽

10.1142/s0192415x18500866 ◽

2018 ◽

Vol 46 (08) ◽

pp. 1701-1726 ◽

Cited By ~ 8

Author(s):

Boram Lee ◽

Chan-Young Kwon ◽

Gyu Tae Chang

Keyword(s):

Cerebral Palsy ◽

Herbal Medicine ◽

Antiepileptic Drugs ◽

Systematic Reviews ◽

Neurological Disorders ◽

Methodological Quality ◽

Alternative Therapy ◽

Autism Spectrum ◽

Quality Of Evidence

Oriental herbal medicine (OHM) has been widely used in pediatric neurological disorders and has attracted attention as a safe and effective treatment. We aim to summarize and evaluate the evidence for OHM in pediatric neurological disorders for evidence-based decision-making. Without language restrictions, up-to-date research data were obtained from nine electronic databases. Systematic reviews (SRs) assessing the efficacy of OHM for pediatric neurological disorders were included. The methodological quality of each review was assessed using the AMSTAR instrument. The quality of evidence for the main findings was evaluated using the GRADE approach. Sixteen SRs comprising 169 randomized controlled trials with 19,542 participants were included. In epilepsy (six SRs, [Formula: see text]), OHM as an adjunctive or alternative therapy to antiepileptic drugs showed higher clinical symptom improvements than did antiepileptic drugs alone. The Activities of Daily Living scale score was significantly higher in children with cerebral palsy (one SR, [Formula: see text]) when OHM was added to rehabilitation. There were inconsistent results for tic disorder (four SRs, [Formula: see text]) and enuresis (two SRs, [Formula: see text]) and unclear results for attention deficit hyperactivity disorder (two SRs, [Formula: see text]) and autism spectrum disorder (one SR, [Formula: see text]). Eleven SRs reported adverse events, but no fatal adverse reaction was reported. The methodological quality of the included reviews was medium-to-high. The overall quality of evidence ranged from “very low” to “moderate.” In conclusion, the efficacy of OHM is promising for some pediatric neurological disorders such as epilepsy and cerebral palsy. However, more high-quality evidence is needed to make clinical recommendations on OHM use.

Download Full-text

Wake up and smell the coffee: caffeine supplementation and exercise performance—an umbrella review of 21 published meta-analyses

British Journal of Sports Medicine ◽

10.1136/bjsports-2018-100278 ◽

2019 ◽

Vol 54 (11) ◽

pp. 681-688 ◽

Cited By ~ 52

Author(s):

Jozo Grgic ◽

Ivana Grgic ◽

Craig Pickering ◽

Brad J Schoenfeld ◽

David J Bishop ◽

...

Keyword(s):

Muscle Strength ◽

Systematic Reviews ◽

Exercise Performance ◽

Methodological Quality ◽

Muscle Endurance ◽

Umbrella Review ◽

Caffeine Ingestion ◽

Quality Of Evidence ◽

Meta Analyses

ObjectiveTo systematically review, summarise and appraise findings of published meta-analyses that examined the effects of caffeine on exercise performance.DesignUmbrella review.Data sourcesTwelve databases.Eligibility criteria for selecting studiesMeta-analyses that examined the effects of caffeine ingestion on exercise performance.ResultsEleven reviews (with a total of 21 meta-analyses) were included, all being of moderate or high methodological quality (assessed using the Assessing the Methodological Quality of Systematic Reviews 2 checklist). In the meta-analyses, caffeine was ergogenic for aerobic endurance, muscle strength, muscle endurance, power, jumping performance and exercise speed. However, not all analyses provided a definite direction for the effect of caffeine when considering the 95% prediction interval. Using the Grading of Recommendations Assessment, Development and Evaluation criteria the quality of evidence was generally categorised as moderate (with some low to very low quality of evidence). Most individual studies included in the published meta-analyses were conducted among young men.Summary/conclusionSynthesis of the currently available meta-analyses suggest that caffeine ingestion improves exercise performance in a broad range of exercise tasks. Ergogenic effects of caffeine on muscle endurance, muscle strength, anaerobic power and aerobic endurance were substantiated by moderate quality of evidence coming from moderate-to-high quality systematic reviews. For other outcomes, we found moderate quality reviews that presented evidence of very low or low quality. It seems that the magnitude of the effect of caffeine is generally greater for aerobic as compared with anaerobic exercise. More primary studies should be conducted among women, middle-aged and older adults to improve the generalisability of these findings.

Download Full-text

An umbrella review comparing computer-assisted and conventional total joint arthroplasty: quality assessment and summary of evidence

BMJ Surgery, Interventions, & Health Technologies ◽

10.1136/bmjsit-2019-000016 ◽

2020 ◽

Vol 2 (1) ◽

pp. e000016 ◽

Cited By ~ 1

Author(s):

Mohamed Mosaad Hasan ◽

Manrui Zhang ◽

Matthew Beal ◽

Hassan M K Ghomrawi

Keyword(s):

Patient Safety ◽

Systematic Reviews ◽

Methodological Quality ◽

Assessment Tool ◽

Measurement Tool ◽

Computer Assisted ◽

Safety Outcomes ◽

Methodological Assessment ◽

Meta Analyses

BackgroundSystematic reviews (SRs) of computer-assisted (CA) total knee arthroplasty (TKA) and total hip arthroplasty (THA) report conflicting evidence on its superiority over conventional surgery. Little is known about the quality of these SRs; variability in their methodological quality may be a contributing factor. We evaluated the methodological quality of all published SRs to date, summarized and examined the consistency of the evidence generated by these SRs.MethodsWe searched four databases through December 31, 2018. A MeaSurement Tool to Assess systematic Reviews 2 (AMSTAR 2) was applied to assess the methodological quality. Evidence from included meta-analyses on functional, radiological and patient-safety outcomes was summarized. The corrected covered area was calculated to assess the overlap between SRs in including the primary studies.ResultsBased on AMSTAR 2, confidence was critically low in 39 of the 42 included SRs and low in 3 SRs. Low rating was mainly due to failure in developing a review protocol (90.5%); providing a list of excluded studies (81%); accounting for risk of bias when discussing the results (67%); using a comprehensive search strategy (50%); and investigating publication bias (50%). Despite inconsistency between SR findings comparing functional, radiological and patient safety outcomes for CA and conventional procedures, most TKA meta-analyses favored CA TKA, whereas most THA meta-analyses showed no difference. Moderate overlap was observed among TKA SRs and high overlap among THA SRs.ConclusionsDespite conclusions of meta-analyses favoring CA arthroplasty, decision makers adopting this technology should be aware of the low confidence in the results of the included SRs. To improve confidence in future SRs, journals should consider using a methodological assessment tool to evaluate the SRs prior to making a publication decision.

Download Full-text

Dietary risk factors for hip fracture in adults: An umbrella review of meta-analyses of prospective cohort studies

PLoS ONE ◽

10.1371/journal.pone.0259144 ◽

2021 ◽

Vol 16 (11) ◽

pp. e0259144

Author(s):

James Webster ◽

Catherine E. Rycroft ◽

Darren C. Greenwood ◽

Janet E. Cade

Keyword(s):

Hip Fracture ◽

Cohort Studies ◽

Systematic Reviews ◽

Methodological Quality ◽

Quality Of Evidence ◽

Prospective Cohort Studies ◽

Dietary Risk Factors ◽

Dietary Risk ◽

Meta Analyses

Aim To summarise the totality of evidence regarding dietary risk factors for hip fracture in adults, evaluating the quality of evidence, to provide recommendations for practice and further research. Design Systematic review of meta-analyses of prospective cohort studies. Eligibility criteria Systematic reviews with meta-analyses reporting summary risk estimates for associations between hip fracture incidence and dietary exposures including oral intake of a food, food group, beverage, or nutrient, or adherence to dietary patterns. Information sources Medline, Embase, Web of Science, and the Cochrane Library from inception until November 2020. Data synthesis The methodological quality of systematic reviews and meta-analyses was assessed using AMSTAR-2, and the quality of evidence for each association was assessed using GRADE. Results were synthesised descriptively. Results Sixteen systematic reviews were identified, covering thirty-four exposures, including dietary patterns (n = 2 meta-analyses), foods, food groups, or beverages (n = 16), macronutrients (n = 3), and micronutrients (n = 13). Identified meta-analyses included 6,282 to 3,730,424 participants with between 322 and 26,168 hip fractures. The methodological quality (AMSTAR-2) of all systematic reviews was low or critically low. The quality of evidence (GRADE) was low for an inverse association between hip fracture incidence and intake of fruits and vegetables combined (adjusted summary relative risk for higher vs lower intakes: 0.92 [95% confidence interval: 0.87 to 0.98]), and very low for the remaining thirty-three exposures. Conclusion Dietary factors may play a role in the primary prevention of hip fracture, but the methodological quality of systematic reviews and meta-analyses was below international standards, and there was a lack of high-quality evidence. More long-term cohort studies reporting absolute risks and robust, well-conducted meta-analyses with dose-response information are needed before policy guidelines can be formed. Systematic review registration PROSPERO CRD42020226190.

Download Full-text

Effectiveness and Safety of Herbal Medicine for Atopic Dermatitis: An Overview of Systematic Reviews

Evidence-based Complementary and Alternative Medicine ◽

10.1155/2020/4140692 ◽

2020 ◽

Vol 2020 ◽

pp. 1-15

Author(s):

Chan-Young Kwon ◽

Boram Lee ◽

Suran Kim ◽

Jaesuk Lee ◽

Minjung Park ◽

...

Keyword(s):

Atopic Dermatitis ◽

Herbal Medicine ◽

Systematic Reviews ◽

Adjunctive Therapy ◽

Methodological Quality ◽

Symptom Relief ◽

Effective Rate ◽

Quality Of Evidence ◽

Overview Of Systematic Reviews

Objectives. Herbal medicine (HM) is attracting attention for treating atopic dermatitis (AD). This overview was conducted to summarize and critically evaluate the current systematic reviews (SRs) on HM for the treatment of AD. Methods. Through comprehensive searches, all relevant SRs on HM for AD published until May 2020 were included. The quality of included SRs was assessed using the AMSTAR-2 tool. Moreover, original randomized controlled trials (RCTs) included in the SRs were resynthesized to investigate the efficacy and safety of oral HM for AD. The quality of evidence for the main findings was evaluated using the GRADE approach. Results. Nine SRs were included in this overview. HM showed significantly better efficacy in terms of total effective rate (TER), itching and sleep symptom scores, quality of life, and the dose of topical treatment used compared with placebo. HM as a monotherapy and/or an adjunctive therapy to conventional medication (CM) showed significantly better results on the efficacy, symptom relief, and some laboratory parameters related to the inflammatory response. The methodological quality was generally low. When 58 original RCTs were reanalyzed, HM showed significantly lower SCORing Atopic Dermatitis (SCORAD) score and higher TER than the placebo or CM. In terms of the safety profile, HM was not significantly different from the placebo and was better than CM. The quality of evidence ranged from “moderate” to “very low.” Conclusion. The results suggested that HM as a monotherapy or an adjunctive therapy is promising for the treatment of AD. However, due to low methodological quality and low quality of evidence, further rigorous, well-designed, high-quality SRs, and RCTs are needed to make clinical recommendations on HM use.

Download Full-text

Non-Vitamin K Antagonist Oral Anticoagulants in Pulmonary Embolism: An Overview of Systematic Reviews

Current Pharmaceutical Design ◽

10.2174/1381612826666200506114450 ◽

2020 ◽

Vol 26 (23) ◽

pp. 2686-2691 ◽

Cited By ~ 1

Author(s):

Ioannis Doundoulakis ◽

Christina Antza ◽

Haralambos Karvounis ◽

George Giannakoulas

Keyword(s):

Pulmonary Embolism ◽

Vitamin K ◽

Systematic Reviews ◽

Methodological Quality ◽

Randomized Clinical Trials ◽

Oral Anticoagulants ◽

Vitamin K Antagonist ◽

Language Restriction ◽

Overview Of Systematic Reviews

Background: Anticoagulation in patients with pulmonary embolism. Objective: To identify how non-vitamin K antagonist oral anticoagulants are associated with multiple outcomes in patients with pulmonary embolism. Methods: We performed a systematic search of systematic reviews via multiple electronic databases from inception to August 19th, 2019, without language restriction. Two authors independently extracted data and assessed the methodological quality of the included systematic reviews using the ROBIS tool. Results: We found twelve systematic reviews. Eleven SRs collected their data from randomized clinical trials and one from observational studies. All the included studies were published between 2014 and 2019 in English. The methodological quality of the 12 systematic reviews was low to high. None of the systematic reviews, which are included in our overview of systematic reviews, has evaluated the overall quality of evidence outcome using the Grading of Recommendations Assessments, Development and Evaluation (GRADE) approach. Conclusion: This is the first effort to summarize evidence about non-vitamin K antagonist oral anticoagulants in an overview of systematic reviews focusing exclusively on patients with pulmonary embolism. The evidence suggests that the non-vitamin K antagonist oral anticoagulants seem to be more effective and safer than a dualdrug approach with LMWH- VKA.

Download Full-text