Overview of systematic reviews - a new type of study. Part II

CONTEXT AND OBJECTIVE: Overviews of Systematic Reviews (OoRs) are a new type of study in which multiple evidence from systematic reviews (SRs) is compiled into an accessible and useful document. The aim here was to describe the state of the art and critically assess Cochrane OoRs that have been published.DESIGN AND SETTING: Descriptive study conducted at a research center.METHODS: The OoRs identified through the filter developed in Part I of this study were evaluated in five domains: methodological quality; quality of evidence; implications for practice; general profile of OoRs; and length of work.RESULTS: All 13 OoRs included had high methodological quality. Some OoRs did not present sufficient data to judge the quality of evidence; using sensitivity analysis, the quality of evidence of the OoRs increased. Regarding implications for practice, 64% of the interventions were judged as beneficial or harmful, while 36% of them showed insufficient evidence for judgment. It is expected (with 95% confidence interval) that one OoR will include 9,462 to 64,469 patients, 9 to 29 systematic reviews and 80 to 344 primary studies, and assess 6 to 21 interventions; and that 50 to 92% of OoRs will produce meta-analysis. The OoRs generated 2 to 26 meta-analyses over a period of 18 to 31 months.CONCLUSION: The OoRs presented high methodological quality; the quality of evidence tended to be moderate/high; most interventions were judged to be beneficial/harmful; the mean length of work was 24 months. The OoR profile adds power to decision-making.

Download Full-text

Acupuncture for Primary Dysmenorrhea: An Overview of Systematic Reviews

Evidence-based Complementary and Alternative Medicine ◽

10.1155/2018/8791538 ◽

2018 ◽

Vol 2018 ◽

pp. 1-11 ◽

Cited By ~ 12

Author(s):

Furong Zhang ◽

Mingsheng Sun ◽

Shanshan Han ◽

Xiaoyu Shen ◽

Yanan Luo ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Current Evidence ◽

Primary Dysmenorrhea ◽

Overview Of Systematic Reviews ◽

Meta Analyses ◽

Positive Results ◽

Total Average

Objectives. As current evidence of the effectiveness on acupuncture for primary dysmenorrhea (PD) is inconsistent, we aimed to critically appraise the evidence from relevant systematic reviews (SRs). Methods. SRs of randomized controlled trials (RCTs) concerning acupuncture and PD were searched in four databases. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and latest Assessment of Multiple Systematic Reviews 2 (AMSTAR2) checklists were used to assess reporting characteristics and methodological quality, respectively. Results. The literature search yielded 38 potential records, of which five met the inclusion criteria. The total average (SD) for PRISMA was 20.60 (1.14) out of 27. All five SRs have more than one critical weakness in AMSTAR2, so their methodological qualities were considered as critically low. The most frequent problems included nonregistration of study protocol, absence of a list of excluded studies, and unclear acknowledgment of conflicts of interests. The three studies of higher methodological quality reported positive results in pain relief. Conclusion. The reporting and methodological quality of systematic reviews and meta-analysis studies were suboptimal, which demands further improvement. More efforts are needed to improve validity of systematic reviews and RCTs in this area.

Download Full-text

Systematic Reviews and Meta-Analysis in Spine Surgery—How Good Are They in Methodological Quality? A Systematic Review

Global Spine Journal ◽

10.1177/2192568220906810 ◽

2020 ◽

pp. 219256822090681 ◽

Cited By ~ 2

Author(s):

Muthu Sathish ◽

Ramakrishnan Eswar

Keyword(s):

Systematic Review ◽

Spine Surgery ◽

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Eligibility Criteria ◽

Cochrane Database ◽

Critical Flaw ◽

Meta Analyses

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

Download Full-text

P14.71 An assessment of the reporting and methodological quality of meningioma systematic reviews and meta-analyses

Neuro-Oncology ◽

10.1093/neuonc/noab180.178 ◽

2021 ◽

Vol 23 (Supplement_2) ◽

pp. ii51-ii52

Author(s):

A M George ◽

S Gupta ◽

S M Keshwara ◽

M A Mustafa ◽

C S Gillespie ◽

...

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Neurofibromatosis Type ◽

Risk Of Bias ◽

Measurement Tool ◽

High Quality ◽

Meta Analyses

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.

Download Full-text

Intake of Unprocessed and Processed Meat and the Association with Cardiovascular Disease: An Overview of Systematic Reviews

Nutrients ◽

10.3390/nu13103303 ◽

2021 ◽

Vol 13 (10) ◽

pp. 3303

Author(s):

Marianne Uhre Jakobsen ◽

Anette Bysted ◽

Heddie Mejborn ◽

Anders Stockmarr ◽

Ellen Trolle

Keyword(s):

Cardiovascular Disease ◽

Cohort Studies ◽

Systematic Reviews ◽

Meta Analysis ◽

Positive Association ◽

Red Meat ◽

Processed Meat ◽

Overview Of Systematic Reviews ◽

Meta Analyses

We conducted an overview of systematic reviews to summarize reviews of cohort studies on intake of unprocessed and processed meat and the risk of cardiovascular disease (CVD), coronary heart disease (CHD), and stroke. Systematic reviews of cohort studies published between January 2010 and August 2020 were identified through a systematic literature search in PubMed, Embase, and Web of Science. The quality of how each review was conducted was assessed and the overall confidence in the results of each review was rated using AMSTAR 2. The quality of evidence of each meta-analysis was graded using NutriGrade. Three reviews were included, with meta-analyses of unprocessed red meat and CVD (n = 1) and stroke (n = 2); unprocessed poultry and stroke (n = 1); and processed meat and CVD (n = 1), CHD (n = 1), and stroke (n = 3). The overall confidence in the results of each review was rated as critically low. The meta-evidence was graded moderate for a positive association between unprocessed red meat and stroke and moderate for a positive association between processed meat and CHD and stroke. For other associations the meta-evidence was graded as low or very low. In conclusion, the associations between unprocessed and processed meat with CVD and major subtypes of CVD have not been extensively investigated.

Download Full-text

Evaluating the Quality of the Iranian Systematic Reviews, Meta-Analyses, and Economic Evaluations in Healthcare from 2005 to 2015

Health Technology Assessment in Action ◽

10.18502/htaa.v3i1.5852 ◽

2021 ◽

Author(s):

Morteza Arab-Zozani ◽

Zahra Heidarifard ◽

Efat Jabarpour

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Methodological Quality ◽

Time Horizon ◽

Meta Analysis ◽

Economic Evaluations ◽

Evidence Based ◽

Economic Perspective ◽

Meta Analyses

Context: The number of studies on health is increasing rapidly worldwide and in Iran. Systematic review studies, meta-analyses, and economic evaluation are of great importance in evidence-based decision making because of their standing in the evidence-based pyramid. The purpose of this study was to evaluate the reporting and methodological quality of Iranian systematic reviews, meta-analysis studies and economic evaluations on healthcare. Evidence Acquisition: PubMed and Scopus databases were searched to find considered studies, including systematic reviews, meta analyses and economic evaluations published from 2005 to 2015. Because of the high volume of review studies, 10% of all systematic reviews and meta-analyses were selected as a random sample. Also, all economic evaluations were included. Articles were evaluated using checklists, including PRISMA, AMSTAR and QHES with a maximum score of 27, 11 and 100, respectively. The quality score for each criterion as well as the epidemiological and descriptive characteristics of all articles was determined. Data were analyzed using SPSS V. 16 software. Results: After searching the databases, 1084 systematic reviews and meta-analyses were obtained, 10% of which were included in the study. A total of 41 economic evaluations were also included. The mean scores of systematic reviews and meta-analyses based on PRISMA and AMSTAR checklists were 17.04 (5.35) and 5.42 (1.97), respectively, and 68.21 (12.44) for economic evaluations based on QHES. Only three systematic reviews and meta-analysis articles had recorded protocols and 85% of the studies included the terms “systematic review” and “meta-analysis” in their titles. Only one study had been updated. In addition, 81% of the systematic reviews and meta-analyses were published in specialized journals and 47% in Iranian journals. Financial resources and conflict of interests had been mentioned in 33% and 66% of the studies, respectively. Of the selected studies, 60% had evaluated the quality of the articles and 35% of the studies had assessed publication bias. In economic evaluations, 56% had used CEA analysis, 22% CUA analysis, 12% CBA analysis, and one study had used CMA analysis. Of these studies, 54% were model-based health economic studies and 12% were trial-based. The economic perspective was the health care system in most studies. Forty-four percent of the studies had a short time horizon of one year or less, whereas 33% had a lifetime horizon. Moreover, 68% of the studies showed sensitivity analysis and only 5 included the magnitude and direction of the bias. Conclusions: Overall, the reporting and methodological quality of the selected studies were estimated at a moderate level. Based on these results, it is recommended to adopt strategies to reduce preventable errors in studies. Having a primary plan and protocol and registering it as a systematic review can be an important factor in improving the quality of studies. Economic evaluations should also focus on issues, such as economic perspective, time horizon, available bias, and sensitivity analysis.

Download Full-text

An Overview of Systematic Reviews and Meta-Analyses on Acupuncture for Post-Acute Stroke Dysphagia

Geriatrics ◽

10.3390/geriatrics4040068 ◽

2019 ◽

Vol 4 (4) ◽

pp. 68 ◽

Cited By ~ 1

Author(s):

Zi-Yu Tian ◽

Xing Liao ◽

Ying Gao ◽

Shi-Bing Liang ◽

Chong-Yang Zhang ◽

...

Keyword(s):

Acute Stroke ◽

Systematic Reviews ◽

Primary Outcome ◽

Acupuncture Treatment ◽

Meta Analysis ◽

Future Research ◽

Randomized Controlled ◽

Overview Of Systematic Reviews ◽

Meta Analyses

Background: Many randomized controlled trials (RCTs) and systematic reviews (SRs) on acupuncture treatment for post-acute stroke dysphagia have been published. Conflicting results from different SRs necessitated an overview to summarize and assess the quality of this evidence to determine whether acupuncture is effective for this condition. The aim was to evaluate methodological quality and summarizing the evidence for important outcomes. Methods: Seven databases were searched for SRs and/or meta-analysis of RCTs and quasi-RCTs on acupuncture for post-acute stroke dysphagia. Two authors independently identified SRs and meta-analyses, collected data to assess the quality of included SRs and meta analyses according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the revised Assessment of Multiple Systematic Reviews (AMSTAR 2). Results: Searches yielded 382 SRs, 31 were included. The quality of 22 SRs was critically low, five SRs were low, and four Cochrane SRs were moderate when evaluated by AMSTAR2. A total of 17 SRs reported 85.2–96.3% of PRISMA items. Five SRs included explanatory RCTs, 16 SRs included pragmatic RCTs, and 10 SRs included both. Conclusion: Currently, evidence on the effectiveness of acupuncture on post-acute stroke dysphagia is of a low quality. The type of study appeared to have no direct influence on the result, but the primary outcome measures showed a relationship with the quality of SRs. High quality trials with large sample sizes should be the focus of future research.

Download Full-text

Wake up and smell the coffee: caffeine supplementation and exercise performance—an umbrella review of 21 published meta-analyses

British Journal of Sports Medicine ◽

10.1136/bjsports-2018-100278 ◽

2019 ◽

Vol 54 (11) ◽

pp. 681-688 ◽

Cited By ~ 52

Author(s):

Jozo Grgic ◽

Ivana Grgic ◽

Craig Pickering ◽

Brad J Schoenfeld ◽

David J Bishop ◽

...

Keyword(s):

Muscle Strength ◽

Systematic Reviews ◽

Exercise Performance ◽

Methodological Quality ◽

Muscle Endurance ◽

Umbrella Review ◽

Caffeine Ingestion ◽

Quality Of Evidence ◽

Meta Analyses

ObjectiveTo systematically review, summarise and appraise findings of published meta-analyses that examined the effects of caffeine on exercise performance.DesignUmbrella review.Data sourcesTwelve databases.Eligibility criteria for selecting studiesMeta-analyses that examined the effects of caffeine ingestion on exercise performance.ResultsEleven reviews (with a total of 21 meta-analyses) were included, all being of moderate or high methodological quality (assessed using the Assessing the Methodological Quality of Systematic Reviews 2 checklist). In the meta-analyses, caffeine was ergogenic for aerobic endurance, muscle strength, muscle endurance, power, jumping performance and exercise speed. However, not all analyses provided a definite direction for the effect of caffeine when considering the 95% prediction interval. Using the Grading of Recommendations Assessment, Development and Evaluation criteria the quality of evidence was generally categorised as moderate (with some low to very low quality of evidence). Most individual studies included in the published meta-analyses were conducted among young men.Summary/conclusionSynthesis of the currently available meta-analyses suggest that caffeine ingestion improves exercise performance in a broad range of exercise tasks. Ergogenic effects of caffeine on muscle endurance, muscle strength, anaerobic power and aerobic endurance were substantiated by moderate quality of evidence coming from moderate-to-high quality systematic reviews. For other outcomes, we found moderate quality reviews that presented evidence of very low or low quality. It seems that the magnitude of the effect of caffeine is generally greater for aerobic as compared with anaerobic exercise. More primary studies should be conducted among women, middle-aged and older adults to improve the generalisability of these findings.

Download Full-text

Evidence-based umbrella review of cognitive effects of prefrontal tDCS

Social Cognitive and Affective Neuroscience ◽

10.1093/scan/nsaa084 ◽

2020 ◽

Author(s):

Luis C Farhat ◽

Andre F Carvalho ◽

Marco Solmi ◽

Andre R Brunoni

Keyword(s):

Systematic Reviews ◽

Meta Analysis ◽

Measurement Tool ◽

Affective Neuroscience ◽

Umbrella Review ◽

Quality Of Evidence ◽

Neuroscience Research ◽

Cochrane Database ◽

Meta Analyses

Abstract Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique, which has been increasingly used as an investigational tool in neuroscience. In social and affective neuroscience research, the prefrontal cortex has been primarily targeted, since this brain region is critically involved in complex psychobiological processes subserving both ‘hot’ and ‘cold’ domains. Although several studies have suggested that prefrontal tDCS can enhance neuropsychological outcomes, meta-analyses have reported conflicting results. Therefore, we aimed to assess the available evidence by performing an umbrella review of meta-analyses. We evaluated the effects of prefrontal active vs sham tDCS on different domains of cognition among healthy and neuropsychiatric individuals. A MeaSurement Tool to Assess Systematic Reviews 2 was employed to evaluate the quality of meta-analyses, and the GRADE system was employed to grade the quality of evidence of every comparison from each meta-analysis. PubMed/MEDLINE, PsycINFO and the Cochrane Database of Systematic Reviews were searched, and 11 meta-analyses were included resulting in 55 comparisons. Only 16 comparisons reported significant effects favoring tDCS, but 13 of them had either very low or low quality of evidence. Of the remaining 39 comparisons which reported non-significant effects, 38 had either very low or low quality of evidence. Meta-analyses were rated as having critically low and low quality. Among several reasons to explain these findings, the lack of consensus and reproducibility in tDCS research is discussed.

Download Full-text

Methodological Quality Assessment of Systematic Review or Meta-Analysis Using AMSTAR-2: the Long-Term Effectiveness or Efficacy of Opioids for Chronic Non-Cancer Pain

10.21203/rs.2.12469/v1 ◽

2019 ◽

Author(s):

Qian Li ◽

Ke Deng ◽

Xiaoyuan Jiang ◽

Huan Tao ◽

Hui Liu ◽

...

Keyword(s):

Cancer Pain ◽

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Risk Of Bias ◽

Cochrane Central Register ◽

Cochrane Reviews ◽

Meta Analyses

Abstract Background：Systematic review or meta-analysis, the strong study design of high quality evidence, give inconsistent conclusion of long-term effectiveness or efficacy of opioids for chronic non-cancer pain. We appraised the methodological quality of systematic reviews or meta-analyses. Methods: We found the relevant systematic reviews or meta-analyses by searching Medline, EMBASE, the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, the International prospective register of systematic reviews, Psyc ARTICLES/OVID, the Chinese Bio-Medical Literature Database, the China National Knowledge Infrastructure, and the Wan Fang Data and VIP Database on March 1st, 2019. The methodological quality was assessed by A Measurement Tool to Assess Systematic Reviews-2(AMSTAR-2). Spearman correlation analysis and non-parametric tests were used to assess the association between quality and factors. Results: Twenty-one systematic reviews or meta-analyses were included in our study. One has no individual study. In terms of methodological quality, twelve reviews were critically low in overall confidence, four reviews were low, two reviews were moderate, two reviews were high. When referring to the systematic reviews or meta-analyses of relatively better methodological quality with more credible results and conclusions, the effectiveness or efficacy of opioids was small to questionable. Cochrane reviews performed better than non-Cochrane reviews in establishing prior protocol (100% vs 17%, P<0.05), providing an excluded studies list (100% vs 50%, P<0.05) and taking risk of bias into account when interpreting the results of the review (100% vs 75%, P<0.05). There was a strong correlation (ρ=0.526, P<0.05) between the impact factor of systematic reviews or meta-analyses in published journals and methodological quality. Conclusion The methodological quality of the included systematic reviews or meta-analyses is far from satisfactory and needs improvement, especially in establishing prior protocol and justifying significant deviations from the protocol, providing an excluded primary studies list, reporting the funding information of primary studies, and assessing the potential impact of risk of bias on individual studies.

Download Full-text

The Effects of Tai Chi Exercise Among Adults With Chronic Heart Failure: An Overview of Systematic Review and Meta-Analysis

Frontiers in Cardiovascular Medicine ◽

10.3389/fcvm.2021.589267 ◽

2021 ◽

Vol 8 ◽

Author(s):

Jinke Huang ◽

Xiaohui Qin ◽

Min Shen ◽

Yanjuan Xu ◽

Yong Huang

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Tai Chi ◽

Meta Analysis ◽

Reporting Quality ◽

Risk Of Bias ◽

Quality Reporting ◽

Evidence Quality ◽

Meta Analyses

Background: Tai chi (TC) is a popular form of exercise among adults with chronic heart failure (CHF), yet services are greatly underutilized. The aim of the current study was to identify and summarize the existing evidence and to systematically determine the clinical effectiveness of Tai Chi in the management of CHF using a systematic overview.Methods: Both English and Chinese databases were searched for systematic reviews (SRs)/meta-analyses (MAs) on TC for CHF from their inception to June 2020. The methodological quality, reporting quality, and risk of bias of SRs/MAs were assessed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, and Risk of Bias in Systematic reviews (ROBIS), respectively. The evidence quality of outcome measures was assessed by the Grades of Recommendations, Assessment, Development and Evaluation (GRADE).Results: Six SRs/MAs using a quantitative synthesis to assess various outcomes of TC in CHF were included in this overview. The methodological quality, reporting quality and risk of bias of the SRs/MAs and the evidence quality of the outcome measures are generally unsatisfactory. The limitations of the past SRs/MAs included the lack of either the protocol or registration, the list of excluded studies, and the computational details of meta-analysis were inadequately reported. The critical problems were that qualitative data synthesis relied on trials with small sample sizes and critical low quality.Conclusions: TC may be a promising complementary treatment for CHF. However, further rigorous and comprehensive SRs/MAs and RCTs are required to provide robust evidence for definitive conclusions.

Download Full-text