Concordance between systematic reviews of randomized controlled trials in assisted reproduction: an overview

Abstract STUDY QUESTION Are systematic reviews published within a 3-year period on interventions in ART concordant in their conclusions? SUMMARY ANSWER The majority of the systematic reviews published within a 3-year period in the field of assisted reproduction on the same topic had discordant conclusions. WHAT IS KNOWN ALREADY Systematic reviews and meta-analyses have now replaced individual randomized controlled trials (RCTs) at the top of the evidence pyramid. There has been a proliferation of systematic reviews and meta-analyses, many of which suffer from methodological issues and provide varying conclusions. STUDY DESIGN, SIZE, DURATION We assessed nine interventions in women undergoing ART with at least three systematic reviews each, published from January 2015 to December 2017. PARTICIPANTS/MATERIALS, SETTING, METHODS The systematic reviews which included RCTs were considered eligible for inclusion. The primary outcome was extent of concordance between systematic reviews on the same topic. Secondary outcomes included assessment of quality of systematic reviews, differences in included studies in meta-analyses covering the same search period, selective reporting and reporting the quality of evidence. MAIN RESULTS AND THE ROLE OF CHANCE Concordant results and conclusions were found in only one topic, with reviews in the remaining eight topics displaying partial discordance. The AMSTAR grading for the majority of the non-Cochrane reviews was critically low whilst it was categorized as high for all of the Cochrane reviews. For three of the nine topics, none of the included systematic reviews assessed the quality of evidence. We were unable to assess selective reporting as most of the reviews did not have a pre-specified published protocol. LIMITATIONS, REASONS FOR CAUTION We were limited by the high proportion of reviews lacking a pre-specified protocol, which made it impossible to assess for selective reporting. Furthermore, many reviews did not specify primary and secondary outcomes which made it difficult to assess reporting bias. All the authors of this review were Cochrane review authors which may introduce some assessment bias. The categorization of the review’s conclusions as beneficial, harmful or neutral was subjective, depending on the tone and wording of the conclusion section of the review. WIDER IMPLICATIONS OF THE FINDINGS The majority of the systematic reviews published within a 3-year period on the same topic in the field of assisted reproduction revealed discordant conclusions and suffered from serious methodological issues, hindering the process of informed healthcare decision-making. STUDY FUNDING/COMPETING INTEREST(S) All the authors are Cochrane authors. M.S.K. is an editorial board member of Cochrane Gynaecology and Fertility group. No grant from funding agencies in the public, commercial or not-for-profit sectors was obtained.

Download Full-text

Tools to Assess the External Validity of Randomized Controlled Trials in Systematic Reviews: a Systematic Review of Measurement Properties

10.21203/rs.3.rs-830181/v1 ◽

2021 ◽

Author(s):

Andres Jung ◽

Julia Balzer ◽

Tobias Braun ◽

Kerstin Luedtke

Keyword(s):

Randomized Controlled Trials ◽

Systematic Reviews ◽

External Validity ◽

Measurement Properties ◽

Controlled Trials ◽

Measurement Instruments ◽

Health Measurement ◽

Quality Of Evidence ◽

Randomized Controlled

Abstract Background: Internal and external validity are the most relevant components when critically appraising randomized controlled trials (RCTs) for systematic reviews. However, there is no gold standard to assess external validity. This might be related to the heterogeneity of terminology as well as to unclear evidence of the measurement properties of available tools. The aim of this review was to identify tools to assess the external validity of RCTs in systematic reviews and to evaluate the quality of evidence regarding their measurement properties.Methods: A two-phase systematic literature search was performed in four databases: MEDLINE via PubMed, Scopus, PsycINFO via OVID, and CINAHL via EBSCO. First, tools to assess the external validity of RCTs were identified. Second, studies aiming to investigate the measurement properties of these tools were selected. The measurement properties of each included tool were appraised using an adapted version of the COnsensus based Standards for the selection of health Measurement INstruments (COSMIN) guidelines.Results: 34 publications reporting on the development or validation of 26 included tools were included. For 62% of the included tools, there was no evidence of any measurement property. For the remaining tools, reliability was assessed most frequently. Reliability was judged as “sufficient” for three tools (very low quality of evidence). Content validity was rated as “sufficient” for one tool (moderate quality of evidence).Conclusions: Based on these results, no available tool can be fully recommended to assess the external validity of RCTs in systematic reviews. Several steps are required to overcome the identified difficulties to either adapt and validate available tools or to develop a new one. There is a need for more research for this purpose.Trial registration: Prospective registration at Open Science Framework (OSF): https://doi.org/10.17605/OSF.IO/PTG4D

Download Full-text

An Overview of Meta-Analyses of Danhong Injection for Unstable Angina

Evidence-based Complementary and Alternative Medicine ◽

10.1155/2015/358028 ◽

2015 ◽

Vol 2015 ◽

pp. 1-8 ◽

Cited By ~ 6

Author(s):

Xiaoxia Zhang ◽

Hui Wang ◽

Yanxu Chang ◽

Yuefei Wang ◽

Xiang Lei ◽

...

Keyword(s):

Unstable Angina ◽

Systematic Reviews ◽

Conventional Medicine ◽

Reporting Quality ◽

Randomized Controlled ◽

Danhong Injection ◽

Statistical Heterogeneity ◽

Comprehensive Search ◽

Meta Analyses

Objective. To systematically collect evidence and evaluate the effects of Danhong injection (DHI) for unstable angina (UA).Methods. A comprehensive search was conducted in seven electronic databases up to January 2015. The methodological and reporting quality of included studies was assessed by using AMSTAR and PRISMA.Result. Five articles were included. The conclusions suggest that DHI plus conventional medicine treatment was effective for UA pectoris treatment, could alleviate symptoms of angina and ameliorate electrocardiograms. Flaws of the original studies and systematic reviews weaken the strength of evidence. Limitations of the methodology quality include performing an incomprehensive literature search, lacking detailed characteristics, ignoring clinical heterogeneity, and not assessing publication bias and other forms of bias. The flaws of reporting systematic reviews included the following: not providing a structured summary, no standardized search strategy. For the pooled findings, researchers took statistical heterogeneity into consideration, but clinical and methodology heterogeneity were ignored.Conclusion. DHI plus conventional medicine treatment generally appears to be effective for UA treatment. However, the evidence is not hard enough due to methodological flaws in original clinical trials and systematic reviews. Furthermore, rigorous designed randomized controlled trials are also needed. The methodology and reporting quality of systematic reviews should be improved.

Download Full-text

Self-concept in poor readers: a systematic review and meta-analysis

PeerJ ◽

10.7717/peerj.8772 ◽

2020 ◽

Vol 8 ◽

pp. e8772 ◽

Cited By ~ 2

Author(s):

Genevieve M. McArthur ◽

Nicola Filardi ◽

Deanna A. Francis ◽

Mark E. Boyes ◽

Nicholas A. Badcock

Keyword(s):

Systematic Review ◽

Primary Outcome ◽

Strong Association ◽

Physical Appearance ◽

Self Concept ◽

Poor Reading ◽

Quality Of Evidence ◽

Secondary Outcomes ◽

Meta Analyses

Background The aims of this systematic review and meta-analyses were to determine if there is a statistically reliable association between poor reading and poor self-concept, and if such an association is moderated by domain of self-concept, type of reading impairment, or contextual factors including age, gender, reading instruction, and school environment. Methodology We searched 10 key databases for published and unpublished studies, as well as reference lists of included studies, and studies that cited included studies. We calculated standardised mean differences (SMDs) and 95% confidence intervals for one primary outcome (average self-concept) and 10 secondary outcomes (10 domains of self-concept). We assessed the data for risk of bias, heterogeneity, sensitivity, reporting bias, and quality of evidence. Results Thirteen studies with 3,348 participants met our selection criteria. Meta-analyses revealed statistically significant SMDs for average self-concept (−0.57) and five domains of self-concept (reading/writing/spelling: −1.03; academic: −0.67; math: −0.64; behaviour: −0.32; physical appearance: −0.28). The quality of evidence for the primary outcome was moderate, and for secondary outcomes was low, due to lack of data. Conclusions These outcomes suggest a probable moderate association between poor reading and average self-concept; a possible strong association between poor reading and reading-writing-spelling self-concept; and possible moderate associations between poor reading and self-concept in the self-concept domains of academia, mathematics, behaviour, and physical appearance.

Download Full-text

Overview of systematic reviews - a new type of study. Part II

Sao Paulo Medical Journal ◽

10.1590/1516-3180.2013.8150015 ◽

2014 ◽

Vol 133 (3) ◽

pp. 206-217 ◽

Cited By ~ 16

Author(s):

Valter Silva ◽

Antonio Jose Grande ◽

Alan Pedrosa Viegas de Carvalho ◽

Ana Luiza Cabrera Martimbianco ◽

Rachel Riera

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

Meta Analysis ◽

Quality Of Evidence ◽

Overview Of Systematic Reviews ◽

New Type ◽

The Mean ◽

Meta Analyses ◽

General Profile

CONTEXT AND OBJECTIVE: Overviews of Systematic Reviews (OoRs) are a new type of study in which multiple evidence from systematic reviews (SRs) is compiled into an accessible and useful document. The aim here was to describe the state of the art and critically assess Cochrane OoRs that have been published.DESIGN AND SETTING: Descriptive study conducted at a research center.METHODS: The OoRs identified through the filter developed in Part I of this study were evaluated in five domains: methodological quality; quality of evidence; implications for practice; general profile of OoRs; and length of work.RESULTS: All 13 OoRs included had high methodological quality. Some OoRs did not present sufficient data to judge the quality of evidence; using sensitivity analysis, the quality of evidence of the OoRs increased. Regarding implications for practice, 64% of the interventions were judged as beneficial or harmful, while 36% of them showed insufficient evidence for judgment. It is expected (with 95% confidence interval) that one OoR will include 9,462 to 64,469 patients, 9 to 29 systematic reviews and 80 to 344 primary studies, and assess 6 to 21 interventions; and that 50 to 92% of OoRs will produce meta-analysis. The OoRs generated 2 to 26 meta-analyses over a period of 18 to 31 months.CONCLUSION: The OoRs presented high methodological quality; the quality of evidence tended to be moderate/high; most interventions were judged to be beneficial/harmful; the mean length of work was 24 months. The OoR profile adds power to decision-making.

Download Full-text

Impact of Prophylactic Hydroxychloroquine on People at High Risk of COVID-19: A Systematic Review and Meta-Analysis

Journal of Clinical Medicine ◽

10.3390/jcm10122609 ◽

2021 ◽

Vol 10 (12) ◽

pp. 2609

Author(s):

Adrian V. Hernandez ◽

John Ingemi ◽

Michael Sherman ◽

Vinay Pasupuleti ◽

Joshuan J. Barboza ◽

...

Keyword(s):

Meta Analysis ◽

Post Exposure Prophylaxis ◽

Rt Pcr ◽

Randomized Controlled ◽

All Cause Mortality ◽

Secondary Outcomes ◽

Meta Analyses ◽

Exposure Prophylaxis ◽

Clinical Worsening

There are no proven prophylactic interventions for COVID-19. We systematically reviewed the efficacy of prophylactic hydroxychloroquine for COVID-19. Studies evaluating hydroxychloroquine for prophylaxis of COVID-19 were searched in several engines until 8 December 2020. Primary outcomes included RT-PCR positivity, COVID-19 infections (positive RT-PCR or compatible COVID-19 symptoms), and all-cause mortality. Random effects meta-analyses were performed for all outcomes. Five randomized controlled trials (RCTs) (n = 5579) and one cohort (n = 106) were included. Placebo was the comparator in four RCTs, and usual care in one RCT. Compared to the controls, five RCTs showed that hydroxychloroquine prophylaxis did not reduce RT-PCR positivity (RR 1.01, 95% CI 0.88–1.16), COVID-19 infection (RR 0.98, 95% CI 0.78–1.22), or all-cause mortality (RR 0.73, 95% CI 0.27–1.99). There were no differences of effects by pre- or post-exposure prophylaxis. Prophylaxis with hydroxychloroquine increased the risk of diarrhea, abdominal pain, or vomiting (RR 4.56, 95% CI 1.58–13.19). There were no effects of hydroxychloroquine on other secondary outcomes. Quality of evidence was low to very low for all outcomes. Hydroxychloroquine was not efficacious as a prophylaxis for COVID-19 infections, defined either as RT-PCR positivity or as a composite of RT-PCR positivity or compatible symptoms. Hydroxychloroquine did not reduce all-cause mortality, clinical worsening, or adverse events.

Download Full-text

How to Put It Plainly? Findings From Two Randomized Controlled Studies on Writing Plain Language Summaries for Psychological Meta-Analyses

Frontiers in Psychology ◽

10.3389/fpsyg.2021.771399 ◽

2021 ◽

Vol 12 ◽

Author(s):

Martin Kerwer ◽

Marlene Stoll ◽

Mark Jonas ◽

Gesa Benz ◽

Anita Chasiotis

Keyword(s):

User Experience ◽

Plain Language ◽

Quality Of Evidence ◽

Randomized Controlled ◽

Societal Relevance ◽

Randomized Controlled Studies ◽

Research Findings ◽

Meta Analyses ◽

Technical Terms

Plain language summaries (PLS) aim to communicate research findings to laypersons in an easily understandable manner. Despite the societal relevance of making psychological research findings available to the public, our empirical knowledge on how to write PLS of psychology studies is still scarce. In this article, we present two experimental studies investigating six characteristics of PLS for psychological meta-analyses. We specifically focused on approaches for (1) handling technical terms, (2) communicating the quality of evidence by explaining the methodological approach of meta-analyses, (3) explaining how synthesized studies operationalized their research questions, (4) handling statistical terms, (5) structuring PLS, and (6) explaining complex meta-analytic designs. To develop empirically validated guidelines on writing PLS, two randomized controlled studies including large samples stratified for education status, age, and gender (NStudy1=2,288 and NStudy2=2,211) were conducted. Eight PLS of meta-analyses from different areas of psychology were investigated as study materials. Main outcome variables were user experience (i.e., perceived accessibility, perceived understanding, and perceived empowerment) and knowledge acquisition, as well as understanding and knowledge of the quality of evidence. Overall, our hypotheses were partially confirmed, with our results underlining, among other things, the importance of explaining or replacing content-related technical terms (i.e., theoretical concepts) and indicating the detrimental effects of providing too many details on statistical concepts on user experience. Drawing on these and further findings, we derive five empirically well-founded rules on the lay-friendly communication of meta-analytic research findings in psychology. Implications for PLS authors and future research on PLS are discussed.

Download Full-text

UPDATE: Should Hydroxychloroquine (HCQ) or Chloroquine (CQ) be used in the treatment of COVID-19?

Acta Medica Philippina ◽

10.47895/amp.v54i0.2446 ◽

2020 ◽

Vol 54 ◽

Author(s):

Lia M. Palileo-Villanueva ◽

Elenore Judy B. Uy

Keyword(s):

Clinical Benefit ◽

Common Adverse Event ◽

Hospitalized Patients ◽

Efficacy And Safety ◽

Quality Of Evidence ◽

Randomized Controlled ◽

The Us ◽

Us Fda ◽

Meta Analyses

KEY FINDINGS There is insufficient evidence to support the routine use of HCQ or CQ for the treatment of COVID-19. Results from interim analyses of 2 large RCTs, the Recovery and the Solidarity trials, reportedly showed no clinical benefit from HCQ for hospitalized patients with COVID-19. There are 3 randomized controlled trials that investigated the efficacy and safety of HCQ compared to standard therapy. Overall quality of evidence was very low. Meta-analyses from the “COVID-19 Living Data” project suggests that the use of HCQ may increase the incidence of adverse events at day 14 to day 28 (RR 2.49, 95% confidence interval: 1.04 to 5.98, moderate quality of evidence); the most common adverse event across the two trials is diarrhea (n=8). In a statement dated June 5, 2020, the investigators of the Recovery trial announced their decision to halt further enrollment to the HCQ arm of the trial because an interim analysis showed no clinical benefit from the use of HCQ in hospitalized patients with COVID. On June 15, 2020, the US FDA revoked the emergency use authorization for HCQ and CQ as treatment for COVID-19. On June 18, 2020, the WHO announced that recruitment to the HCQ arm of the Solidarity trial has been halted.

Download Full-text

An Overview of Systematic Reviews and Meta-Analyses on Acupuncture for Post-Acute Stroke Dysphagia

Geriatrics ◽

10.3390/geriatrics4040068 ◽

2019 ◽

Vol 4 (4) ◽

pp. 68 ◽

Cited By ~ 1

Author(s):

Zi-Yu Tian ◽

Xing Liao ◽

Ying Gao ◽

Shi-Bing Liang ◽

Chong-Yang Zhang ◽

...

Keyword(s):

Acute Stroke ◽

Systematic Reviews ◽

Primary Outcome ◽

Acupuncture Treatment ◽

Meta Analysis ◽

Future Research ◽

Randomized Controlled ◽

Overview Of Systematic Reviews ◽

Meta Analyses

Background: Many randomized controlled trials (RCTs) and systematic reviews (SRs) on acupuncture treatment for post-acute stroke dysphagia have been published. Conflicting results from different SRs necessitated an overview to summarize and assess the quality of this evidence to determine whether acupuncture is effective for this condition. The aim was to evaluate methodological quality and summarizing the evidence for important outcomes. Methods: Seven databases were searched for SRs and/or meta-analysis of RCTs and quasi-RCTs on acupuncture for post-acute stroke dysphagia. Two authors independently identified SRs and meta-analyses, collected data to assess the quality of included SRs and meta analyses according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the revised Assessment of Multiple Systematic Reviews (AMSTAR 2). Results: Searches yielded 382 SRs, 31 were included. The quality of 22 SRs was critically low, five SRs were low, and four Cochrane SRs were moderate when evaluated by AMSTAR2. A total of 17 SRs reported 85.2–96.3% of PRISMA items. Five SRs included explanatory RCTs, 16 SRs included pragmatic RCTs, and 10 SRs included both. Conclusion: Currently, evidence on the effectiveness of acupuncture on post-acute stroke dysphagia is of a low quality. The type of study appeared to have no direct influence on the result, but the primary outcome measures showed a relationship with the quality of SRs. High quality trials with large sample sizes should be the focus of future research.

Download Full-text

Wake up and smell the coffee: caffeine supplementation and exercise performance—an umbrella review of 21 published meta-analyses

British Journal of Sports Medicine ◽

10.1136/bjsports-2018-100278 ◽

2019 ◽

Vol 54 (11) ◽

pp. 681-688 ◽

Cited By ~ 52

Author(s):

Jozo Grgic ◽

Ivana Grgic ◽

Craig Pickering ◽

Brad J Schoenfeld ◽

David J Bishop ◽

...

Keyword(s):

Muscle Strength ◽

Systematic Reviews ◽

Exercise Performance ◽

Methodological Quality ◽

Muscle Endurance ◽

Umbrella Review ◽

Caffeine Ingestion ◽

Quality Of Evidence ◽

Meta Analyses

ObjectiveTo systematically review, summarise and appraise findings of published meta-analyses that examined the effects of caffeine on exercise performance.DesignUmbrella review.Data sourcesTwelve databases.Eligibility criteria for selecting studiesMeta-analyses that examined the effects of caffeine ingestion on exercise performance.ResultsEleven reviews (with a total of 21 meta-analyses) were included, all being of moderate or high methodological quality (assessed using the Assessing the Methodological Quality of Systematic Reviews 2 checklist). In the meta-analyses, caffeine was ergogenic for aerobic endurance, muscle strength, muscle endurance, power, jumping performance and exercise speed. However, not all analyses provided a definite direction for the effect of caffeine when considering the 95% prediction interval. Using the Grading of Recommendations Assessment, Development and Evaluation criteria the quality of evidence was generally categorised as moderate (with some low to very low quality of evidence). Most individual studies included in the published meta-analyses were conducted among young men.Summary/conclusionSynthesis of the currently available meta-analyses suggest that caffeine ingestion improves exercise performance in a broad range of exercise tasks. Ergogenic effects of caffeine on muscle endurance, muscle strength, anaerobic power and aerobic endurance were substantiated by moderate quality of evidence coming from moderate-to-high quality systematic reviews. For other outcomes, we found moderate quality reviews that presented evidence of very low or low quality. It seems that the magnitude of the effect of caffeine is generally greater for aerobic as compared with anaerobic exercise. More primary studies should be conducted among women, middle-aged and older adults to improve the generalisability of these findings.

Download Full-text

Evidence-based umbrella review of cognitive effects of prefrontal tDCS

Social Cognitive and Affective Neuroscience ◽

10.1093/scan/nsaa084 ◽

2020 ◽

Author(s):

Luis C Farhat ◽

Andre F Carvalho ◽

Marco Solmi ◽

Andre R Brunoni

Keyword(s):

Systematic Reviews ◽

Meta Analysis ◽

Measurement Tool ◽

Affective Neuroscience ◽

Umbrella Review ◽

Quality Of Evidence ◽

Neuroscience Research ◽

Cochrane Database ◽

Meta Analyses

Abstract Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique, which has been increasingly used as an investigational tool in neuroscience. In social and affective neuroscience research, the prefrontal cortex has been primarily targeted, since this brain region is critically involved in complex psychobiological processes subserving both ‘hot’ and ‘cold’ domains. Although several studies have suggested that prefrontal tDCS can enhance neuropsychological outcomes, meta-analyses have reported conflicting results. Therefore, we aimed to assess the available evidence by performing an umbrella review of meta-analyses. We evaluated the effects of prefrontal active vs sham tDCS on different domains of cognition among healthy and neuropsychiatric individuals. A MeaSurement Tool to Assess Systematic Reviews 2 was employed to evaluate the quality of meta-analyses, and the GRADE system was employed to grade the quality of evidence of every comparison from each meta-analysis. PubMed/MEDLINE, PsycINFO and the Cochrane Database of Systematic Reviews were searched, and 11 meta-analyses were included resulting in 55 comparisons. Only 16 comparisons reported significant effects favoring tDCS, but 13 of them had either very low or low quality of evidence. Of the remaining 39 comparisons which reported non-significant effects, 38 had either very low or low quality of evidence. Meta-analyses were rated as having critically low and low quality. Among several reasons to explain these findings, the lack of consensus and reproducibility in tDCS research is discussed.

Download Full-text