scholarly journals Systematic differences in effect estimates between observational studies and randomized control trials in meta-analyses in nephrology

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Miho Kimachi ◽  
Akira Onishi ◽  
Aran Tajika ◽  
Kimihiko Kimachi ◽  
Toshi A. Furukawa
Keyword(s):  
Systematic Reviews ◽  
Observational Studies ◽  
Statistical Significance ◽  
Randomized Control Trials ◽  
Causal Inferences ◽  
Randomized Controlled ◽  
The Past ◽  
Randomized Control ◽  
Meta Analyses ◽  
Study Designs

AbstractThe limited availability of randomized controlled trials (RCTs) in nephrology undermines causal inferences in meta-analyses. Systematic reviews of observational studies have grown more common under such circumstances. We conducted systematic reviews of all comparative observational studies in nephrology from 2006 to 2016 to assess the trends in the past decade. We then focused on the meta-analyses combining observational studies and RCTs to evaluate the systematic differences in effect estimates between study designs using two statistical methods: by estimating the ratio of odds ratios (ROR) of the pooled OR obtained from observational studies versus those from RCTs and by examining the discrepancies in their statistical significance. The number of systematic reviews of observational studies in nephrology had grown by 11.7-fold in the past decade. Among 56 records combining observational studies and RCTs, ROR suggested that the estimates between study designs agreed well (ROR 1.05, 95% confidence interval 0.90–1.23). However, almost half of the reviews led to discrepant interpretations in terms of statistical significance. In conclusion, the findings based on ROR might encourage researchers to justify the inclusion of observational studies in meta-analyses. However, caution is needed, as the interpretations based on statistical significance were less concordant than those based on ROR.

scholarly journals Systematic Differences in Effect Estimates Between Observational Studies and Randomized Control Trials in Meta-Analyses Combining Both Study Designs in Nephrology

2020 ◽  
Author(s):  
Miho Kimachi ◽  
Akira Onishi ◽  
Aran Tajika ◽  
Kimihiko Kimachi ◽  
Toshi Furukawa
Keyword(s):  
Systematic Reviews ◽  
Observational Studies ◽  
Statistical Significance ◽  
Randomized Control Trials ◽  
Causal Inferences ◽  
Randomized Controlled ◽  
The Past ◽  
Randomized Control ◽  
Meta Analyses ◽  
Study Designs

Abstract The limited availability of randomized controlled trials (RCTs) in nephrology undermines causal inferences in meta-analyses. Systematic reviews of observational studies have grown more common under such circumstances. We conducted systematic reviews of all comparative observational studies in nephrology from 2006 to 2016 to assess the trends in the past decade. We then focused on the meta-analyses combining observational studies and RCTs to evaluate the systematic differences in effect estimates between study designs using two statistical methods: by estimating the ratio of odds ratios (ROR) of the pooled OR obtained from observational studies versus those from RCTs and by examining the discrepancies in their statistical significance. The number of systematic reviews of observational studies in nephrology had grown by 11.7-fold in the past decade. Among 56 records combining observational studies and RCTs, ROR suggested that the estimates between study designs agreed well (ROR: 1.05, 95% confidence interval: 0.90-1.23). However, almost half of the reviews led to discrepant interpretations in terms of statistical significance. In conclusion, the findings based on ROR might encourage researchers to justify the inclusion of observational studies in meta-analyses. However, caution is needed as the interpretations based on statistical significance were less concordant than those based on ROR.

scholarly journals Analysis of the Type and Study Design of Publications in the Journal of Indian Orthodontic Society and the Changes in Trends Over 2 Decades

2021 ◽  
pp. 030157422098458
Author(s):  
Shivangi Ramteke ◽  
Balasubramanian Madhan
Keyword(s):  
Study Design ◽  
Systematic Reviews ◽  
Case Reports ◽  
Chi Square ◽  
Cross Sectional ◽  
Randomized Controlled ◽  
The Past ◽  
Hierarchy Of Evidence ◽  
Number Of Publications ◽  
Study Designs

Background: The aim of the study was to analyze the type and study design of publications in the Journal of Indian Orthodontic Society (JIOS) and the changes in their trends from 2001 to 2020. Materials and Methods: All the online publications in JIOS were classified as background resource, original articles, and case reports. The original articles were further divided into 7 types based on study design. The annual and overall data were compiled. To evaluate the change in trends, the study period was divided into 2 block years: 2001 to 2010 and 2011 to 2020. The differences in the total number of articles and distribution of articles by type and study design were analyzed using Mann–Whitney and chi-square/Fisher’s exact tests, respectively. Results: Background resource, original articles, and case reports comprised 38.7%, 43.9%, and 17.4% of the total articles (n = 845), respectively. Non-clinical (34.5%) and cross-sectional (32.35%) studies formed the bulk of original articles. There were 17 randomized controlled trials (RCTs) (4.58%) and 5 systematic reviews (1.35%) among the original articles. The total number of publications were higher for years in the second block compared to those in the first (Mdn of 63.5 vs 14.5, P < .001). The differences in the distribution according to type were not statistically significant [ X2(2) = 2.052, P = .35]. A reduction in the share of cohort studies and increase in RCTs were found in the second block (F = 19.174, P = .002). Conclusions: The higher proportion of background resource publications and those with study designs lower in the hierarchy of evidence is a matter of concern. Though slow paced, the increase in the number of RCTs and systematic reviews over the past few years is encouraging.

scholarly journals Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics

BMC Medicine ◽  
2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Perrine Janiaud ◽  
Arnav Agarwal ◽  
Ioanna Tzoulaki ◽  
Evropi Theodoratou ◽  
Konstantinos K. Tsilidis ◽  
...  
Keyword(s):  
Protective Factors ◽  
Observational Studies ◽  
Prediction Interval ◽  
Statistical Significance ◽  
Median Number ◽  
Small Study ◽  
Study Heterogeneity ◽  
Large Heterogeneity ◽  
Meta Analyses ◽  
No Gold Standard

Abstract Background The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies. Methods We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10−6, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance. Results 3744 associations (in 57 umbrella reviews) assessed by a median number of 7 (interquartile range 4 to 11) observational studies were eligible. Most associations were statistically significant at P < 0.05 (61.1%, 2289/3744). Only 2.6% of associations had P < 10−6, ≥1000 cases (or ≥20,000 participants for continuous factors), P < 0.05 in the largest study, 95% prediction interval excluding the null, and no large between-study heterogeneity, small study effects, or excess significance. Across the 57 topics, large heterogeneity was observed in the proportion of associations fulfilling various quantitative criteria. The quantitative criteria were mostly independent from one another. Across 62 associations assessed in both RCTs and in observational studies, 37.1% had effect estimates in opposite directions and 43.5% had effect estimates differing beyond chance in the two designs. Across 94 comparisons assessed in both MR and observational studies, such discrepancies occurred in 30.8% and 54.7%, respectively. Conclusions Acknowledging that no gold-standard exists to judge whether an observational association is genuine, statistically significant results are common in observational studies, but they are rarely convincing or corroborated by randomized evidence.

scholarly journals Concordance between systematic reviews of randomized controlled trials in assisted reproduction: an overview

2020 ◽  
Vol 2020 (4) ◽  
Author(s):  
Mariano Mascarenhas ◽  
Theodoros Kalampokas ◽  
Sesh Kamal Sunkara ◽  
Mohan S Kamath
Keyword(s):  
Assisted Reproduction ◽  
Systematic Reviews ◽  
Selective Reporting ◽  
Methodological Issues ◽  
Quality Of Evidence ◽  
Randomized Controlled ◽  
Cochrane Reviews ◽  
Secondary Outcomes ◽  
Meta Analyses

Abstract STUDY QUESTION Are systematic reviews published within a 3-year period on interventions in ART concordant in their conclusions? SUMMARY ANSWER The majority of the systematic reviews published within a 3-year period in the field of assisted reproduction on the same topic had discordant conclusions. WHAT IS KNOWN ALREADY Systematic reviews and meta-analyses have now replaced individual randomized controlled trials (RCTs) at the top of the evidence pyramid. There has been a proliferation of systematic reviews and meta-analyses, many of which suffer from methodological issues and provide varying conclusions. STUDY DESIGN, SIZE, DURATION We assessed nine interventions in women undergoing ART with at least three systematic reviews each, published from January 2015 to December 2017. PARTICIPANTS/MATERIALS, SETTING, METHODS The systematic reviews which included RCTs were considered eligible for inclusion. The primary outcome was extent of concordance between systematic reviews on the same topic. Secondary outcomes included assessment of quality of systematic reviews, differences in included studies in meta-analyses covering the same search period, selective reporting and reporting the quality of evidence. MAIN RESULTS AND THE ROLE OF CHANCE Concordant results and conclusions were found in only one topic, with reviews in the remaining eight topics displaying partial discordance. The AMSTAR grading for the majority of the non-Cochrane reviews was critically low whilst it was categorized as high for all of the Cochrane reviews. For three of the nine topics, none of the included systematic reviews assessed the quality of evidence. We were unable to assess selective reporting as most of the reviews did not have a pre-specified published protocol. LIMITATIONS, REASONS FOR CAUTION We were limited by the high proportion of reviews lacking a pre-specified protocol, which made it impossible to assess for selective reporting. Furthermore, many reviews did not specify primary and secondary outcomes which made it difficult to assess reporting bias. All the authors of this review were Cochrane review authors which may introduce some assessment bias. The categorization of the review’s conclusions as beneficial, harmful or neutral was subjective, depending on the tone and wording of the conclusion section of the review. WIDER IMPLICATIONS OF THE FINDINGS The majority of the systematic reviews published within a 3-year period on the same topic in the field of assisted reproduction revealed discordant conclusions and suffered from serious methodological issues, hindering the process of informed healthcare decision-making. STUDY FUNDING/COMPETING INTEREST(S) All the authors are Cochrane authors. M.S.K. is an editorial board member of Cochrane Gynaecology and Fertility group. No grant from funding agencies in the public, commercial or not-for-profit sectors was obtained.

The Dietary Inflammatory Index and Human Health: An Umbrella Review of Meta-Analyses of Observational Studies

2021 ◽  
Author(s):  
Wolfgang Marx ◽  
Nicola Veronese ◽  
Jaimon T Kelly ◽  
Lee Smith ◽  
Meghan Hockey ◽  
...  
Keyword(s):  
Chronic Disease ◽  
Health Outcomes ◽  
Observational Studies ◽  
Dietary Inflammatory Index ◽  
Credibility Assessment ◽  
Umbrella Review ◽  
Inflammatory Index ◽  
Increased Risk ◽  
Meta Analyses ◽  
Study Designs

ABSTRACT Numerous observational studies have investigated the role of the Dietary Inflammatory Index (DII®) in chronic disease risk. The aims of this umbrella review and integrated meta-analyses were to systematically synthesize the observational evidence reporting on the associations between the DII and health outcomes based on meta-analyses, and to assess the quality and strength of the evidence for each associated outcome. This umbrella review with integrated meta-analyses investigated the association between the DII and a range of health outcomes based on meta-analyses of observational data. A credibility assessment was conducted for each outcome using the following criteria: statistical heterogeneity, 95% prediction intervals, evidence for small-study effect and/or excess significance bias, as well as effect sizes and P values using calculated random effects meta-analyses. In total, 15 meta-analyses reporting on 38 chronic disease-related outcomes were included, incorporating a total population of 4,360,111 subjects. Outcomes (n = 38) were examined through various study designs including case-control (n = 8), cross-sectional (n = 5), prospective (n = 5), and combination (n = 20) study designs. Adherence to a pro-inflammatory dietary pattern had a significant positive association with 27 (71%) of the included health outcomes (P value &lt; 0.05). Using the credibility assessment, Class I (Convincing) evidence was identified for myocardial infarction only, Class II (Highly suggestive) evidence was identified for increased risk of all-cause mortality, overall risk of incident cancer, and risk of incident site-specific cancers (colorectal, pancreatic, respiratory, and oral cancers) with increasing (more pro-inflammatory) DII score. Most outcomes (n = 31) presented Class III (Suggestive) or lower evidence (Weak or No association). Pro-inflammatory dietary patterns were nominally associated with an increased risk of many chronic disease outcomes. However, the strength of evidence for most outcomes was limited. Further prospective studies are required to improve the precision of the effect size.

Postoperative patient-controlled analgesia: Meta-analyses of initial randomized control trials

1993 ◽  
Vol 5 (3) ◽  
pp. 182-193 ◽  
Author(s):  
Jane C. Ballantyne ◽  
Daniel B. Carr ◽  
Thomas C. Chalmers ◽  
Keith B.G. Dear ◽  
Italo F. Angelillo ◽  
...  
Keyword(s):  
Randomized Control Trials ◽  
Patient Controlled Analgesia ◽  
Postoperative Patient ◽  
Randomized Control ◽  
Meta Analyses

Biomedical conferences’ author instructions rarely mention guidelines for reporting abstracts of trials and systematic reviews

2020 ◽  
Vol 9 (2) ◽  
pp. 83-91
Author(s):  
Lenko Saric ◽  
Svjetlana Dosenovic ◽  
Jakov Mihanovic ◽  
Livia Puljak
Keyword(s):  
Systematic Review ◽  
Randomized Controlled Trial ◽  
Systematic Reviews ◽  
Controlled Trial ◽  
Reporting Guidelines ◽  
Randomized Controlled ◽  
Meta Analyses

Aim: To analyze whether instructions for authors of biomedical conference abstracts mention guidelines for writing randomized controlled trial and systematic review abstracts and to evaluate reasons for their absence from instructions. Materials & methods: We analyzed instructions for authors of biomedical conferences advertized in 2019 and assessed whether they mentioned Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. We surveyed contact persons from abstract/publication committees of selected conferences to analyze why relevant guidelines were missing. Results: Instructions for abstracts were available for 819 conferences. Only two (0.2%) had reporting instructions for randomized controlled trial/systematic review authors. Almost half of the contacted conference organizers whose response we received were not aware of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. Conclusion: Conference organizers do not require and are not familiar enough with reporting guidelines.

scholarly journals An Overview of Meta-Analyses of Danhong Injection for Unstable Angina

2015 ◽  
Vol 2015 ◽  
pp. 1-8 ◽  
Author(s):  
Xiaoxia Zhang ◽  
Hui Wang ◽  
Yanxu Chang ◽  
Yuefei Wang ◽  
Xiang Lei ◽  
...  
Keyword(s):  
Unstable Angina ◽  
Systematic Reviews ◽  
Conventional Medicine ◽  
Reporting Quality ◽  
Randomized Controlled ◽  
Danhong Injection ◽  
Statistical Heterogeneity ◽  
Comprehensive Search ◽  
Meta Analyses

Objective. To systematically collect evidence and evaluate the effects of Danhong injection (DHI) for unstable angina (UA).Methods. A comprehensive search was conducted in seven electronic databases up to January 2015. The methodological and reporting quality of included studies was assessed by using AMSTAR and PRISMA.Result. Five articles were included. The conclusions suggest that DHI plus conventional medicine treatment was effective for UA pectoris treatment, could alleviate symptoms of angina and ameliorate electrocardiograms. Flaws of the original studies and systematic reviews weaken the strength of evidence. Limitations of the methodology quality include performing an incomprehensive literature search, lacking detailed characteristics, ignoring clinical heterogeneity, and not assessing publication bias and other forms of bias. The flaws of reporting systematic reviews included the following: not providing a structured summary, no standardized search strategy. For the pooled findings, researchers took statistical heterogeneity into consideration, but clinical and methodology heterogeneity were ignored.Conclusion. DHI plus conventional medicine treatment generally appears to be effective for UA treatment. However, the evidence is not hard enough due to methodological flaws in original clinical trials and systematic reviews. Furthermore, rigorous designed randomized controlled trials are also needed. The methodology and reporting quality of systematic reviews should be improved.

Sign in / Sign up

Export Citation Format

Share Document