scholarly journals The effect of the CONSORT statement on the amount of “unclear” Risk of Bias reporting in Cochrane Systematic Reviews

PLoS ONE ◽  
2020 ◽  
Vol 15 (7) ◽  
pp. e0235535
Author(s):  
Maaike M. Rademaker ◽  
Geerte G. J. Ramakers ◽  
Adriana L. Smit ◽  
Lotty Hooft ◽  
Inge Stegeman
Keyword(s):  
Systematic Reviews ◽  
Consort Statement ◽  
Risk Of Bias ◽  
The Consort Statement ◽  
Unclear Risk

scholarly journals RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials

2015 ◽  
Vol 23 (1) ◽  
pp. 193-201 ◽  
Author(s):  
Iain J Marshall ◽  
Joël Kuiper ◽  
Byron C Wallace
Keyword(s):  
Machine Learning ◽  
Clinical Trials ◽  
Systematic Reviews ◽  
Risk Of Bias ◽  
Reasonable Accuracy ◽  
Bias Assessment ◽  
Cochrane Database ◽  
The Difference ◽  
Using Data ◽  
Unclear Risk

Abstract Objective To develop and evaluate RobotReviewer, a machine learning (ML) system that automatically assesses bias in clinical trials. From a (PDF-formatted) trial report, the system should determine risks of bias for the domains defined by the Cochrane Risk of Bias (RoB) tool, and extract supporting text for these judgments. Methods We algorithmically annotated 12,808 trial PDFs using data from the Cochrane Database of Systematic Reviews (CDSR). Trials were labeled as being at low or high/unclear risk of bias for each domain, and sentences were labeled as being informative or not. This dataset was used to train a multi-task ML model. We estimated the accuracy of ML judgments versus humans by comparing trials with two or more independent RoB assessments in the CDSR. Twenty blinded experienced reviewers rated the relevance of supporting text, comparing ML output with equivalent (human-extracted) text from the CDSR. Results By retrieving the top 3 candidate sentences per document (top3 recall), the best ML text was rated more relevant than text from the CDSR, but not significantly (60.4% ML text rated ‘highly relevant' v 56.5% of text from reviews; difference +3.9%, [−3.2% to +10.9%]). Model RoB judgments were less accurate than those from published reviews, though the difference was <10% (overall accuracy 71.0% with ML v 78.3% with CDSR). Conclusion Risk of bias assessment may be automated with reasonable accuracy. Automatically identified text supporting bias assessment is of equal quality to the manually identified text in the CDSR. This technology could substantially reduce reviewer workload and expedite evidence syntheses.

scholarly journals Quality Evaluation of Randomized Controlled Trials of Rhodiola Species: A Systematic Review

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Xiuzhu Li ◽  
Weijie Chen ◽  
Yingqi Xu ◽  
Zuanji Liang ◽  
Hao Hu ◽  
...  
Keyword(s):  
Randomized Controlled Trials ◽  
Reporting Quality ◽  
Consort Statement ◽  
Risk Of Bias ◽  
Cochrane Library ◽  
Controlled Trials ◽  
China National Knowledge Infrastructure ◽  
Randomized Controlled ◽  
The Consort Statement ◽  
The Cochrane Library

Background. Rhodiola is a worldwide used medicinal plant for its various medicinal functions, and the number of randomized controlled trials (RCTs) of Rhodiola is increasing in recent years. This study aims to evaluate the reporting quality and risk of bias of the current RCT reports of different Rhodiola species. Methods. Six databases including Embase, PubMed, Web of Science, the Cochrane Library, ClinicalTrial.gov, and China National Knowledge Infrastructure were searched to identify RCTs that used Rhodiola as a single intervention and were published in English or Chinese from inception to December 2020. The Consolidated Standards of Reporting Trials (CONSORT) 2010 statement was used as the checklist for assessment, and a scoring system was applied to the evaluation of RCTs. Score 0 represents no reporting or inadequate reporting, and score 1 represents adequate reporting. The risk of bias of the included studies was also assessed using the Cochrane Risk of Bias tool. Results. A total of 39 RCTs were included in this study, including 23 RCTs of Rhodiola rosea (R. rosea), 8 RCTs of Rhodiola crenulata (R. crenulata), and 8 RCTs of Rhodiola wallichiana (R. wallichiana). None of the included studies met all the CONSORT statement criteria, and the reporting quality of RCTs of the three Rhodiola species was all generally poor. Based on the risk of bias assessment, the majority of included studies were judged to have an unclear risk of bias in most domains due to inadequate reporting. Conclusions. There is inadequate reporting among the included RCTs of different Rhodiola species, and RCTs of Rhodiola with higher reporting quality and better methodological quality are needed.

A critical appraisal of the risk of bias in systematic reviews and metaanalyses pertaining to COVID-19

Coronaviruses ◽  
2020 ◽  
Vol 01 ◽  
Author(s):  
Amit Dang ◽  
Sheshank Madiraju ◽  
Jagan Mohan Venkateswara Rao P ◽  
Navya Sri Gurram ◽  
Sandeep Digijarala ◽  
...  
Keyword(s):  
High Risk ◽  
Systematic Reviews ◽  
Risk Of Bias ◽  
Therapy Outcomes ◽  
Data Synthesis ◽  
Bias Assessment ◽  
Meta Analyses ◽  
Unclear Risk ◽  
Overall Risk

Background: We critically evaluated the risk of bias in published systematic reviews (SRs) and meta-analyses (MAs) pertaining to COVID-19 using ROBIS tool. Materials And Methods: Medline and Cochrane Central Library were searched for SRs/MAs on 14th May 2020, including studies of all designs describing various facets of COVID-19 in humans; no restrictions were applied for interventions, comparators, and outcomes. Two reviewers independently assessed all the SRs/MAs with ROBIS. Results: Out of 204 identified records, 48 SRs/MAs were included. The most frequently reviewed topics were therapy outcomes, diagnosis, and comorbidities (15, 8, and 6 papers respectively). Only 29/48(60.41%) papers had made a mention of using PRISMA or other guidelines for drafting the SR/MA. Only 5/48(10.42%) of all included SRs/MAs had low overall risk of bias as per ROBIS tool; 41/48(85.42%) had high risk of bias, 2/48(4.17%) had unclear risk of bias. The highest proportion of bias was found in data synthesis and findings (30/48, 62.50% of studies had high risk of bias), followed by study identification and selection (29/48, 60.42%). The IRR for methodological quality assessment was substantial, with the Cohen’s kappa values being 0.64, 0.68, 0.62, and 0.75 for domains 1-4 of ROBIS tool, and 0.66 for overall risk of bias assessment. Conclusion: There are serious concerns about the methodology employed to generate the results of the SRs/ MAs pertaining to COVID-19, with ‘quantity’ seemingly being given more importance than ‘quality’ of the paper.

scholarly journals Effect of Crocus sativus (Saffron) Intake on Top of Standard Treatment, on Disease Outcomes and Comorbidities in Patients with Rheumatic Diseases: Synthesis without Meta-Analysis (SWiM) and Level of Adherence to the CONSORT Statement for Randomized Controlled Trials Delivering Herbal Medicine Interventions

Nutrients ◽  
2021 ◽  
Vol 13 (12) ◽  
pp. 4274
Author(s):  
Sotirios G. Tsiogkas ◽  
Maria G. Grammatikopoulou ◽  
Konstantinos Gkiouras ◽  
Efterpi Zafiriou ◽  
Iordanis Papadopoulos ◽  
...  
Keyword(s):  
Herbal Medicine ◽  
Randomized Controlled Trials ◽  
Meta Analysis ◽  
Consort Statement ◽  
Risk Of Bias ◽  
Crocus Sativus ◽  
Controlled Trials ◽  
Randomized Controlled ◽  
Disease Outcomes ◽  
The Consort Statement

Rheumatic diseases (RDs) are often complicated by chronic symptoms and frequent side-effects associated with their treatment. Saffron, a spice derived from the Crocus sativus L. flower, is a popular complementary and alternative medicine among patients with RDs. The present systematic review aimed to summarize the available evidence regarding the efficacy of supplementation with saffron on disease outcomes and comorbidities in patients with RD diagnoses. PubMed, CENTRAL, clinicaltrials.gov and the grey literature were searched until October 2021, and relevant randomized controlled trials (RCTs) were screened for eligibility using Rayyan. Risk of bias was assessed using the Cochrane’s Risk of Bias-2.0 (RoB) tool. A synthesis without meta-analysis (SWiM) was performed by vote counting and an effect direction plot was created. Out of 125 reports, seven fulfilled the eligibility criteria belonging to five RCTs and were included in the SWiM. The RCTs involved patients with rheumatoid arthritis, osteoarthritis and fibromyalgia, and evaluated outcomes related to pain, disease activity, depression, immune response, inflammation, oxidative stress, health, fatigue and functional ability. The majority of trials demonstrated some concerns regarding overall bias. Moreover, the majority of trialists failed to adhere to the formula elaborations suggested by the CONSORT statement for RCTs incorporating herbal medicine interventions. Standardization of herbal medicine confirms its identity, purity and quality; however, the majority of trials failed to adhere to these guidelines. Due to the great heterogeneity and the lack of important information regarding the standardization and content of herbal interventions, it appears that the evidence is not enough to secure a direction of effect for any of the examined outcomes.

scholarly journals The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement

PLoS Biology ◽  
2021 ◽  
Vol 19 (4) ◽  
pp. e3001162
Author(s):  
Christiaan H. Vinkers ◽  
Herm J. Lamberink ◽  
Joeri K. Tijdink ◽  
Pauline Heus ◽  
Lex Bouter ◽  
...  
Keyword(s):  
Outcome Assessment ◽  
Random Sequence ◽  
Consort Statement ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Sequence Generation ◽  
Randomized Controlled ◽  
The Consort Statement ◽  
Random Sequence Generation

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.

scholarly journals Compliance of RCTs in endodontic instrumentation with the CONSORT Statement: A systematic review

2021 ◽  
Vol 10 (8) ◽  
pp. e41910817350
Author(s):  
Karla Aguiar Cabral ◽  
Ana Cristina Rodrigues Martins ◽  
Letícia Maíra Wambier ◽  
Nilton Vivacqua-Gomes ◽  
Márcia Rezende ◽  
...  
Keyword(s):  
Systematic Review ◽  
Significant Influence ◽  
Clinical Studies ◽  
Consort Statement ◽  
Risk Of Bias ◽  
Controlled Trials ◽  
Eligibility Criteria ◽  
Randomized Controlled ◽  
The Consort Statement ◽  
Quality Assessments

This systematic review aimed to evaluate the adherence to the CONSORT Statement in clinical studies comparing different instrumentation techniques in Endodontics.  Methodology: A systematic search was carried out through Medline databases. There was no restriction on publication year or idiom. Solely randomized controlled trials (RCTs) were included. The articles were evaluated in compliance with CONSORT. Descriptive analyses of the number of studies by journal, country, and quality assessments were performed with RoB 2.0 Cochrane Collaboration’s tool for assessing risk of bias in RCTs. A total of 6779 studies were identified, and only 33 meet the eligibility criteria. In relation to the overall risk of bias, eight out of 33 were classified as at ‘low’ risk of bias. The overall CONSORT score for the included studies in this review was 22 ± 6.2 points. Significant differences among countries were observed (p = 0.03). The range of years had a significant influence on the average CONSORT score (p = 0.01). The adherence of RCTs of endodontic instrumentation to the CONSORT Statement requires improvements. Adherence to the CONSORT statement will also reduce the high or some concerns risk of bias of studies in the field. CRD42020160552

scholarly journals Adherence to the CONSORT statement and risk of bias assessment in Randomized Controlled Trials in rehabilitation journals: a protocol for a meta-research study

2021 ◽  
Author(s):  
Tiziano Innocenti ◽  
Silvia Giagio ◽  
Stefano Salvioli ◽  
Daniel Feller ◽  
Silvia Minnucci ◽  
...  
Keyword(s):  
Randomized Controlled Trials ◽  
Random Sample ◽  
Journal Citation Report ◽  
Consort Statement ◽  
Risk Of Bias ◽  
Reporting Guidelines ◽  
Controlled Trials ◽  
Publication Date ◽  
Randomized Controlled ◽  
The Consort Statement

ABSTRACTObjectiveThe aim of this study will be to assess the adherence to the reporting quality standards set forth in the CONSORT Statement checklist of a random sample of randomized controlled trials (RCTs) published in rehabilitation journals, and to assess the association between this adherence and the risk of bias of these RCTs.Methods and AnalysisA cross-sectional analysis is planned on a random sample of 200 RCTs published between 2011and 2020 in the 68 journals indexed under “rehabilitation” category in InCites Journal Citation Report.Randomization will be stratified by publication date and journal ranking (quartile range; Q1-2 and Q3-4) to include an equal number of studies from 2011 to 2015 (Q1-Q2=50 and Q3-Q4=50) and from 2016 to 2020 (Q1-Q2=50 and Q3-Q4=50). RCT with parallel group design will be included. Observational or cohort studies, interim analyses, economic analyses of RCTs, RCT protocols, quasi-experimental design post-trial follow-up studies, subgroup and secondary analyses of previously reported RCTs, RCT with cross-over design, pilot feasibility RCTs, n-of-1 trials, cluster trials, editorials, letters and news reports will be excluded. The primary analysis will address the completeness of the reporting for each study and the relationship between CONSORT adherence and risk of bias. This will be a descriptive analysis through descriptive statistics and graphical representation.Ethics and DisseminationSeveral studies have shown the positive influence of reporting guidelines on the completeness of research reporting but no one investigated the use and the appropriateness of reporting guidelines in physical therapy research. Therefore, this study will add relevant knowledge that may contribute to improve further the reporting of rehabilitation research. The results of this research will be published in a peer-reviewed journal and will be presented at relevant (inter)national scientific events.

A Critical Overview of Systematic Reviews of Chemotherapy for Advanced and Locally Advanced Pancreatic Cancer using both AMSTAR2 and ROBIS as Quality Assessment Tools

2020 ◽  
Vol 15 ◽  
Author(s):  
Amit Dang ◽  
Surendar Chidirala ◽  
Prashanth Veeranki ◽  
BN Vallish
Keyword(s):  
Pancreatic Cancer ◽  
High Risk ◽  
Systematic Reviews ◽  
Locally Advanced ◽  
Advanced Pancreatic Cancer ◽  
Risk Of Bias ◽  
Low Risk ◽  
Locally Advanced Pancreatic Cancer ◽  
Grade 3 ◽  
Critical Overview

Background: We performed a critical overview of published systematic reviews (SRs) of chemotherapy for advanced and locally advanced pancreatic cancer, and evaluated their quality using AMSTAR2 and ROBIS tools. Materials and Methods: PubMed and Cochrane Central Library were searched for SRs on 13th June 2020. SRs with metaanalysis which included only randomized controlled trials and that had assessed chemotherapy as one of the treatment arms were included. The outcome measures, which were looked into, were progression-free survival (PFS), overall survival (OS), and adverse events (AEs) of grade 3 or above. Two reviewers independently assessed all the SRs with both ROBIS and AMSTAR2. Results: Out of the 1,879 identified records, 26 SRs were included for the overview. Most SRs had concluded that gemcitabine-based combination regimes, prolonged OS and PFS, but increased the incidence of grade 3-4 toxicities, when compared to gemcitabine monotherapy, but survival benefits were not consistent when gemcitabine was combined with molecular targeted agents. As per ROBIS, 24/26 SRs had high risk of bias, with only 1/26 SR having low risk of bias. As per AMSTAR2, 25/26 SRs had critically low, and 1/26 SR had low, confidence in the results. The study which scored ‘low’ risk of bias in ROBIS scored ‘low confidence in results’ in AMSTAR2. The inter-rater reliability for scoring the overall confidence in the SRs with AMSTAR2 and the overall domain in ROBIS was substantial; ROBIS: kappa=0.785, SEM=0.207, p<0.001; AMSTAR2: kappa=0.649, SEM=0.323, p<0.001. Conclusion: Gemcitabine-based combination regimens can prolong OS and PFS but also worsen AEs when compared to gemcitabine monotherapy. The included SRs have an overall low methodological quality and high risk of bias as per AMSTAR2 and ROBIS respectively.

Sign in / Sign up

Export Citation Format

Share Document