scholarly journals Judgments of risk of bias associated with random sequence generation in trials included in Cochrane systematic reviews are frequently erroneous

2018 ◽  
Author(s):  
Ognjen Barcot ◽  
Matija Boric ◽  
Tina Poklepovic Pericic ◽  
Marija Cavar ◽  
Svjetlana Dosenovic ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Random Number ◽  
Random Sequence ◽  
Computer Software ◽  
Risk Of Bias ◽  
Controlled Trials ◽  
Sequence Generation ◽  
Randomized Controlled ◽  
Random Sequence Generation ◽  
Random Number Table

AbstractBackgroundPurpose of this study was to analyze adequacy of judgments about risk of bias (RoB) for random sequence generation in Cochrane systematic reviews (CSRs) of randomized controlled trials (RCTs).MethodsInformation was extracted from RoB tables of CSRs using automated data scraping. We categorized all comments provided as supports for judgments for RoB related to randomization. We analyzed number and type of various supporting comments and assessed adequacy of RoB judgment for randomization in line with recommendations from the Cochrane Handbook.ResultsWe analyzed 10527 RCTs that were included in 729 CSRs. For 5682 RCTs randomization was not described; for the others it was indicated randomization was done using computer/software/internet (N=2886), random number table (N=888), mechanic method (N=366), or it was incomplete/inappropriate (N=303).Overall, 1194/10125 trials (12%) had erroneous RoB judgment about randomization. The highest proportion of errors was found for trials with high RoB (28%), followed by those with low (19%), or unclear (3%). Therefore, one in eight judgments for the analyzed domain in CSRs was erroneous, and one in three if the judgment was “high risk”.ConclusionCochrane systematic reviews cannot be necessarily trusted when it comes to judgments for risk of bias related to randomized sequence generation.

scholarly journals Randomized clinical trial quality has improved over time but is still not good enough: an analysis of 176,620 randomized controlled trials published between 1966 and 2018

2020 ◽  
Author(s):  
Christiaan H. Vinkers ◽  
Herm J. Lamberink ◽  
Joeri K. Tijdink ◽  
Pauline Heus ◽  
Lex Bouter ◽  
...  
Keyword(s):  
Random Sequence ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Trial Registration ◽  
Controlled Trials ◽  
Sequence Generation ◽  
Randomized Controlled ◽  
Random Sequence Generation ◽  
Over Time ◽  
Number Of Authors

AbstractBackgroundMany randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices including reporting guidelines, but it is unknown whether these efforts have improved RCT quality (i.e. reduced risk of bias). We therefore mapped trends over time in trial publication, trial registration, reporting according to CONSORT, and characteristics of publication and authors.MethodsMeta-information of 176,620 RCTs published between 1966 and 2018 was extracted. Risk of bias probability (four domains: random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using validated risk-of-bias machine learning tools. In addition, trial registration and reporting according to CONSORT were assessed with automated searches. Characteristics were extracted related to publication (number of authors, journal impact factor, medical discipline) and authors (gender and Hirsch-index).FindingsThe annual number of published RCTs substantially increased over four decades, accompanied by increases in the number of authors (5.2 to 7.8), institutions (2.9 to 4.8), female authors (20 to 42%, first authorship; 17 to 29%, last authorship), and Hirsch-indices (10 to 14, first authorship; 16 to 28, last authorship). Risk of bias remained present in most RCTs but decreased over time for the domains allocation concealment (63 to 51%), random sequence generation (57 to 36%), and blinding of outcome assessment (58 to 52%). Trial registration (37 to 47%) and CONSORT (1 to 20%) rapidly increased in the latest period. In journals with higher impact factor (>10), risk of bias was consistently lower, higher levels of trial registration more frequent, and mentioning CONSORT.InterpretationThe likelihood of bias in RCTs has generally decreased over the last decades. This may be driven by increased knowledge and improved education, augmented by mandatory trial registration, and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.FundingThis study was funded by The Netherlands Organisation for Health Research and Development (445001002).

scholarly journals Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Ognjen Barcot ◽  
Matija Boric ◽  
Tina Poklepovic Pericic ◽  
Marija Cavar ◽  
Svjetlana Dosenovic ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Random Sequence ◽  
Risk Of Bias ◽  
Sequence Generation ◽  
Random Sequence Generation

scholarly journals Effectiveness, quality and process evaluation of interventions to prevent childhood obesity.

2020 ◽  
Vol 79 (OCE2) ◽  
Author(s):  
Miguel Seral-Cortes ◽  
Pilar De Miguel-Etayo ◽  
Paola Zapata ◽  
Luis Moreno-Aznar
Keyword(s):  
Childhood Obesity ◽  
Process Evaluation ◽  
Systematic Reviews ◽  
Random Sequence ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Global Public Health ◽  
Program Outcomes ◽  
Sequence Generation ◽  
Random Sequence Generation

AbstractChildhood obesity is one of the most serious global public health challenges of the 21stcentury. Health and scientific organizations demand early interventions, although their complexity generates difficulties in their implementation. In systematic reviews, quality assessments are intended to limit mislead reporting and it is important they are conducted in a manner that minimizes bias. Process evaluation (PE) is used to monitor and document program implementation and can aid in understanding the relationship between specific program elements and program outcomes. Failure to deliver an intervention as intended can lead to unclear conclusions about the effectiveness of the intervention. The aim of the present study is to evaluate the effectiveness of the studies based on the process evaluation and the quality on its interventions. We performed a systematic review of randomized control trials aiming to prevent childhood obesity. The Cochrane Handbook for Systematic Reviews of Interventions tool was used to assess and report methodological risk of bias. Each item was judged as being at high (HR), low (LR) or unclear (UN) risk of bias as per criteria. Key domains random sequence generation and allocation concealment were also assessed. Effectiveness was estimated when the study showed changes in body composition as the main outcome (BMI z-score or waist circumference). Process evaluation was evaluated if at least one process evaluation indicator was identified as being adequately implemented. From the 41 studies, 26 showed any degree of effectiveness and 15 were not effective. Almost half of studies scored high risk of bias: 20/41 (Effective 13/26; non effective 7/15). Moreover, 16/41 studies reported to have unclear risk of bias (effective 10/26; and non-effective 6/15). Only 5/41 articles reported to have low risk (effective 3/26; non effective 2/15). Regarding random sequence generation, there was no difference according to the quality of the studies (effective: LR 25/26, UN 1/26 AND HR 0/26; non effective: LR 15/15, UN 0/15 AND HR 0/15). Concerning the allocation concealment domain, there were no differences found either (effective: LR 11/26, UN 10/26 AND HR 5/26; non effective: LR 7/15, UN 5/15 AND HR 3/15). PE was used in 7 papers (effective 3/7 and non-effective 4/7). Quality seems to have slightly more influence in the non-effective studies (LR 13.3%) than in the effective studies (LR 11.5%). The non-effective studies showed the highest proportion of performing PE. There seems to be a relationship between the quality and PE performance.

scholarly journals The Risk of Bias in Iranian Randomized Controlled Trials Included in Cochrane Reviews

2020 ◽  
Author(s):  
Ali Kabir ◽  
Ahmad Sofi-Mahmudi ◽  
Arman Karimi Behnagh ◽  
Vahid Eidkhani ◽  
Hamid Reza Baradaran ◽  
...  
Keyword(s):  
Randomized Controlled Trials ◽  
Systematic Reviews ◽  
Random Sequence ◽  
Outcome Data ◽  
Conclusive Evidence ◽  
Risk Of Bias ◽  
Controlled Clinical Trial ◽  
Controlled Trials ◽  
Randomized Controlled ◽  
Cochrane Reviews

Abstract Background: Among interventional studies, randomized controlled trials (RCTs) provide the most conclusive evidence. However, RCTs can be susceptible to the risk of bias (RoB). Systematic reviews can be performed to appraise the RoB in the included articles using evaluative tools. This study aimed to describe the main characteristics and focus on the ROB of RCTs conducted in Iran and included in Cochrane Reviews (CRs).Methods: We searched "Iran" by selecting the "Search All Text" and "Review" fields in the Cochrane Database of Systematic Reviews within Ovid. CRs that included the RCTs conducted in Iran were retrieved. A trial was selected only if it was included in CRs, described as a controlled clinical trial, involved human subjects and its RoB was assessed by CR authors. The trials were characterized by investigating the relevant articles and the table "Characteristics of included studies" in each CR. The RoB was investigated by collecting the judgments of the review authors made based on tables of RoB assessment in the CRs.Results: Out of 1166 Iranian RCTs included by 571 CRs, a low RoB was found in 44.9% for random sequence generation, 20.8% for allocation concealment, 32.3% for blinding of participants/personnel, 36.5% for blinding of outcome assessors, 56.3% for incomplete outcome data, 41.3% for selective outcome reporting and 53.8% for other sources of bias.Conclusion: The RoB in Iranian RCTs was found to be mostly high or unclear. It is therefore recommended that the methodological quality of RCTs be seriously addressed in Iran.

scholarly journals The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement

PLoS Biology ◽  
2021 ◽  
Vol 19 (4) ◽  
pp. e3001162
Author(s):  
Christiaan H. Vinkers ◽  
Herm J. Lamberink ◽  
Joeri K. Tijdink ◽  
Pauline Heus ◽  
Lex Bouter ◽  
...  
Keyword(s):  
Outcome Assessment ◽  
Random Sequence ◽  
Consort Statement ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Sequence Generation ◽  
Randomized Controlled ◽  
The Consort Statement ◽  
Random Sequence Generation

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.

scholarly journals Randomized Controlled Trials on Complementary and Traditional Medicine in the Korean Literature

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Chang-Kyu Kim ◽  
Da-Hee Kim ◽  
Myeong Soo Lee ◽  
Jong-In Kim ◽  
L. Susan Wieland ◽  
...  
Keyword(s):  
Randomized Controlled Trials ◽  
Traditional Medicine ◽  
Systematic Reviews ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Controlled Trials ◽  
Korean Literature ◽  
Language Bias ◽  
Randomized Controlled ◽  
Study Characteristics

Objective. This study aimed to identify all of the features of complementary and alternative (CAM) randomized controlled trials (RCTs) in the Korean literature and then introduce English-speaking researchers to the bibliometric and risk of bias characteristics of this literature.Methods. Eleven electronic databases and sixteen Korean journals were searched to August 2013 for RCTs of CAM therapies. Key study characteristics were extracted and risk of bias was assessed using the Cochrane Collaboration’s tool for assessing risk of bias.Results. Three hundred and sixty publications met our inclusion criteria. Complementary and traditional medicine RCTs in the Korean literature emerged in the mid-1990s and increased in the mid-2000s. The most common CAM interventions include acupuncture (59.4%) and herbal medicine (8.3%). The largest proportion of trials evaluated CAM for musculoskeletal conditions (20.7%). Adequate methods of randomization were reported in 41.7% of the RCTs, whereas only 8.3% reported adequate allocation concealment. A low proportion of trials reported participant blinding (34.2%) and outcome assessor blinding (22.5%).Conclusions. Korean CAM RCTs are typically omitted from systematic reviews resulting in the potential for language bias. This study will enable these trials of diverse quality to be identified and assessed for inclusion in future systematic reviews on CAM interventions.

scholarly journals Impact of risk of bias on the magnitude of treatment effects of randomized controlled trials in implant dentistry

2020 ◽  
Author(s):  
Ruolin Ding ◽  
Wenxin Lu ◽  
Jianru Yi ◽  
Liang Zhang ◽  
Zhihe Zhao
Keyword(s):  
Randomized Controlled Trials ◽  
Treatment Effects ◽  
Permutation Test ◽  
Random Sequence ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Selective Reporting ◽  
Controlled Trials ◽  
Randomized Controlled ◽  
Implant Dentistry

Abstract Background: Risk of bias (RoB) could influence the magnitude of treatment effects of randomized controlled trials (RCTs). This study aims to investigate the potential influence of RoB on treatment effects estimates in RCTs in implant dentistry. Methods: The RCTs published in five leading oral implant journals during the recent five years were electronically searched. The RoB was assessed using the Cochrane Collaboration RoB tool. The meta-regression analysis and Monte Carlo permutation test were performed to identify the association between RoB and the magnitude of treatment effects.Results: A considerable amount of studies have high RoB in blinding of participants and personnel, and unclear RoB in allocation concealment and selective reporting. The treatment effects were exaggerated by flaws in allocation concealment for binary outcomes and by deficiencies in random sequence generation and selective reporting for continuous outcomes.Conclusion: RoB frequently exists in RCTs recently published in implant dentistry, which may lead to the exaggeration of treatment effects. Better study design, implementation, and reporting are required for clinical trials in implant dentistry to ensure more reliable evidence.

Systematic review protocol examining the effectiveness of non-Invasive Stimulation Techniques to Collect Urine in Newborn and Infants (Preprint)

2020 ◽  
Author(s):  
Ting Wan Tan ◽  
Han Ling Tan ◽  
Chih Ming Chang
Keyword(s):  
Systematic Review ◽  
Urine Sample ◽  
Systematic Reviews ◽  
Meta Analysis ◽  
Risk Of Bias ◽  
Systemic Review ◽  
Controlled Trials ◽  
Non Invasive ◽  
Randomized Controlled ◽  
Stimulation Technique

BACKGROUND Urinary tract infection (UTI) is one of common illness in newborn and infants. An accurate urine sample to diagnosis of UTI is crucial to ensure appropriate treatment to avoid delayed and false-treatment. However, data to assess the effectiveness and safety of non-invasive stimulation technique of collecting urine in newborn and infants are yet to be critically evaluated. OBJECTIVE This protocol aimed to conduct a systematic review and meta-analysis to evaluate the evidences related to the effectiveness and safety of non-invasive stimulation technique of collecting urine in newborn and infants. METHODS This protocol follows guideline according to Preferred Reporting Items for Systematic reviews and Meta-Analysis Protocol and registered on the International Prospective Register of Systematic Reviews. Seven databases will be searched from their inception to March 2020: PubMed, EBSCO, Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE and cumulative index to nursing and allied health literature database. RESULTS This Systemic review will include randomized controlled clinical trials (RCTs) including quasi-randomized controlled trials (quasi-RCTs), observational studies (prospective cohort or retrospective cohort studies). We will perform data extraction, study selection, assessment with risk of bias and data analysis. Primary outcomes measure success in obtaining a urine sample. Secondary outcome measures included gender, age group, weight analysis, contamination rate and adverse effects. Study methodological appraisal of the studies will be assessed by the Cochrane Risk of Bias Tools. For sufficient data are available, a meta-analysis will be conducted. I2 statistics will be used to assess heterogeneity and identify potential sources. CONCLUSIONS This systematic review wills no requirement for ethical approval, without human beings involvement. This systematic review will be published electronically in a peer-reviewed journal, and will give healthcare practitioners good practical guide and information for the evidences related to the effectiveness of non-invasive stimulation techniques to collect urine in newborn and infants, aims to improve clinical outcomes.

scholarly journals Risk of bias assessment of randomised controlled trials referenced in the 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care: a cross-sectional review

BMJ Open ◽  
2019 ◽  
Vol 9 (5) ◽  
pp. e023725 ◽  
Author(s):  
Yongil Cho ◽  
Changsun Kim ◽  
Bossng Kang
Keyword(s):  
Random Sequence ◽  
Heart Association ◽  
Allocation Concealment ◽  
Risk Of Bias ◽  
Cross Sectional ◽  
Sequence Generation ◽  
Random Sequence Generation ◽  
Randomised Controlled ◽  
Cardiovascular Care ◽  
Unclear Risk

ObjectivesTo identify the risk of bias of randomised controlled trials (RCTs) referenced in the 2015 American Heart Association (AHA) guidelines update for cardiopulmonary resuscitation (CPR) and emergency cardiovascular care (ECC).DesignA cross-sectional review.SettingAll RCTs cited as references in the 2015 AHA guidelines update for CPR and ECC were extracted. After excluding non-human trials, studies that analysed existing RCTs, and RCTs published in a letter format, two reviewers assessed the risk of bias among RCTs included in this study.Outcome measuresThe Cochrane Collaboration’s tool for assessing the risk of bias in six domains (random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data and selective reporting) was used.ResultsTwo hundred seventy-three RCTs were selected for the analyses. Of these RCTs, 78.8% had a high risk of bias for blinding of participants and personnel, mostly (87.7%) non-drug trials. In drug trials, the proportion of trials with a low risk of bias for blinding of participants and personnel was 73.0%. The proportion of RCTs with an unclear risk of bias were higher for random sequence generation (38.5%) and allocation concealment (34.1%) than in other domains. Unclear risk of bias proportions was 65.4% for random sequence generation and 57.7% for allocation concealment before the introduction of Consolidated Standards of Reporting Trials (CONSORT) but decreased to 31.3% and 32.2% after the 2010 CONSORT update, respectively.ConclusionsThe proportion of RCTs with an unclear risk of bias was still high for random sequence generation and allocation concealment in the 2015 AHA guidelines for CPR and ECC. The risk of bias should be considered when interpreting and applying the CPR guidelines. Authors should plan and report their research using CONSORT guidelines and the Cochrane Collaboration’s tool to reduce the risk of bias.

scholarly journals Comparison of Occlusal Contact Changes During Retention Between Hawley-Type Retainers and Other Retention Appliances: A Systematic Review

2020 ◽  
Vol 54 (2) ◽  
pp. 92-99
Author(s):  
Priya B. ◽  
Ravindra Kumar Jain ◽  
Balavenkata Bharathi Chaturvedula
Keyword(s):  
Systematic Review ◽  
Orthodontic Treatment ◽  
Data Extraction ◽  
Random Sequence ◽  
Risk Of Bias ◽  
Posterior Teeth ◽  
Occlusal Contact ◽  
Sequence Generation ◽  
Outcome Comparison ◽  
Random Sequence Generation

Background: Achieving adequate and broad occlusal contacts following orthodontic treatment usually is performed during retention phase, and it ensures good intercuspation of posterior teeth and post-treatment stability. Objective: To investigate the changes in occlusal contacts with the use of Hawley-type retainers (Hawley’s and wrap around retainers) and compare them with other retention appliances. Methods: The search included articles that were published until December 2018 in three popular databases. Selection criteria comprised studies evaluating number and area of occlusal contact changes during or at the end of retention phase, following orthodontic treatment. After study retrieval and selection, data extraction and individual study risk of bias assessment was performed using the Cochrane Risk of Bias tool. Results: A total sum of eight studies reporting on outcome comparison between Hawley-type retainers with other retention appliances and untreated controls were selected. In all the eight studies, the risk of bias was unclear since blinding and random sequence generation was not reported. In all the eight reported studies, it was concluded that the number of occlusal contacts improved during retention period with Hawley type retainers, but when comparisons were done in between retainers, only two studies reported that Hawley-type retainers were better. Conclusion: This systematic review concludes that the number and area of occlusal contacts improved during retention with Hawley-type retainers. The overall quality of available literature is poor and unclear to support the conclusion that Hawley-type retainers are better than other existing retainers in improving occlusal contacts.

Sign in / Sign up

Export Citation Format

Share Document