AbstractBackgroundPurpose of this study was to analyze adequacy of judgments about risk of bias (RoB) for random sequence generation in Cochrane systematic reviews (CSRs) of randomized controlled trials (RCTs).MethodsInformation was extracted from RoB tables of CSRs using automated data scraping. We categorized all comments provided as supports for judgments for RoB related to randomization. We analyzed number and type of various supporting comments and assessed adequacy of RoB judgment for randomization in line with recommendations from the Cochrane Handbook.ResultsWe analyzed 10527 RCTs that were included in 729 CSRs. For 5682 RCTs randomization was not described; for the others it was indicated randomization was done using computer/software/internet (N=2886), random number table (N=888), mechanic method (N=366), or it was incomplete/inappropriate (N=303).Overall, 1194/10125 trials (12%) had erroneous RoB judgment about randomization. The highest proportion of errors was found for trials with high RoB (28%), followed by those with low (19%), or unclear (3%). Therefore, one in eight judgments for the analyzed domain in CSRs was erroneous, and one in three if the judgment was “high risk”.ConclusionCochrane systematic reviews cannot be necessarily trusted when it comes to judgments for risk of bias related to randomized sequence generation.
Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook
