Introduction & Objective: Unadjusted analyses, fully adjusted analyses, or adjusted analyses based on tests of significance on covariate imbalance are recommended for covariate adjustment in randomized controlled trials. It has been indicated that the tests of significance on baseline comparability is inappropriate, rather it is important to indicate the strength of relationship with outcomes. Our goal is to understand when the adjustment should be used in randomized controlled trials.
Methods: Unadjusted analysis, fully adjusted analysis, and adjusted analysis based on baseline comparability were examined under null and alternative hypothesis by simulation studies. Each data set was simulated 3000 times for a total of 9 scenarios for sample sizes of 20, 40, and 100 each with baseline thresholds of 0.05, 0.1, and 0.2. Each scenario was examined by the change in magnitude of correlation from 0.1 to 0.9.
Results: Power of fully adjusted analysis under alternative hypothesis was increased as the correlation increased while adjusted analysis based on the covariate imbalance did not compare favorably to the unadjusted analysis. Type 1 error was decreased in adjusted analysis based on the covariate imbalance under null hypothesis. It was then observed that p-value does not follow a uniform distribution under the null hypothesis.
Conclusion: Unadjusted and fully adjusted analyses were valid analyses. Full adjustment could potentially increase the power if adjustment is known. However, adjusted analysis based on the test of significance on covariate imbalance may not be a valid analysis. Tests of significance should not be used for comparing baseline comparability.
Covariate adjustment had similar benefits in small and large randomized controlled trials
