Clarifying selection bias in cluster randomized trials

Background In cluster randomized trials, patients are typically recruited after clusters are randomized, and the recruiters and patients may not be blinded to the assignment. This often leads to differential recruitment and consequently systematic differences in baseline characteristics of the recruited patients between intervention and control arms, inducing post-randomization selection bias. We aim to rigorously define causal estimands in the presence of selection bias. We elucidate the conditions under which standard covariate adjustment methods can validly estimate these estimands. We further discuss the additional data and assumptions necessary for estimating causal effects when such conditions are not met. Methods Adopting the principal stratification framework in causal inference, we clarify there are two average treatment effect (ATE) estimands in cluster randomized trials: one for the overall population and one for the recruited population. We derive analytical formula of the two estimands in terms of principal-stratum-specific causal effects. Furthermore, using simulation studies, we assess the empirical performance of the multivariable regression adjustment method under different data generating processes leading to selection bias. Results When treatment effects are heterogeneous across principal strata, the average treatment effect on the overall population generally differs from the average treatment effect on the recruited population. A naïve intention-to-treat analysis of the recruited sample leads to biased estimates of both average treatment effects. In the presence of post-randomization selection and without additional data on the non-recruited subjects, the average treatment effect on the recruited population is estimable only when the treatment effects are homogeneous between principal strata, and the average treatment effect on the overall population is generally not estimable. The extent to which covariate adjustment can remove selection bias depends on the degree of effect heterogeneity across principal strata. Conclusion There is a need and opportunity to improve the analysis of cluster randomized trials that are subject to post-randomization selection bias. For studies prone to selection bias, it is important to explicitly specify the target population that the causal estimands are defined on and adopt design and estimation strategies accordingly. To draw valid inferences about treatment effects, investigators should (1) assess the possibility of heterogeneous treatment effects, and (2) consider collecting data on covariates that are predictive of the recruitment process, and on the non-recruited population from external sources such as electronic health records.

Methodological issues in the design and analysis of cluster randomized trials

Randomized controlled trials are the accepted gold standard for evaluating the effects of interventions to improve health. In the majority of such trials, individuals are randomly allocated to the experimental conditions under study, for example, to treatment and control arms. However, in some situations it is more appropriate to randomly allocate groups of individuals to the treatment arms. These groups are referred to as clusters, and trials of this kind are known as cluster randomized trials (CRTs). Examples of clusters include schools, villages, workplaces, or health facilities, but there are many other possible choices. In some CRTs, all individuals within the selected clusters are automatically included. In others, there may be additional eligibility criteria. Similarly, the impact of the intervention may be measured in all individuals in the cluster, or in a random subsample. This chapter aims to discuss methodological issues that arise in the design and analysis of CRTs

Machine learning algorithms to identify cluster randomized trials from MEDLINE and EMBASE

Background: Cluster randomized trials (CRTs) are becoming an increasingly important design. However, authors do not always adhere to requirements to explicitly identify the design as cluster randomized in titles and abstracts, making retrieval from bibliographic databases difficult. Machine learning algorithms may improve their identification and retrieval. Therefore, we aimed to develop machine learning algorithms that accurately determine whether a bibliographic citation is a CRT report. Methods: We trained, internally validated, and externally validated two convolutional neural networks and one support vector machines (SVM) algorithms to predict whether a citation is a CRT report or not. We exclusively used the information in an article citation, including the title, abstract, keywords, and subject headings. The algorithms' output was a probability from 0 to 1. We assessed algorithm performance using the area under the receiver operating characteristic (AUC) curves. Each algorithm's performance was evaluated individually and together as an ensemble. We randomly selected 5000 from 87,633 citations to train and internally validate our algorithms. Of the 5000 selected citations, 589 (12%) were confirmed CRT reports. We then externally validated our algorithms on an independent set of 1916 randomized trial citations, with 665 (35%) confirmed CRT reports. Results: In internal validation, the ensemble algorithm discriminated best for identifying CRT reports with an AUC of 98.6% (95% confidence interval: 97.8%, 99.4%), sensitivity of 97.7% (94.3%, 100%), and specificity of 85.0% (81.8%, 88.1%). In external validation, the ensemble algorithm had an AUC of 97.8 % (97.0%, 98.5%), sensitivity of 97.6% (96.4%, 98.6%), and specificity of 78.2% (75.9%, 80.4%)). All three individual algorithms performed well, but less so than the ensemble. Conclusions: We successfully developed high-performance algorithms that identified whether a citation was a CRT report with high sensitivity and moderately high specificity. We provide open-source software to facilitate the use of our algorithms in practice.

Estimating intervention effectiveness in trials of malaria interventions with contamination

Background In cluster randomized trials (CRTs) or stepped wedge cluster randomized trials (SWCRTs) of malaria interventions, mosquito movement leads to contamination between trial arms unless buffer zones separate the clusters. Contamination can be accounted for in the analysis, yielding an estimate of the contamination range, the distance over which contamination measurably biases the effectiveness. Methods A previously described analysis for CRTs is extended to SWCRTs and estimates of effectiveness are provided as a function of intervention coverage. The methods are applied to two SWCRTs of malaria interventions, the SolarMal trial on the impact of mass trapping of mosquitoes with odor-baited traps and the AvecNet trial on the effect of adding pyriproxyfen to long-lasting insecticidal nets. Results For the SolarMal trial, the contamination range was estimated to be 146 m ($$95\%$$ 95 % credible interval $$[0.052,\,0.923]$$ [ 0.052 , 0.923 ] km), together with a $$31.9\%$$ 31.9 % ($$95\%$$ 95 % credible interval $$[15.3,\,45.8]\%$$ [ 15.3 , 45.8 ] % ) reduction of Plasmodium infection, compared to the $$30.0\%$$ 30.0 % reduction estimated without accounting for contamination. The estimated effectiveness had an approximately linear relationship with coverage. For the AvecNet trial, estimated contamination effects were minimal, with insufficient data from the cluster boundary regions to estimate the effectiveness as a function of coverage. Conclusions The contamination range in these trials of malaria interventions is much less than the distances Anopheles mosquitoes can fly. An appropriate analysis makes buffer zones unnecessary, enabling the design of more cost-efficient trials. Estimation of the contamination range requires information from the cluster boundary regions and trials should be designed to collect this.