Expected Value of Sample Information to Guide the Design of Group Sequential Clinical Trials

Introduction Adaptive designs allow changes to an ongoing trial based on prespecified early examinations of accrued data. Opportunities are potentially being missed to incorporate health economic considerations into the design of these studies. Methods We describe how to estimate the expected value of sample information for group sequential design adaptive trials. We operationalize this approach in a hypothetical case study using data from a pilot trial. We report the expected value of sample information and expected net benefit of sampling results for 5 design options for the future full-scale trial including the fixed-sample-size design and the group sequential design using either the Pocock stopping rule or the O’Brien-Fleming stopping rule with 2 or 5 analyses. We considered 2 scenarios relating to 1) using the cost-effectiveness model with a traditional approach to the health economic analysis and 2) adjusting the cost-effectiveness analysis to incorporate the bias-adjusted maximum likelihood estimates of trial outcomes to account for the bias that can be generated in adaptive trials. Results The case study demonstrated that the methods developed could be successfully applied in practice. The results showed that the O’Brien-Fleming stopping rule with 2 analyses was the most efficient design with the highest expected net benefit of sampling in the case study. Conclusions Cost-effectiveness considerations are unavoidable in budget-constrained, publicly funded health care systems, and adaptive designs can provide an alternative to costly fixed-sample-size designs. We recommend that when planning a clinical trial, expected value of sample information methods be used to compare possible adaptive and nonadaptive trial designs, with appropriate adjustment, to help justify the choice of design characteristics and ensure the cost-effective use of research funding. Highlights Opportunities are potentially being missed to incorporate health economic considerations into the design of adaptive clinical trials. Existing expected value of sample information analysis methods can be extended to compare possible group sequential and nonadaptive trial designs when planning a clinical trial. We recommend that adjusted analyses be presented to control for the potential impact of the adaptive designs and to maintain the accuracy of the calculations. This approach can help to justify the choice of design characteristics and ensure the cost-effective use of limited research funding.

Advantages of multi-arm non-randomised sequentially allocated cohort designs for Phase II oncology trials

Background Efficient trial designs are required to prioritise promising drugs within Phase II trials. Adaptive designs are examples of such designs, but their efficiency is reduced if there is a delay in assessing patient responses to treatment. Methods Motivated by the WIRE trial in renal cell carcinoma (NCT03741426), we compare three trial approaches to testing multiple treatment arms: (1) single-arm trials in sequence with interim analyses; (2) a parallel multi-arm multi-stage trial and (3) the design used in WIRE, which we call the Multi-Arm Sequential Trial with Efficient Recruitment (MASTER) design. The MASTER design recruits patients to one arm at a time, pausing recruitment to an arm when it has recruited the required number for an interim analysis. We conduct a simulation study to compare how long the three different trial designs take to evaluate a number of new treatment arms. Results The parallel multi-arm multi-stage and the MASTER design are much more efficient than separate trials. The MASTER design provides extra efficiency when there is endpoint delay, or recruitment is very quick. Conclusions We recommend the MASTER design as an efficient way of testing multiple promising cancer treatments in non-comparative Phase II trials.

Costs and staffing resource requirements for adaptive clinical trials: quantitative and qualitative results from the Costing Adaptive Trials project

Background Adaptive designs offer great promise in improving the efficiency and patient-benefit of clinical trials. An important barrier to further increased use is a lack of understanding about which additional resources are required to conduct a high-quality adaptive clinical trial, compared to a traditional fixed design. The Costing Adaptive Trials (CAT) project investigated which additional resources may be required to support adaptive trials. Methods We conducted a mock costing exercise amongst seven Clinical Trials Units (CTUs) in the UK. Five scenarios were developed, derived from funded clinical trials, where a non-adaptive version and an adaptive version were described. Each scenario represented a different type of adaptive design. CTU staff were asked to provide the costs and staff time they estimated would be needed to support the trial, categorised into specified areas (e.g. statistics, data management, trial management). This was calculated separately for the non-adaptive and adaptive version of the trial, allowing paired comparisons. Interviews with 10 CTU staff who had completed the costing exercise were conducted by qualitative researchers to explore reasons for similarities and differences. Results Estimated resources associated with conducting an adaptive trial were always (moderately) higher than for the non-adaptive equivalent. The median increase was between 2 and 4% for all scenarios, except for sample size re-estimation which was 26.5% (as the adaptive design could lead to a lengthened study period). The highest increase was for statistical staff, with lower increases for data management and trial management staff. The percentage increase in resources varied across different CTUs. The interviews identified possible explanations for differences, including (1) experience in adaptive trials, (2) the complexity of the non-adaptive and adaptive design, and (3) the extent of non-trial specific core infrastructure funding the CTU had. Conclusions This work sheds light on additional resources required to adequately support a high-quality adaptive trial. The percentage increase in costs for supporting an adaptive trial was generally modest and should not be a barrier to adaptive designs being cost-effective to use in practice. Informed by the results of this research, guidance for investigators and funders will be developed on appropriately resourcing adaptive trials.

Dynamic Programming for Response-Adaptive Dose-Finding Clinical Trials

Identifying the right dose is one of the most important decisions in drug development. Adaptive designs are promoted to conduct dose-finding clinical trials as they are more efficient and ethical compared with static designs. However, current techniques in response-adaptive designs for dose allocation are complex and need significant computational effort, which is a major impediment for implementation in practice. This study proposes a Bayesian nonparametric framework for estimating the dose-response curve, which uses a piecewise linear approximation to the curve by consecutively connecting the expected mean response at each dose. Our extensive numerical results reveal that a first-order Bayesian nonparametric model with a known correlation structure in prior for the expected mean response performs competitively when compared with the standard approach and other more complex models in terms of several relevant metrics and enjoys computational efficiency. Furthermore, structural properties for the optimal learning problem, which seeks to minimize the variance of the target dose, are established under this simple model. Summary of Contribution: In this work, we propose a methodology to derive efficient patient allocation rules in response-adaptive dose-finding clinical trials, where computational issues are the main concern. We show that our methodologies are competitive with the state-of-the-art methodology in terms of solution quality, are significantly more computationally efficient, and are more robust in terms of the shape of the dose-response curve, among other parameter changes. This research fits in “the intersection of computing and operations research” as it adapts operations research techniques to produce computationally attractive solutions to patient allocation problems in dose-finding clinical trials.

Adaptive group sequential survival comparisons based on log-rank and pointwise test statistics

Whereas the theory of confirmatory adaptive designs is well understood for uncensored data, implementation of adaptive designs in the context of survival trials remains challenging. Commonly used adaptive survival tests are based on the independent increments structure of the log-rank statistic. This implies some relevant limitations: On the one hand, essentially only the interim log-rank statistic may be used for design modifications (such as data-dependent sample size recalculation). Furthermore, the treatment arm allocation ratio in these classical methods is assumed to be constant throughout the trial period. Here, we propose an extension of the independent increments approach to adaptive survival tests that addresses some of these limitations. We present a confirmatory adaptive two-sample log-rank test that allows rejection regions and sample size recalculation rules to be based not only on the interim log-rank statistic, but also on point-wise survival rate estimates, simultaneously. In addition, the possibility is opened to adapt the treatment arm allocation ratio after each interim analysis in a data-dependent way. The ability to include point-wise survival rate estimators in the rejection region of a test for comparing survival curves might be attractive, e.g., for seamless phase II/III designs. Data-dependent adaptation of the allocation ratio could be helpful in multi-arm trials in order to successively steer recruitment into the study arms with the greatest chances of success. The methodology is motivated by the LOGGIC Europe Trial from pediatric oncology. Distributional properties are derived using martingale techniques in the large sample limit. Small sample properties are studied by simulation.

Adaptive treatment allocation and selection in multi-arm clinical trials: a Bayesian perspective

Background: Adaptive designs offer added flexibility in the execution of clinical trials, including the possibilities of allocating more patients to the treatments that turned out more successful, and early stopping due to either declared success or futility. Commonly applied adaptive designs, such as group sequential methods, are based on the frequentist paradigm and on ideas from statistical significance testing. Interim checks during the trial will have the effect of inflating the Type 1 error rate, or, if this rate is controlled and kept fixed, lowering the power. Results: The purpose of the paper is to demonstrate the usefulness of the Bayesian approach in the design and in the actual running of randomized clinical trials during Phase II and III. This approach is based on comparing the performance of the different treatment arm in terms of the respective joint posterior probabilities evaluated sequentially from the accruing outcome data, and then taking a control action if such posterior probabilities fall below a pre-specified critical threshold value. Two types of actions are considered: treatment allocation, putting on hold at least temporarily further accrual of patients to a treatment arm (Rule 1), and treatment selection, removing an arm from the trial permanently (Rule 2). The main development in the paper is in terms of binary outcomes, but extensions for handling time-to-event data, including data from vaccine trials, are also discussed. The performance of the proposed methodology is tested in extensive simulation experiments, with numerical results and graphical illustrations documented in a Supplement to the main text. As a companion to this paper, an implementation of the methods is provided in the form of a freely available R package. Conclusion: The proposed methods for trial design provide an attractive alternative to their frequentist counterparts.

Clinical research methodology process: what is changing with COVID-19?

The repercussions of the pandemic in progress on clinical research have been the systematic interruption of ongoing research and the explosion of fragmented, uncoordinated, often technically insufficient anti-COVID-19 research. Networks of expert centres have emerged setting up well-structured research, adopting much more efficient and aggressive designs than traditional ones. Adaptive designs, characterized by flexibility and mouldability even in the course of studies, which is essential in an epidemic with thousands of simultaneous studies aimed at the same objectives. Some studies are structured with networks of hospitals around guidance centres, such as RECOVERY (Oxford University, UK) and SOLIDARITY (WHO, 30 countries); others with networks of expert centres mostly organized in a combined model: some expert centres test new molecules in Phase 2 in a limited number of patients, and orient promising ones towards connected networks for Phase 3. Cortisones and tentatively cytokines are acquired in the official recommendation. Another emerging model is the pragmatic trial, also called, more expressively, ‘remote’ or ‘virtual’. So it is in fact: the web replaces the direct link between patients and doctors/research operators (CROs included), behind which there will be omnipresent big-techs.