Adjusting for treatment switching in randomised controlled trials – A simulation study and a simplified two-stage method

2014 ◽  
Vol 26 (2) ◽  
pp. 724-751 ◽  
Author(s):  
Nicholas R Latimer ◽  
KR Abrams ◽  
PC Lambert ◽  
MJ Crowther ◽  
AJ Wailoo ◽  
...  
Keyword(s):  
Treatment Effect ◽  
Survival Benefit ◽  
Randomised Controlled Trials ◽  
Health Technology ◽  
Control Group ◽  
Controlled Trials ◽  
Inverse Probability ◽  
Treatment Switching ◽  
Overall Survival Benefit ◽  
Randomised Controlled

Estimates of the overall survival benefit of new cancer treatments are often confounded by treatment switching in randomised controlled trials (RCTs) – whereby patients randomised to the control group are permitted to switch onto the experimental treatment upon disease progression. In health technology assessment, estimates of the unconfounded overall survival benefit associated with the new treatment are needed. Several switching adjustment methods have been advocated in the literature, some of which have been used in health technology assessment. However, it is unclear which methods are likely to produce least bias in realistic RCT-based scenarios. We simulated RCTs in which switching, associated with patient prognosis, was permitted. Treatment effect size and time dependency, switching proportions and disease severity were varied across scenarios. We assessed the performance of alternative adjustment methods based upon bias, coverage and mean squared error, related to the estimation of true restricted mean survival in the absence of switching in the control group. We found that when the treatment effect was not time-dependent, rank preserving structural failure time models (RPSFTM) and iterative parameter estimation methods produced low levels of bias. However, in the presence of a time-dependent treatment effect, these methods produced higher levels of bias, similar to those produced by an inverse probability of censoring weights method. The inverse probability of censoring weights and structural nested models produced high levels of bias when switching proportions exceeded 85%. A simplified two-stage Weibull method produced low bias across all scenarios and provided the treatment switching mechanism is suitable, represents an appropriate adjustment method.

scholarly journals Assessing methods for dealing with treatment switching in clinical trials: A follow-up simulation study

2016 ◽  
Vol 27 (3) ◽  
pp. 765-784 ◽  
Author(s):  
Nicholas R Latimer ◽  
Keith R Abrams ◽  
Paul C Lambert ◽  
James P Morden ◽  
Michael J Crowther
Keyword(s):  
Sample Size ◽  
Treatment Effect ◽  
Failure Time ◽  
Control Group ◽  
Structural Failure ◽  
Inverse Probability ◽  
Intention To Treat ◽  
Treatment Switching ◽  
Randomised Controlled ◽  
The Impact

When patients randomised to the control group of a randomised controlled trial are allowed to switch onto the experimental treatment, intention-to-treat analyses of the treatment effect are confounded because the separation of randomised groups is lost. Previous research has investigated statistical methods that aim to estimate the treatment effect that would have been observed had this treatment switching not occurred and has demonstrated their performance in a limited set of scenarios. Here, we investigate these methods in a new range of realistic scenarios, allowing conclusions to be made based upon a broader evidence base. We simulated randomised controlled trials incorporating prognosis-related treatment switching and investigated the impact of sample size, reduced switching proportions, disease severity, and alternative data-generating models on the performance of adjustment methods, assessed through a comparison of bias, mean squared error, and coverage, related to the estimation of true restricted mean survival in the absence of switching in the control group. Rank preserving structural failure time models, inverse probability of censoring weights, and two-stage methods consistently produced less bias than the intention-to-treat analysis. The switching proportion was confirmed to be a key determinant of bias: sample size and censoring proportion were relatively less important. It is critical to determine the size of the treatment effect in terms of an acceleration factor (rather than a hazard ratio) to provide information on the likely bias associated with rank-preserving structural failure time model adjustments. In general, inverse probability of censoring weight methods are more volatile than other adjustment methods.

scholarly journals Incorporating single-arm studies in meta-analysis of randomised controlled trials: a simulation study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Janharpreet Singh ◽  
Keith R. Abrams ◽  
Sylwia Bujkiewicz
Keyword(s):  
Simulation Study ◽  
Randomised Controlled Trials ◽  
Meta Analysis ◽  
Health Technology ◽  
Small Error ◽  
Aggregate Data ◽  
Controlled Trials ◽  
Real World Data ◽  
Study Heterogeneity ◽  
Randomised Controlled

Abstract Background Use of real world data (RWD) from non-randomised studies (e.g. single-arm studies) is increasingly being explored to overcome issues associated with data from randomised controlled trials (RCTs). We aimed to compare methods for pairwise meta-analysis of RCTs and single-arm studies using aggregate data, via a simulation study and application to an illustrative example. Methods We considered contrast-based methods proposed by Begg & Pilote (1991) and arm-based methods by Zhang et al (2019). We performed a simulation study with scenarios varying (i) the proportion of RCTs and single-arm studies in the synthesis (ii) the magnitude of bias, and (iii) between-study heterogeneity. We also applied methods to data from a published health technology assessment (HTA), including three RCTs and 11 single-arm studies. Results Our simulation study showed that the hierarchical power and commensurate prior methods by Zhang et al provided a consistent reduction in uncertainty, whilst maintaining over-coverage and small error in scenarios where there was limited RCT data, bias and differences in between-study heterogeneity between the two sets of data. The contrast-based methods provided a reduction in uncertainty, but performed worse in terms of coverage and error, unless there was no marked difference in heterogeneity between the two sets of data. Conclusions The hierarchical power and commensurate prior methods provide the most robust approach to synthesising aggregate data from RCTs and single-arm studies, balancing the need to account for bias and differences in between-study heterogeneity, whilst reducing uncertainty in estimates. This work was restricted to considering a pairwise meta-analysis using aggregate data.

scholarly journals Efficacy of PD-1 or PD-L1 inhibitors and PD-L1 expression status in cancer: meta-analysis

BMJ ◽  
2018 ◽  
pp. k3529 ◽  
Author(s):  
Xian Shen ◽  
Bin Zhao
Keyword(s):  
Cell Death ◽  
Programmed Cell Death ◽  
Randomised Controlled Trials ◽  
Meta Analysis ◽  
Control Group ◽  
Controlled Trials ◽  
Clinical Benefits ◽  
American Society ◽  
Randomised Controlled ◽  
Expression Status

Abstract Objective To evaluate the relative efficacy of programmed cell death 1 (PD-1) or programmed cell death ligand 1 (PD-L1) inhibitors versus conventional drugs in patients with cancer that were PD-L1 positive and PD-L1 negative. Design Meta-analysis of randomised controlled trials. Data sources PubMed, Embase, Cochrane database, and conference abstracts presented at the American Society of Clinical Oncology and European Society of Medical Oncology up to March 2018. Review methods Studies of PD-1 or PD-L1 inhibitors (avelumab, atezolizumab, durvalumab, nivolumab, and pembrolizumab) that had available hazard ratios for death based on PD-L1 positivity or negativity were included. The threshold for PD-L1 positivity or negativity was that PD-L1 stained cell accounted for 1% of tumour cells, or tumour and immune cells, assayed by immunohistochemistry staining methods. Results 4174 patients with advanced or metastatic cancers from eight randomised controlled trials were included in this study. Compared with conventional agents, PD-1 or PD-L1 inhibitors were associated with significantly prolonged overall survival in both patients that were PD-L1 positive (n=2254, hazard ratio 0.66, 95% confidence interval 0.59 to 0.74) and PD-L1 negative (1920, 0.80, 0.71 to 0.90). However, the efficacies of PD-1 or PD-L1 blockade treatment in patients that were PD-L1 positive and PD-L1 negative were significantly different (P=0.02 for interaction). Additionally, in both patients that were PD-L1 positive and PD-L1 negative, the long term clinical benefits from PD-1 or PD-L1 blockade were observed consistently across interventional agent, cancer histotype, method of randomisation stratification, type of immunohistochemical scoring system, drug target, type of control group, and median follow-up time. Conclusions PD-1 or PD-L1 blockade therapy is a preferable treatment option over conventional therapy for both patients that are PD-L1 positive and PD-L1 negative. This finding suggests that PD-L1 expression status alone is insufficient in determining which patients should be offered PD-1 or PD-L1 blockade therapy.

scholarly journals ‘Not clinically effective but cost-effective’ - paradoxical conclusions in randomised controlled trials with ‘doubly null’ results: a cross-sectional study

BMJ Open ◽  
2020 ◽  
Vol 10 (1) ◽  
pp. e029596 ◽  
Author(s):  
James Raftery ◽  
HC Williams ◽  
Aileen Clarke ◽  
Jim Thornton ◽  
John Norrie ◽  
...  
Keyword(s):  
Cost Effectiveness ◽  
Technology Assessment ◽  
Randomised Controlled Trials ◽  
Cost Effective ◽  
Health Technology ◽  
Controlled Trials ◽  
Cross Sectional ◽  
Null Results ◽  
Health Technology Assessment Programme ◽  
Randomised Controlled

ObjectivesRandomised controlled trials in healthcare increasingly include economic evaluations. Some show small differences which are not statistically significant. Yet these sometimes come to paradoxical conclusions such as: ‘the intervention is not clinically effective’ but ‘is probably cost-effective’. This study aims to quantify the extent of non-significant results and the types of conclusions drawn from them.DesignCross-sectional retrospective analysis of randomised trials published by the UK’s National Institute for Health Research (NIHR) Health Technology Assessment programme. We defined as ‘doubly null’ those trials that found non-statistically significant differences in both primary outcome and cost per patient. Paradoxical was defined as concluding in favour of an intervention, usually compared with placebo or usual care. No human participants were involved. Our sample was 226 randomised trial projects published by the Health Technology Assessment programme 2004 to 2017. All are available free online.ResultsThe 226 projects contained 193 trials with a full economic evaluation. Of these 76 (39%) had at least one ‘doubly null’ comparison. These 76 trials contained 94 comparisons. In these 30 (32%) drew economic conclusions in favour of an intervention. Overall report conclusions split roughly equally between those favouring the intervention (14), and those favouring either the control (7) or uncertainty (9).DiscussionTrials with ‘doubly null’ results and paradoxical conclusions are not uncommon. The differences observed in cost and quality-adjustedlife year were small and non-statistically significant. Almost all these trials were also published in leading peer-reviewed journals. Although some guidelines for reporting economic results require cost-effectiveness estimates regardless of statistical significance, the interpretability of paradoxical results has nowhere been addressed.ConclusionsReconsideration is required of the interpretation of cost-effectiveness analyses in randomised controlled trials with ‘doubly null’ results, particularly when economics favours a novel intervention.

scholarly journals Adverse events of exercise therapy in randomised controlled trials: a systematic review and meta-analysis

2019 ◽  
Vol 54 (18) ◽  
pp. 1073-1080 ◽  
Author(s):  
Andre Niemeijer ◽  
Hans Lund ◽  
Signe Nilssen Stafne ◽  
Thomas Ipsen ◽  
Cathrine Luhaäär Goldschmidt ◽  
...  
Keyword(s):  
Systematic Review ◽  
Relative Risk ◽  
Adverse Events ◽  
Exercise Therapy ◽  
Randomised Controlled Trials ◽  
Meta Analysis ◽  
Control Group ◽  
Controlled Trials ◽  
Serious Adverse Events ◽  
Randomised Controlled

ObjectiveTo evaluate the relative risk (RR) of serious and non-serious adverse events in patients treated with exercise therapy compared with those in a non-exercising control group.DesignSystematic review and meta-analysis.Data sourcesPrimary studies were identified based on The Cochrane Database of Systematic Reviews investigating the effect of exercise therapy.Eligibility criteriaAt least two of the authors independently evaluated all identified reviews and primary studies. Randomised controlled trials were included if they compared any exercise therapy intervention with a non-exercising control. Two authors independently extracted data. The RR of serious and non-serious adverse events was estimated separately.Results180 Cochrane reviews were included and from these, 773 primary studies were identified. Of these, 378 studies (n=38 368 participants) reported serious adverse events and 375 studies (n=38 517 participants) reported non-serious adverse events. We found no increase in risk of serious adverse events (RR=0.96 (95%CI 0.90 to 1.02, I2: 0.0%) due to exercise therapy. There was, however, an increase in non-serious adverse events (RR=1.19 (95%CI 1.09 to 1.30, I2: 0.0%). The number needed to treat for an additional harmful outcome for non-serious adverse events was 6 [95%CI 4 to 11).ConclusionParticipating in an exercise intervention increased the relative risk of non-serious adverse events, but not of serious adverse events. Exercise therapy may therefore be recommended as a relatively safe intervention.PROSPERO registration numberCRD42014014819.

Effects of jump training on jumping performance of handball players: A systematic review with meta-analysis of randomised controlled trials

2020 ◽  
Vol 15 (4) ◽  
pp. 584-594 ◽  
Author(s):  
Rodrigo Ramirez-Campillo ◽  
Cristian Alvarez ◽  
Antonio Garcia-Hermoso ◽  
Justin WL Keogh ◽  
Felipe García-Pinillos ◽  
...  
Keyword(s):  
Methodological Quality ◽  
Randomised Controlled Trials ◽  
Meta Analysis ◽  
Control Group ◽  
Controlled Trials ◽  
Countermovement Jump ◽  
Jumping Performance ◽  
Jump Training ◽  
Jump Exercise ◽  
Randomised Controlled

This study aimed to assess the effects of jump training on the jumping performance of handball players compared with a control condition. The data sources utilised were PubMed, MEDLINE, Web of Science Core Collection and SCOPUS. Only peer-review randomised controlled trials were included. The inclusion criteria comprised: a jump training programme of at least 2 weeks; a control group; the assessment of the countermovement jump. The Physiotherapy Evidence Database scale was used to assess the risk of bias and methodological quality of eligible studies included in the meta-analysis. Risk of publication bias across studies was assessed using the extended Egger’s test. Cohen’s d effect sizes (ESs) were calculated from the countermovement jump and presented together with 95% confidence intervals (CIs). From 6108 records initially identified through database searching, 5 were eligible for meta-analysis. A significant improvement in countermovement jump height was observed, corresponding to 6.4 cm (95% CI = 4.9–7.9; Z = 8.4, p < 0.001), showing moderate heterogeneity ( I2 = 51.4%). The magnitude of the main effect was very large (ES = 2.2 (95% CI = 0.95–3.4), Z = 3.5, p < 0.001). Jump training is effective in increasing vertical jump performance in handball players. However, the insufficient number of studies conducted precluded analyses of moderator variables. In future, researchers are advised to conduct jump training studies of high methodological quality (e.g. randomised controlled trials) and assess different jump exercise prescriptions across handball players of different sexes, ages and competitive levels to analyse if exercise prescription and player characteristics may influence training responses.

Treating depression with physical activity in adolescents and young adults: a systematic review and meta-analysis of randomised controlled trials

2017 ◽  
Vol 48 (7) ◽  
pp. 1068-1083 ◽  
Author(s):  
A. P. Bailey ◽  
S. E. Hetrick ◽  
S. Rosenbaum ◽  
R. Purcell ◽  
A. G. Parker
Keyword(s):  
Physical Activity ◽  
Young Adults ◽  
Treatment Effect ◽  
Randomised Controlled Trials ◽  
Meta Analysis ◽  
Risk Of Bias ◽  
Adolescents And Young Adults ◽  
Controlled Trials ◽  
Randomised Controlled

AbstractWe aimed to establish the treatment effect of physical activity for depression in young people through meta-analysis. Four databases were searched to September 2016 for randomised controlled trials of physical activity interventions for adolescents and young adults, 12–25 years, experiencing a diagnosis or threshold symptoms of depression. Random-effects meta-analysis was used to estimate the standardised mean difference (SMD) between physical activity and control conditions. Subgroup analysis and meta-regression investigated potential treatment effect modifiers. Acceptability was estimated using dropout. Trials were assessed against risk of bias domains and overall quality of evidence was assessed using GRADE criteria. Seventeen trials were eligible and 16 provided data from 771 participants showing a large effect of physical activity on depression symptoms compared to controls (SMD = −0.82, 95% CI = −1.02 to −0.61, p < 0.05, I2 = 38%). The effect remained robust in trials with clinical samples (k = 5, SMD = −0.72, 95% CI = −1.15 to −0.30), and in trials using attention/activity placebo controls (k = 7, SMD = −0.82, 95% CI = −1.05 to −0.59). Dropout was 11% across physical activity arms and equivalent in controls (k = 12, RD = −0.01, 95% CI = −0.04 to 0.03, p = 0.70). However, the quality of RCT-level evidence contributing to the primary analysis was downgraded two levels to LOW (trial-level risk of bias, suspected publication bias), suggesting uncertainty in the size of effect and caution in its interpretation. While physical activity appears to be a promising and acceptable intervention for adolescents and young adults experiencing depression, robust clinical effectiveness trials that minimise risk of bias are required to increase confidence in the current finding. The specific intervention characteristics required to improve depression remain unclear, however best candidates given current evidence may include, but are not limited to, supervised, aerobic-based activity of moderate-to-vigorous intensity, engaged in multiple times per week over eight or more weeks. Further research is needed. (Registration: PROSPERO-CRD 42015024388).

Who Benefits from Electroconvulsive Therapy?

1992 ◽  
Vol 160 (3) ◽  
pp. 355-359 ◽  
Author(s):  
Heather Buchan ◽  
Eve Johnstone ◽  
K. McPherson ◽  
R. L. Palmer ◽  
T. J. Crow ◽  
...  
Keyword(s):  
Electroconvulsive Therapy ◽  
Treatment Effect ◽  
Randomised Controlled Trials ◽  
Controlled Trials ◽  
Combining Data ◽  
Randomised Controlled ◽  
Improved Outcome

This paper describes the results obtained by combining data from the Northwick Park and Leicester randomised controlled trials of ECT. Patients who suffered from depression in which retardation and delusions were features and who received real ECT had a significantly improved outcome at the end of four weeks of treatment (as measured by improvement in the HRSD) in comparison with those who received simulated ECT. However, this treatment effect was not detectable at six-month follow-up. Patients who were neither retarded nor deluded did not benefit significantly from real as opposed to simulated ECT.

Sign in / Sign up

Export Citation Format

Share Document