Estimating treatment effect for individuals with progressive multiple sclerosis using deep learning

Modeling treatment effect could identify a subgroup of individuals who experience greater benefit from disease modifying therapy, allowing for predictive enrichment to increase the power of future clinical trials. We use deep learning to estimate the conditional average treatment effect for individuals taking disease modifying therapies for multiple sclerosis, using their baseline clinical and imaging characteristics. Data were obtained as part of three placebo-controlled randomized clinical trials: ORATORIO, OLYMPUS and ARPEGGIO, investigating the efficacy of ocrelizumab, rituximab and laquinimod, respectively. A shuffled mix of participants having received ocrelizumab or rituximab, anti-CD20-antibodies, was separated into a training (70%) and testing (30%) dataset, but we also performed nested cross-validation to improve the generalization error estimate. Data from ARPEGGIO served as additional external validation. An ensemble of multitask multilayer perceptrons was trained to predict the rate of disability progression on both active treatment and placebo to estimate the conditional average treatment effect. The model was able to separate responders and non-responders across a range of predicted effect sizes. Notably, the average treatment effect for the anti-CD20-antibody test set during nested cross-validation was significantly greater when selecting the model's prediction for the top 50% (HR 0.625, p=0.008) or the top 25% (HR 0.521, p=0.013) most responsive individuals, compared to HR 0.835 (p=0.154) for the entire group. The model trained on the anti-CD20-antibody dataset could also identify responders to laquinimod, finding a significant treatment effect in the top 30% of individuals (HR 0.352, p=0.043). We observed enrichment across a broad range of baseline features in the responder subgroups: younger, more men, shorter disease duration, higher disability scores, and more lesional activity. By simulating a 1-year study where only the 50% predicted to be most responsive are randomized, we could achieve 80% power to detect a significant difference with 6 times less participants than a clinical trial without enrichment. Subgroups of individuals with primary progressive multiple sclerosis who respond favourably to disease modifying therapies can therefore be identified based on their baseline characteristics, even when no significant treatment effect can be found at the whole-group level. The approach allows for predictive enrichment of future clinical trials, as well as personalized treatment selection in the clinic.

Download Full-text

A randomized, placebo-controlled, phase 2 trial of laquinimod in primary progressive multiple sclerosis

Neurology ◽

10.1212/wnl.0000000000010284 ◽

2020 ◽

Vol 95 (8) ◽

pp. e1027-e1040 ◽

Cited By ~ 2

Author(s):

Gavin Giovannoni ◽

Volker Knappertz ◽

Joshua R. Steinerman ◽

Aaron P. Tansy ◽

Thomas Li ◽

...

Keyword(s):

Multiple Sclerosis ◽

Primary Endpoint ◽

Treatment Effect ◽

Brain Volume ◽

Progressive Multiple Sclerosis ◽

Primary Progressive Multiple Sclerosis ◽

Phase 2 ◽

Significant Treatment Effect ◽

Significant Treatment ◽

Primary Progressive

ObjectiveTo evaluate efficacy, safety, and tolerability of laquinimod in patients with primary progressive multiple sclerosis (PPMS).MethodsIn the randomized, double-blind, placebo-controlled, phase 2 study, ARPEGGIO (A Randomized Placebo-controlled Trial Evaluating Laquinimod in PPMS, Gauging Gradations in MRI and Clinical Outcomes), eligible patients with PPMS were randomized 1:1:1 to receive once-daily oral laquinimod 0.6 mg or 1.5 mg or matching placebo. Percentage brain volume change (PBVC; primary endpoint) from baseline to week 48 was assessed by MRI. Secondary and exploratory endpoints included clinical and MRI measures. Efficacy endpoints were evaluated using a predefined, hierarchical statistical testing procedure. Safety was monitored throughout the study. The laquinimod 1.5 mg dose arm was discontinued on January 1, 2016, due to findings of cardiovascular events.ResultsA total of 374 patients were randomized to laquinimod 0.6 mg (n = 139) or 1.5 mg (n = 95) or placebo (n = 140). ARPEGGIO did not meet the primary endpoint of significant treatment effect with laquinimod 0.6 mg vs placebo on PBVC from baseline to week 48 (adjusted mean difference = 0.016%, p = 0.903). Laquinimod 0.6 mg reduced the number of new T2 brain lesions at week 48 (risk ratio 0.4; 95% confidence interval, 0.26–0.69; p = 0.001). Incidence of adverse events was higher among patients treated with laquinimod 0.6 mg (83%) vs laquinimod 1.5 mg (66%) and placebo (78%).ConclusionsLaquinimod 0.6 mg did not demonstrate a statistically significant effect on brain volume loss in PPMS at week 48.Clinicaltrials.gov identifierNCT02284568.Classification of evidenceThis study provides Class I evidence that, although well tolerated, laquinimod 0.6 mg did not demonstrate a significant treatment effect on PBVC in patients with PPMS.

Download Full-text

The Cannabinoid Use in Progressive Inflammatory brain Disease (CUPID) trial: a randomised double-blind placebo-controlled parallel-group multicentre trial and economic evaluation of cannabinoids to slow progression in multiple sclerosis

Health Technology Assessment ◽

10.3310/hta19120 ◽

2015 ◽

Vol 19 (12) ◽

pp. 1-188 ◽

Cited By ~ 25

Author(s):

Susan Ball ◽

Jane Vickery ◽

Jeremy Hobart ◽

Dave Wright ◽

Colin Green ◽

...

Keyword(s):

Multiple Sclerosis ◽

Economic Evaluation ◽

Treatment Effect ◽

Treatment Effects ◽

Rating Scales ◽

Measurement Instruments ◽

Double Blind ◽

Significant Treatment ◽

Parallel Group ◽

Disease Modifying

BackgroundThe Cannabinoid Use in Progressive Inflammatory brain Disease (CUPID) trial aimed to determine whether or not oral Δ9-tetrahydrocannabinol (Δ9-THC) slowed the course of progressive multiple sclerosis (MS); evaluate safety of cannabinoid administration; and, improve methods for testing treatments in progressive MS.ObjectivesThere were three objectives in the CUPID study: (1) to evaluate whether or not Δ9-THC could slow the course of progressive MS; (2) to assess the long-term safety of Δ9-THC; and (3) to explore newer ways of conducting clinical trials in progressive MS.DesignThe CUPID trial was a randomised, double-blind, placebo-controlled, parallel-group, multicentre trial. Patients were randomised in a 2 : 1 ratio to Δ9-THC or placebo. Randomisation was balanced according to Expanded Disability Status Scale (EDSS) score, study site and disease type. Analyses were by intention to treat, following a pre-specified statistical analysis plan. A cranial magnetic resonance imaging (MRI) substudy, Rasch measurement theory (RMT) analyses and an economic evaluation were undertaken.SettingTwenty-seven UK sites.ParticipantsAdults aged 18–65 years with primary or secondary progressive MS, 1-year evidence of disease progression and baseline EDSS 4.0–6.5.InterventionsOral Δ9-THC (maximum 28 mg/day) or matching placebo.Assessment visitsThree and 6 months, and then 6-monthly up to 36 or 42 months.Main outcome measuresPrimary outcomes were time to EDSS progression, and change in Multiple Sclerosis Impact Scale-29 version 2 (MSIS-29v2) 20-point physical subscale (MSIS-29phys) score. Various secondary patient- and clinician-reported outcomes and MRI outcomes were assessed. RMT analyses examined performance of MS-specific rating scales as measurement instruments and tested for a symptomatic or disease-modifying treatment effect. Economic evaluation estimated mean incremental costs and quality-adjusted life-years (QALYs).ResultsEffectiveness– recruitment targets were achieved. Of the 498 randomised patients (332 to active and 166 to placebo), 493 (329 active and 164 placebo) were analysed. Primary outcomes: no significant treatment effect; hazard ratio EDSS score progression (active : placebo) 0.92 [95% confidence interval (CI) 0.68 to 1.23]; and estimated between-group difference in MSIS-29phys score (active–placebo) –0.9 points (95% CI –2.0 to 0.2 points). Secondary clinical and MRI outcomes: no significant treatment effects.Safety– at least one serious adverse event: 35% and 28% of active and placebo patients, respectively.RMT analyses– scale evaluation: MSIS-29 version 2, MS Walking Scale-12 version 2 and MS Spasticity Scale-88 were robust measurement instruments. There was no clear symptomatic or disease-modifying treatment effect.Economic evaluation– estimated mean incremental cost to NHS over usual care, over 3 years £27,443.20 per patient. No between-group difference in QALYs.ConclusionsThe CUPID trial failed to demonstrate a significant treatment effect in primary or secondary outcomes. There were no major safety concerns, but unwanted side effects seemed to affect compliance. Participants were more disabled than in previous studies and deteriorated less than expected, possibly reducing our ability to detect treatment effects. RMT analyses supported performance of MS-specific rating scales as measures, enabled group- and individual person-level examination of treatment effects, but did not influence study inferences. The intervention had significant additional costs with no improvement in health outcomes; therefore, it was dominated by usual care and not cost-effective. Future work should focus on determining further factors to predict clinical deterioration, to inform the development of new studies, and modifying treatments in order to minimise side effects and improve study compliance. The absence of disease-modifying treatments in progressive MS warrants further studies of the cannabinoid pathway in potential neuroprotection.Trial registrationCurrent Controlled Trials ISRCTN62942668.FundingThe National Institute for Health Research Health Technology Assessment programme, the Medical Research Council Efficacy and Mechanism Evaluation programme, Multiple Sclerosis Society and Multiple Sclerosis Trust. The report will be published in full inHealth Technology Assessment; Vol. 19, No. 12. See the NIHR Journals Library website for further project information.

Download Full-text

Measuring Treatment Response in Progressive Multiple Sclerosis—Considerations for Adapting to an Era of Multiple Treatment Options

Biomolecules ◽

10.3390/biom11091342 ◽

2021 ◽

Vol 11 (9) ◽

pp. 1342

Author(s):

Nik Krajnc ◽

Thomas Berger ◽

Gabriel Bsteh

Keyword(s):

Multiple Sclerosis ◽

Clinical Trials ◽

Disease Progression ◽

Outcome Measures ◽

Progressive Multiple Sclerosis ◽

Phase Iii ◽

Disability Progression ◽

Advantages And Disadvantages ◽

Disease Modifying ◽

Disease Modifying Treatment

Disability in multiple sclerosis accrues predominantly in the progressive forms of the disease. While disease-modifying treatment of relapsing MS has drastically evolved over the last quarter-century, the development of efficient drugs for preventing or at least delaying disability in progressive MS has proven more challenging. In that way, many drugs (especially disease-modifying treatments) have been researched in the aspect of delaying disability progression in patients with a progressive course of the disease. While there are some disease-modifying treatments approved for progressive multiple sclerosis, their effect is moderate and limited mostly to patients with clinical and/or radiological signs of disease activity. Several phase III trials have used different primary outcomes with different time frames to define disease progression and to evaluate the efficacy of a disease-modifying treatment. The lack of sufficiently sensitive outcome measures could be a possible explanation for the negative clinical trials in progressive multiple sclerosis. On the other hand, even with a potential outcome measure that would be sensitive enough to determine disease progression and, thus, the efficacy or failure of a disease-modifying treatment, the question of clinical relevance remains unanswered. In this systematic review, we analyzed outcome measures and definitions of disease progression in phase III clinical trials in primary and secondary progressive multiple sclerosis. We discuss advantages and disadvantages of clinical and paraclinical outcome measures aiming for practical ways of combining them to detect disability progression more sensitively both in future clinical trials and current clinical routine.

Download Full-text

Rituximab treatment for multiple sclerosis

Multiple Sclerosis Journal ◽

10.1177/1352458519858604 ◽

2019 ◽

Vol 26 (2) ◽

pp. 137-152 ◽

Cited By ~ 10

Author(s):

Benjamin V Ineichen ◽

Thomas Moridi ◽

Tobias Granberg ◽

Fredrik Piehl

Keyword(s):

Multiple Sclerosis ◽

Current Knowledge ◽

Mechanisms Of Action ◽

Current Evidence ◽

Clinical Aspects ◽

Disease Modifying Therapies ◽

Relapsing Remitting ◽

Cd20 Antibody ◽

Disease Modifying ◽

Anti Cd20

Rituximab, a chimeric anti-CD20-antibody, attracts increasing attention as a treatment option for multiple sclerosis (MS). Apart from smaller controlled trials, an increasing number of studies in real-world populations indicate high efficacy based on clinical and neuroradiological outcomes for rituximab in relapsing-remitting MS patients. Additional evidence also demonstrates efficacy of rituximab with treatment of progressive MS phenotypes. In this topical review, we summarize and discuss current evidence on mechanisms of action, efficacy, safety, tolerance and other clinical aspects of rituximab in the treatment of MS. Finally, we will highlight current knowledge gaps and the need for comparative studies with other disease-modifying therapies in MS.

Download Full-text

Addressing Missing Data in Randomized Clinical Trials: A Causal Inference Perspective

10.21203/rs.2.14910/v1 ◽

2019 ◽

Cited By ~ 1

Author(s):

Ilja Cornelisz ◽

Pim Cuijpers ◽

Tara Donker ◽

Chris van Klaveren

Keyword(s):

Clinical Trials ◽

Random Forest ◽

Causal Inference ◽

Treatment Effect ◽

Randomized Clinical Trials ◽

Outcome Data ◽

Average Treatment Effect ◽

Predictor Variables ◽

Average Treatment ◽

Point Estimates

Abstract Background The importance of randomization in clinical trials has long been acknowledged for avoiding selection bias. Yet, bias concerns re-emerge with selective attrition. This study takes a causal inference perspective in addressing distinct scenarios of missing outcome data (MCAR, MAR and MNAR). Methods This study adopts a causal inference perspective in providing an overview of empirical strategies to estimate the average treatment effect, improve precision of the estimator, and to test whether the underlying identifying assumptions hold. We propose to use Random Forest Lee Bounds (RFLB) to address selective attrition and to obtain more precise average treatment effect intervals. Results When assuming MCAR or MAR, the often untenable identifying assumptions with respect to causal inference can hardly be verified empirically. Instead, missing outcome data in clinical trials should be considered as potentially non-random unobserved events (i.e. MNAR). Using simulated attrition data, we show how average treatment effect intervals can be tightened considerably using RFLB, by exploiting both continuous and discrete attrition predictor variables. Conclusions Bounding approaches should be used to acknowledge selective attrition in randomized clinical trials in acknowledging the resulting uncertainty with respect to causal inference. As such, Random Forest Lee Bounds estimates are more informative than point estimates obtained assuming MCAR or MAR.

Download Full-text

Applying causal models to explore the mechanism of action of simvastatin in progressive multiple sclerosis

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.1818978116 ◽

2019 ◽

Vol 116 (22) ◽

pp. 11020-11027 ◽

Cited By ~ 6

Author(s):

Arman Eshaghi ◽

Rogier A. Kievit ◽

Ferran Prados ◽

Carole H. Sudre ◽

Jennifer Nicholas ◽

...

Keyword(s):

Multiple Sclerosis ◽

Clinical Trials ◽

Treatment Effect ◽

Brain Atrophy ◽

Structural Equation Models ◽

Block Design ◽

Progressive Multiple Sclerosis ◽

Cholesterol Levels ◽

Total Treatment ◽

Independent Model

Understanding the mode of action of drugs is a challenge with conventional methods in clinical trials. Here, we aimed to explore whether simvastatin effects on brain atrophy and disability in secondary progressive multiple sclerosis (SPMS) are mediated by reducing cholesterol or are independent of cholesterol. We applied structural equation models to the MS-STAT trial in which 140 patients with SPMS were randomized to receive placebo or simvastatin. At baseline, after 1 and 2 years, patients underwent brain magnetic resonance imaging; their cognitive and physical disability were assessed on the block design test and Expanded Disability Status Scale (EDSS), and serum total cholesterol levels were measured. We calculated the percentage brain volume change (brain atrophy). We compared two models to select the most likely one: a cholesterol-dependent model with a cholesterol-independent model. The cholesterol-independent model was the most likely option. When we deconstructed the total treatment effect into indirect effects, which were mediated by brain atrophy, and direct effects, simvastatin had a direct effect (independent of serum cholesterol) on both the EDSS, which explained 69% of the overall treatment effect on EDSS, and brain atrophy, which, in turn, was responsible for 31% of the total treatment effect on EDSS [β = −0.037; 95% credible interval (CI) = −0.075, −0.010]. This suggests that simvastatin’s beneficial effects in MS are independent of its effect on lowering peripheral cholesterol levels, implicating a role for upstream intermediate metabolites of the cholesterol synthesis pathway. Importantly, it demonstrates that computational models can elucidate the causal architecture underlying treatment effects in clinical trials of progressive MS.

Download Full-text