A machine learning based two-stage clinical decision support system for predicting patients’ discontinuation from opioid use disorder treatment: retrospective observational study

Background Buprenorphine is a widely used treatment option for patients with opioid use disorder (OUD). Premature discontinuation from this treatment has many negative health and societal consequences. Objective To develop and evaluate a machine learning based two-stage clinical decision-making framework for predicting which patients will discontinue OUD treatment within less than a year. The proposed framework performs such prediction in two stages: (i) at the time of initiating the treatment, and (ii) after two/three months following treatment initiation. Methods For this retrospective observational analysis, we utilized Massachusetts All Payer Claims Data (MA APCD) from the year 2013 to 2015. Study sample included 5190 patients who were commercially insured, initiated buprenorphine treatment between January and December 2014, and did not have any buprenorphine prescription at least one year prior to the date of treatment initiation in 2014. Treatment discontinuation was defined as at least two consecutive months without a prescription for buprenorphine. Six machine learning models (i.e., logistic regression, decision tree, random forest, extreme-gradient boosting, support vector machine, and artificial neural network) were tested using a five-fold cross validation on the input data. The first-stage models used patients’ demographic information. The second-stage models included information on medication adherence during the early phase of treatment based on the proportion of days covered (PDC) measure. Results A substantial percentage of patients (48.7%) who started on buprenorphine discontinued the treatment within one year. The area under receiving operating characteristic curve (C-statistic) for the first stage models varied within a range of 0.55 to 0.59. The inclusion of knowledge regarding patients’ adherence at the early treatment phase in terms of two-months and three-months PDC resulted in a statistically significant increase in the models’ discriminative power (p-value < 0.001) based on the C-statistic. We also constructed interpretable decision classification rules using the decision tree model. Conclusion Machine learning models can predict which patients are most at-risk of premature treatment discontinuation with reasonable discriminative power. The proposed machine learning framework can be used as a tool to help inform a clinical decision support system following further validation. This can potentially help prescribers allocate limited healthcare resources optimally among different groups of patients based on their vulnerability to treatment discontinuation and design personalized support systems for improving patients’ long-term adherence to OUD treatment.