BACKGROUND
Emerging interest in precision health and the increasing availability of patient- and population-level data sets present considerable potential to enable analytical approaches to identify and mitigate the negative effects of social factors on health. These issues are not satisfactorily addressed in typical medical care encounters, and thus, opportunities to improve health outcomes, reduce costs, and improve coordination of care are not realized. Furthermore, methodological expertise on the use of varied patient- and population-level data sets and machine learning to predict need for supplemental services is limited.
OBJECTIVE
The objective of this study was to leverage a comprehensive range of clinical, behavioral, social risk, and social determinants of health factors in order to develop decision models capable of identifying patients in need of various wraparound social services.
METHODS
We used comprehensive patient- and population-level data sets to build decision models capable of predicting need for behavioral health, dietitian, social work, or other social service referrals within a safety-net health system using area under the receiver operating characteristic curve (AUROC), sensitivity, precision, F1 score, and specificity. We also evaluated the value of population-level social determinants of health data sets in improving machine learning performance of the models.
RESULTS
Decision models for each wraparound service demonstrated performance measures ranging between 59.2%% and 99.3%. These results were statistically superior to the performance measures demonstrated by our previous models which used a limited data set and whose performance measures ranged from 38.2% to 88.3% (behavioural health: F1 score <i>P</i><.001, AUROC <i>P</i>=.01; social work: F1 score <i>P</i><.001, AUROC <i>P</i>=.03; dietitian: F1 score <i>P</i>=.001, AUROC <i>P</i>=.001; other: F1 score <i>P</i>=.01, AUROC <i>P</i>=.02); however, inclusion of additional population-level social determinants of health did not contribute to any performance improvements (behavioural health: F1 score <i>P</i>=.08, AUROC <i>P</i>=.09; social work: F1 score <i>P</i>=.16, AUROC <i>P</i>=.09; dietitian: F1 score <i>P</i>=.08, AUROC <i>P</i>=.14; other: F1 score <i>P</i>=.33, AUROC <i>P</i>=.21) in predicting the need for referral in our population of vulnerable patients seeking care at a safety-net provider.
CONCLUSIONS
Precision health–enabled decision models that leverage a wide range of patient- and population-level data sets and advanced machine learning methods are capable of predicting need for various wraparound social services with good performance.
Monotonic classification: An overview on algorithms, performance measures and data sets
