Measuring Adoption of Patient Priorities-Aligned Care Using Natural Language Processing of Electronic Health Records (Preprint)
BACKGROUND Patient Priorities Care (PPC) is a model of care that aligns health care recommendations with priorities of older adults with multiple chronic conditions. Following identification of patient priorities, this information is documented in the patient’s electronic health record (EHR). OBJECTIVE Our goal is to develop and validate a natural language processing (NLP) model that reliably documents when clinicians identify patient priorities (i.e., values, outcome goals and care preferences) within the EHR as a measure of PPC adoption. METHODS Design: Retrospective analysis of unstructured EHR free-text notes using an NLP model. Setting: National Veteran Health Administration (VHA) EHR. Participants (including the sample size): 778 patient notes of 658 patients from encounters with 144 social workers in the primary care setting Measurements: Each patient’s free-text clinical note was reviewed by two independent reviewers for the presence of PPC language such as priorities, values, and goals. We developed a NLP model that utilized statistical machine learning approaches. The performance of the NLP model in training and validation with 10-fold cross-validation is reported via accuracy, recall, and precision in comparison to the chart review. RESULTS Results: Out of 778 notes, 589 (76%) were identified as containing PPC language (Kappa = 0.82, p-value < 0.001). The NLP model in the training stage had an accuracy of 0.98 (0.98, 0.99), a recall of 0.98 (0.98, 0.99), and precision of 0.98 (0.97, 1.00). The NLP model in the validation stage has an accuracy of 0.92 (0.90, 0.94), a recall of 0.84 (0.79, 0.89), and precision of 0.84 (0.77, 0.91). In contrast, an approach using simple search terms for PPC only had a precision of 0.757. CONCLUSIONS Discussion and Implications: An automated NLP model can reliably measure with high precision, recall, and accuracy when clinicians document patient priorities as a key step in the adoption of PPC. CLINICALTRIAL