The Use of Natural Language Processing to Ascertain Suicide Ideation/Attempt from Clinical Notes Within a Large Integrated Healthcare System (Preprint)

UNSTRUCTURED Purpose: Identifying risk factors for suicide using progress notes and administrative data is time consuming and usually requires manual case review. In this study, a natural language processing computerized algorithm was developed and implemented to automatically ascertain suicide ideation/attempt from clinical notes in a large integrated healthcare system, Kaiser Permanente Southern California. Methods: Clinical notes containing prespecified relevant keywords and phrases related to suicidal ideation/attempt between 2010 and 2018 were extracted from our organization’s electronic health record system. A random sample of 864 clinical notes was selected and equally divided into four subsets. These subsets were reviewed and classified as one of the following three suicide ideation/attempt categories: “Current”, “Historical” and “No” for each note by experienced research chart abstractors. The first three training datasets were used to develop the rule-based computerized algorithm sequentially and the fourth validation dataset was used to evaluate the algorithm performance. The validated algorithm was then applied to the entire study sample of clinical notes. Results: The computerized algorithm ascertained 23 of the 26 confirmed “Current” suicide ideation/attempt events and all 10 confirmed “Historical” suicide ideation/attempt events in the validation dataset. This algorithm produced a 88.5% sensitivity and 100.0% positive predictive value (PPV) for “Current” suicide ideation/attempt, and a 100.0% sensitivity and 100.0% PPV for “Historical” suicide ideation/attempt. After applying the computerized process to the entire study population sample, we identified a total of 1,050,289 “Current” ideation/attempt events and 293,038 “Historical” ideation/attempt events during the study period. Among the 400,436 individuals who were identified as having a “Current” suicide ideation/attempt event, 115,197 (28.8%) were 15-24 years old at the first event, 234,924 (58.7%) were female, 165,084 (41.7%) were Hispanic, and 150,645 (37.6%) had two or more events in the study period. Conclusions: Our study demonstrated that a natural language processing computerized algorithm can effectively ascertain suicide ideation/attempt from the free-text clinical notes in the electronic health record of a diverse patient population. This algorithm can be utilized in support of suicide prevention programs and patient care management.