Assessment of Natural Language Processing Methods for Ascertaining the Expanded Disability Status Scale Score From the Electronic Health Records of Patients With Multiple Sclerosis: Algorithm Development and Validation Study

Background The Expanded Disability Status Scale (EDSS) score is a widely used measure to monitor disability progression in people with multiple sclerosis (MS). However, extracting and deriving the EDSS score from unstructured electronic health records can be time-consuming. Objective We aimed to compare rule-based and deep learning natural language processing algorithms for detecting and predicting the total EDSS score and EDSS functional system subscores from the electronic health records of patients with MS. Methods We studied 17,452 electronic health records of 4906 MS patients followed at one of Canada’s largest MS clinics between June 2015 and July 2019. We randomly divided the records into training (80%) and test (20%) data sets, and compared the performance characteristics of 3 natural language processing models. First, we applied a rule-based approach, extracting the EDSS score from sentences containing the keyword “EDSS.” Next, we trained a convolutional neural network (CNN) model to predict the 19 half-step increments of the EDSS score. Finally, we used a combined rule-based–CNN model. For each approach, we determined the accuracy, precision, recall, and F-score compared with the reference standard, which was manually labeled EDSS scores in the clinic database. Results Overall, the combined keyword-CNN model demonstrated the best performance, with accuracy, precision, recall, and an F-score of 0.90, 0.83, 0.83, and 0.83 respectively. Respective figures for the rule-based and CNN models individually were 0.57, 0.91, 0.65, and 0.70, and 0.86, 0.70, 0.70, and 0.70. Because of missing data, the model performance for EDSS subscores was lower than that for the total EDSS score. Performance improved when considering notes with known values of the EDSS subscores. Conclusions A combined keyword-CNN natural language processing model can extract and accurately predict EDSS scores from patient records. This approach can be automated for efficient information extraction in clinical and research settings.

The Electronic Health Record as the Primary Data Source in a Pragmatic Trial: A Case Study

Introduction. Pragmatic clinical trials test interventions in patients representative of real-world medical practice and reduce data collection costs by using data recorded in the electronic health record (EHR) during usual care. We describe our experience using the EHR to measure the primary outcome of a pragmatic trial, hospital readmissions, and important clinical covariates. Methods. The trial enrolled patients recently discharged from the hospital for treatment of heart failure to test whether automated daily monitoring integrated into the EHR could reduce readmissions. The study team used data from the EHR and several data systems that drew on the EHR, supplemented by the hospital admissions files of three states. Results. Almost three-quarters of enrollees’ readmissions over the 12-mo trial period were captured by the EHRs of the study hospitals. State data, which took 7 mo to more than 2 y from first contact to receipt of first data, provided the remaining one-quarter. Considerable expertise was required to resolve differences between the 2 data sources. Common covariates used in trial analyses, such as weight and body mass index during the index hospital stay, were available for >97% of enrollees from the EHR. Ejection fraction, obtained from echocardiograms, was available for only 47.6% of enrollees within the 6-mo window that would likely be expected in a traditional trial. Discussion. In this trial, patient characteristics and outcomes were collected from existing EHR systems, but, as usual for EHRs, they could not be standardized for date or method of measurement and required substantial time and expertise to collect and curate. Hospital admissions, the primary trial outcome, required additional effort to locate and use supplementary sources of data. Highlights Electronic health records are not a single system but a series of overlapping and legacy systems that require time and expertise to use efficiently. Commonly measured patient characteristics such as weight and body mass index are relatively easy to locate for most trial enrollees but less common characteristics, like ejection fraction, are not. Acquiring essential supplementary data—in this trial, state data on hospital admission—can be a lengthy and difficult process.

Characterization of Symptoms and Symptom Clusters for Type 2 Diabetes Using a Large Nationwide Electronic Health Record Database

<b>Purpose. </b>A variety of symptoms may be associated with type 2 diabetes and its complications. Symptoms in chronic diseases may be described in terms of prevalence, severity, and trajectory and often co-occur in groups, known as symptom clusters, which may be representative of a common etiology. The purpose of this study was to characterize type 2 diabetes–related symptoms using a large nationwide electronic health record (EHR) database. <p><b>Methods. </b>We acquired the Cerner Health Facts, a nationwide EHR database. The type 2 diabetes cohort (<i> n </i>= 1,136,301 patients) was identified using a rule-based phenotype method. A multi-step procedure was then used to identify type 2 diabetes–related symptoms based on <i>International Classification of Diseases</i>,<i> </i>9th and 10th revisions, diagnosis codes. Type 2 diabetes–related symptoms and co-occurring symptom clusters, including their temporal patterns, were characterized based the longitudinal EHR data. </p> <p><b>Results.</b> Patients had a mean age of 61.4 years, 51.2% were female, and 70.0% were White. Among 1,136,301 patients, there were 8,008,276 occurrences of 59 symptoms. The most frequently reported symptoms included pain, heartburn, shortness of breath, fatigue, and swelling, which occurred in 21–60% of the patients. We also observed over-represented type 2 diabetes symptoms, including difficulty speaking, feeling confused, trouble remembering, weakness, and drowsiness/sleepiness. Some of these are rare and difficult to detect by traditional patient-reported outcomes studies.</p> <p><b>Conclusion.</b> To the best of our knowledge, this is the first study to use a nationwide EHR database to characterize type 2 diabetes–related symptoms and their temporal patterns. Fifty-nine symptoms, including both over-represented and rare diabetes-related symptoms, were identified. </p>

Characterization of Symptoms and Symptom Clusters for Type 2 Diabetes Using a Large Nationwide Electronic Health Record Database

OBJECTIVE A variety of symptoms may be associated with type 2 diabetes and its complications. Symptoms in chronic diseases may be described in terms of prevalence, severity, and trajectory and often co-occur in groups, known as symptom clusters, which may be representative of a common etiology. The purpose of this study was to characterize type 2 diabetes–related symptoms using a large nationwide electronic health record (EHR) database. Methods We acquired the Cerner Health Facts, a nationwide EHR database. The type 2 diabetes cohort (n = 1,136,301 patients) was identified using a rule-based phenotype method. A multistep procedure was then used to identify type 2 diabetes–related symptoms based on International Classification of Diseases, 9th and 10th revisions, diagnosis codes. Type 2 diabetes–related symptoms and co-occurring symptom clusters, including their temporal patterns, were characterized based the longitudinal EHR data. Results Patients had a mean age of 61.4 years, 51.2% were female, and 70.0% were White. Among 1,136,301 patients, there were 8,008,276 occurrences of 59 symptoms. The most frequently reported symptoms included pain, heartburn, shortness of breath, fatigue, and swelling, which occurred in 21–60% of the patients. We also observed over-represented type 2 diabetes symptoms, including difficulty speaking, feeling confused, trouble remembering, weakness, and drowsiness/sleepiness. Some of these are rare and difficult to detect by traditional patient-reported outcomes studies. Conclusion To the best of our knowledge, this is the first study to use a nationwide EHR database to characterize type 2 diabetes–related symptoms and their temporal patterns. Fifty-nine symptoms, including both over-represented and rare diabetes-related symptoms, were identified.

Characterization of Symptoms and Symptom Clusters for Type 2 Diabetes Using a Large Nationwide Electronic Health Record Database

<b>Purpose. </b>A variety of symptoms may be associated with type 2 diabetes and its complications. Symptoms in chronic diseases may be described in terms of prevalence, severity, and trajectory and often co-occur in groups, known as symptom clusters, which may be representative of a common etiology. The purpose of this study was to characterize type 2 diabetes–related symptoms using a large nationwide electronic health record (EHR) database. <p><b>Methods. </b>We acquired the Cerner Health Facts, a nationwide EHR database. The type 2 diabetes cohort (<i> n </i>= 1,136,301 patients) was identified using a rule-based phenotype method. A multi-step procedure was then used to identify type 2 diabetes–related symptoms based on <i>International Classification of Diseases</i>,<i> </i>9th and 10th revisions, diagnosis codes. Type 2 diabetes–related symptoms and co-occurring symptom clusters, including their temporal patterns, were characterized based the longitudinal EHR data. </p> <p><b>Results.</b> Patients had a mean age of 61.4 years, 51.2% were female, and 70.0% were White. Among 1,136,301 patients, there were 8,008,276 occurrences of 59 symptoms. The most frequently reported symptoms included pain, heartburn, shortness of breath, fatigue, and swelling, which occurred in 21–60% of the patients. We also observed over-represented type 2 diabetes symptoms, including difficulty speaking, feeling confused, trouble remembering, weakness, and drowsiness/sleepiness. Some of these are rare and difficult to detect by traditional patient-reported outcomes studies.</p> <p><b>Conclusion.</b> To the best of our knowledge, this is the first study to use a nationwide EHR database to characterize type 2 diabetes–related symptoms and their temporal patterns. Fifty-nine symptoms, including both over-represented and rare diabetes-related symptoms, were identified. </p>