scholarly journals Identification of asthma control factor in clinical notes using a hybrid deep learning model

2021 ◽  
Vol 21 (S7) ◽  
Author(s):  
Bhavani Singh Agnikula Kshatriya ◽  
Elham Sagheb ◽  
Chung-Il Wi ◽  
Jungwon Yoon ◽  
Hee Yun Seol ◽  
...  
Keyword(s):  
Deep Learning ◽  
Asthma Control ◽  
Chart Review ◽  
Original Data ◽  
Free Text ◽  
Model Errors ◽  
Asthma Diagnosis ◽  
Rule Based ◽  
Clinical Notes ◽  
Distant Supervision

Abstract Background There are significant variabilities in guideline-concordant documentation in asthma care. However, assessing clinician’s documentation is not feasible using only structured data but requires labor-intensive chart review of electronic health records (EHRs). A certain guideline element in asthma control factors, such as review inhaler techniques, requires context understanding to correctly capture from EHR free text. Methods The study data consist of two sets: (1) manual chart reviewed data—1039 clinical notes of 300 patients with asthma diagnosis, and (2) weakly labeled data (distant supervision)—27,363 clinical notes from 800 patients with asthma diagnosis. A context-aware language model, Bidirectional Encoder Representations from Transformers (BERT) was developed to identify inhaler techniques in EHR free text. Both original BERT and clinical BioBERT (cBERT) were applied with a cost-sensitivity to deal with imbalanced data. The distant supervision using weak labels by rules was also incorporated to augment the training set and alleviate a costly manual labeling process in the development of a deep learning algorithm. A hybrid approach using post-hoc rules was also explored to fix BERT model errors. The performance of BERT with/without distant supervision, hybrid, and rule-based models were compared in precision, recall, F-score, and accuracy. Results The BERT models on the original data performed similar to a rule-based model in F1-score (0.837, 0.845, and 0.838 for rules, BERT, and cBERT, respectively). The BERT models with distant supervision produced higher performance (0.853 and 0.880 for BERT and cBERT, respectively) than without distant supervision and a rule-based model. The hybrid models performed best in F1-score of 0.877 and 0.904 over the distant supervision on BERT and cBERT. Conclusions The proposed BERT models with distant supervision demonstrated its capability to identify inhaler techniques in EHR free text, and outperformed both the rule-based model and BERT models trained on the original data. With a distant supervision approach, we may alleviate costly manual chart review to generate the large training data required in most deep learning-based models. A hybrid model was able to fix BERT model errors and further improve the performance.

Natural Language Processing Accurately Categorizes Indications, Findings and Pathology Reports From Multicenter Colonoscopy (Preprint)

2021 ◽  
Author(s):  
Shashank Reddy Vadyala ◽  
Eric A. Sherer
Keyword(s):  
Deep Learning ◽  
Decision Support ◽  
Clinical Decision Support System ◽  
Clinical Decision ◽  
Free Text ◽  
Model Framework ◽  
Clinical Notes ◽  
Pathology Reports ◽  
Deep Learning Model ◽  
Extract Information

BACKGROUND Colonoscopy is used for colorectal cancer (CRC) screening. Extracting details of the colonoscopy findings from free text in electronic health records can be used to determine patient risk for CRC and colorectal screening strategies. We developed a deep learning model framework to extract information for the clinical decision support system to interpret relevant free-text reports, including indications, pathology, and findings notes. OBJECTIVE We developed and evaluated the accuracy of a deep learning model framework to extract information for the clinical decision support system to interpret relevant free-text reports, including indications, pathology, and findings notes. METHODS The Bio-Bi-LSTM-CRF framework was developed using Bidirectional Long Short-term Memory (Bi-LSTM) and Conditional Random Fields (CRF) to extract several clinical features from these free-text reports including indications for the colonoscopy, findings during the colonoscopy, and pathology of resected material. We trained the Bio-Bi-LSTM-CRF and existing Bi-LSTM-CRF models on 80% of 4,000 manually annotated notes from 3,867 patients. These clinical notes were from a group of patients over 40 years of age enrolled in four Veterans Affairs Medical Centers. A total of 10% of the remaining annotated notes were used to train hyperparameter and the remaining 10% were used to evaluate the accuracy of our model Bio-Bi-LSTM-CRF and compare to Bi-LSTM-CRF. RESULTS Bio-Bi-LSTM-CRF test set accuracy was 93.5% for indications reports, 88.0% for findings reports, 96.5% for pathology reports, 85.0% for number of polyps’ entity, 81.0% for size of polyps’ entity, 94.0% for locations of polyps entity, and 92.0% for poly removal procedure entity, and 96.5% for colon location of polyps. The accuracy of the Bio-Bi-LSTM-CRF methods between facilities ranged from 92.0% to 94.0%. The Bio-Bi-LSTM-CRF model's overall accuracy was between 8.0% and 13.0% points higher than the Bi-LSTM-CRF method in entity identification. The Bio-Bi-LSTM-CRF model achieved a higher accuracy for all outcomes for all four facilities. CONCLUSIONS Our experiments show that the bidirectional encoder representations by integrating dictionary function vector from Bio-Bi-LSTM-CRF and strategies character sequence embedding approach is an effective way to identify colonoscopy features from EHR-extracted clinical notes. The Bio-Bi-LSTM-CRF model creates new opportunities to identify patients at risk for colon cancer and study their health outcomes.

scholarly journals Weakly supervised natural language processing for assessing patient-centered outcome following prostate cancer treatment

JAMIA Open ◽  
2019 ◽  
Vol 2 (1) ◽  
pp. 150-159 ◽  
Author(s):  
Imon Banerjee ◽  
Kevin Li ◽  
Martin Seneviratne ◽  
Michelle Ferrari ◽  
Tina Seto ◽  
...  
Keyword(s):  
Prostate Cancer ◽  
Language Processing ◽  
Chart Review ◽  
Patient Centered ◽  
Prostate Cancer Treatment ◽  
Rule Based ◽  
Clinical Notes ◽  
Domain Specific ◽  
Sentence Level ◽  
Weakly Supervised

Abstract Background The population-based assessment of patient-centered outcomes (PCOs) has been limited by the efficient and accurate collection of these data. Natural language processing (NLP) pipelines can determine whether a clinical note within an electronic medical record contains evidence on these data. We present and demonstrate the accuracy of an NLP pipeline that targets to assess the presence, absence, or risk discussion of two important PCOs following prostate cancer treatment: urinary incontinence (UI) and bowel dysfunction (BD). Methods We propose a weakly supervised NLP approach which annotates electronic medical record clinical notes without requiring manual chart review. A weighted function of neural word embedding was used to create a sentence-level vector representation of relevant expressions extracted from the clinical notes. Sentence vectors were used as input for a multinomial logistic model, with output being either presence, absence or risk discussion of UI/BD. The classifier was trained based on automated sentence annotation depending only on domain-specific dictionaries (weak supervision). Results The model achieved an average F1 score of 0.86 for the sentence-level, three-tier classification task (presence/absence/risk) in both UI and BD. The model also outperformed a pre-existing rule-based model for note-level annotation of UI with significant margin. Conclusions We demonstrate a machine learning method to categorize clinical notes based on important PCOs that trains a classifier on sentence vector representations labeled with a domain-specific dictionary, which eliminates the need for manual engineering of linguistic rules or manual chart review for extracting the PCOs. The weakly supervised NLP pipeline showed promising sensitivity and specificity for identifying important PCOs in unstructured clinical text notes compared to rule-based algorithms. Trial registration This is a chart review study and approved by Institutional Review Board (IRB).

scholarly journals Exploring semantic deep learning for building reliable and reusable one health knowledge from PubMed systematic reviews and veterinary clinical notes

2019 ◽  
Vol 10 (S1) ◽  
Author(s):  
Mercedes Arguello-Casteleiro ◽  
Robert Stevens ◽  
Julio Des-Diz ◽  
Chris Wroe ◽  
Maria Jesus Fernandez-Prieto ◽  
...  
Keyword(s):  
Deep Learning ◽  
Best Practice ◽  
One Health ◽  
Health Knowledge ◽  
Free Text ◽  
Learning Approach ◽  
Text Data ◽  
Clinical Notes ◽  
Short Forms ◽  
F Measure

Abstract Background Deep Learning opens up opportunities for routinely scanning large bodies of biomedical literature and clinical narratives to represent the meaning of biomedical and clinical terms. However, the validation and integration of this knowledge on a scale requires cross checking with ground truths (i.e. evidence-based resources) that are unavailable in an actionable or computable form. In this paper we explore how to turn information about diagnoses, prognoses, therapies and other clinical concepts into computable knowledge using free-text data about human and animal health. We used a Semantic Deep Learning approach that combines the Semantic Web technologies and Deep Learning to acquire and validate knowledge about 11 well-known medical conditions mined from two sets of unstructured free-text data: 300 K PubMed Systematic Review articles (the PMSB dataset) and 2.5 M veterinary clinical notes (the VetCN dataset). For each target condition we obtained 20 related clinical concepts using two deep learning methods applied separately on the two datasets, resulting in 880 term pairs (target term, candidate term). Each concept, represented by an n-gram, is mapped to UMLS using MetaMap; we also developed a bespoke method for mapping short forms (e.g. abbreviations and acronyms). Existing ontologies were used to formally represent associations. We also create ontological modules and illustrate how the extracted knowledge can be queried. The evaluation was performed using the content within BMJ Best Practice. Results MetaMap achieves an F measure of 88% (precision 85%, recall 91%) when applied directly to the total of 613 unique candidate terms for the 880 term pairs. When the processing of short forms is included, MetaMap achieves an F measure of 94% (precision 92%, recall 96%). Validation of the term pairs with BMJ Best Practice yields precision between 98 and 99%. Conclusions The Semantic Deep Learning approach can transform neural embeddings built from unstructured free-text data into reliable and reusable One Health knowledge using ontologies and content from BMJ Best Practice.

Information extraction from clinical notes: A systematic review for Chronic Diseases (Preprint)

2018 ◽  
Author(s):  
Seyedmostafa Sheikhalishahi ◽  
Riccardo Miotto ◽  
Joel T. Dudley ◽  
Alberto Lavelli ◽  
Fabio Rinaldi ◽  
...  
Keyword(s):  
Machine Learning ◽  
Chronic Diseases ◽  
Language Processing ◽  
Metabolic Diseases ◽  
Circulatory System ◽  
Structured Data ◽  
Machine Learning Algorithms ◽  
Free Text ◽  
Rule Based ◽  
Clinical Notes

BACKGROUND Worldwide, the burden of chronic diseases is growing, necessitating novel approaches that complement and go beyond evidence-based medicine. In this respect a promising avenue is the secondary use of Electronic Health Records (EHR) data, where clinical data are analysed to conduct basic and clinical and translational research. Methods based on machine learning algorithms to process EHR are resulting in improved understanding of patients’ clinical trajectories and chronic disease risk prediction, creating a unique opportunity to derive previously unknown clinical insights. However, wealth of patients’ clinical history remains locked behind clinical narratives in free-form text. Consequently, unlocking the full potential of EHR data is contingent on development of Natural Language Processing (NLP) methods to automatically transform clinical text into structured clinical data that can be directly processed using machine learning algorithms. OBJECTIVE To provide a comprehensive overview of the development and uptake on NLP methods applied to free-text clinical notes related to chronic diseases, including investigation of challenges faced by NLP methodologies in understanding clinical narratives. METHODS The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed and searches were conducted in 5 databases using “clinical notes”, “natural language processing” and “chronic disease” as keywords as well as their variations to maximise coverage of the articles. RESULTS Of the 2646 articles considered, 100 met the inclusion criteria. Review of the included papers resulted in identification of 42 chronic diseases, which were then further classified into 10 diseases categories using ICD-10. Majority of the studies focused on diseases of circulatory system (N=38) while endocrine and metabolic diseases were fewest (N=12). This was due to the structure of clinical records related to metabolic diseases that typically contain much more structured data than medical records for diseases of circulatory system, which focus more on unstructured data and consequently have seen a stronger focus of NLP. The review has shown that there is a significant increase in the use of machine learning methods compared to rule-based approaches, however deep learning methods remain emergent (N=3). Consequently, majority of works focus on classification of disease phenotype, while only a handful of papers concern the extraction of comorbidities from the free-text or the integration of clinical notes with structured data. There is a notable use of relatively simple methods, such as shallow classifiers (or combination with rule-based methods), due to the interpretability of predictions, which still represents a significant issue for more complex methods. Finally, scarcity of publicly available data may also have contributed to insufficient development of more advanced methods, such as extraction of word embeddings from clinical notes. CONCLUSIONS Efforts are needed to improve (1) progression of clinical NLP methods from extraction towards understanding; (2) recognition of relations among entities, rather than entities in isolation; (3) temporal extraction to understand past, current and future clinical events; (4) exploitation of alternative sources of clinical knowledge; and (5) availability of large-scale, de-identified clinical corpora.

scholarly journals Extraction of Family History Information From Clinical Notes: Deep Learning and Heuristics Approach

10.2196/22898 ◽  
2020 ◽  
Vol 8 (12) ◽  
pp. e22898
Author(s):  
João Figueira Silva ◽  
João Rafael Almeida ◽  
Sérgio Matos
Keyword(s):  
Deep Learning ◽  
Family History ◽  
Family Member ◽  
Hybrid System ◽  
Family History Information ◽  
Rule Based ◽  
Clinical Notes ◽  
History Information ◽  
Test Sets ◽  
The Impact

Background Electronic health records store large amounts of patient clinical data. Despite efforts to structure patient data, clinical notes containing rich patient information remain stored as free text, greatly limiting its exploitation. This includes family history, which is highly relevant for applications such as diagnosis and prognosis. Objective This study aims to develop automatic strategies for annotating family history information in clinical notes, focusing not only on the extraction of relevant entities such as family members and disease mentions but also on the extraction of relations between the identified entities. Methods This study extends a previous contribution for the 2019 track on family history extraction from national natural language processing clinical challenges by improving a previously developed rule-based engine, using deep learning (DL) approaches for the extraction of entities from clinical notes, and combining both approaches in a hybrid end-to-end system capable of successfully extracting family member and observation entities and the relations between those entities. Furthermore, this study analyzes the impact of factors such as the use of external resources and different types of embeddings in the performance of DL models. Results The approaches developed were evaluated in a first task regarding entity extraction and in a second task concerning relation extraction. The proposed DL approach improved observation extraction, obtaining F1 scores of 0.8688 and 0.7907 in the training and test sets, respectively. However, DL approaches have limitations in the extraction of family members. The rule-based engine was adjusted to have higher generalizing capability and achieved family member extraction F1 scores of 0.8823 and 0.8092 in the training and test sets, respectively. The resulting hybrid system obtained F1 scores of 0.8743 and 0.7979 in the training and test sets, respectively. For the second task, the original evaluator was adjusted to perform a more exact evaluation than the original one, and the hybrid system obtained F1 scores of 0.6480 and 0.5082 in the training and test sets, respectively. Conclusions We evaluated the impact of several factors on the performance of DL models, and we present an end-to-end system for extracting family history information from clinical notes, which can help in the structuring and reuse of this type of information. The final hybrid solution is provided in a publicly available code repository.

scholarly journals Extraction of Family History Information From Clinical Notes: Deep Learning and Heuristics Approach (Preprint)

2020 ◽  
Author(s):  
João Figueira Silva ◽  
João Rafael Almeida ◽  
Sérgio Matos
Keyword(s):  
Deep Learning ◽  
Family History ◽  
Family Member ◽  
Hybrid System ◽  
Family History Information ◽  
Rule Based ◽  
Clinical Notes ◽  
History Information ◽  
Test Sets ◽  
The Impact

BACKGROUND Electronic health records store large amounts of patient clinical data. Despite efforts to structure patient data, clinical notes containing rich patient information remain stored as free text, greatly limiting its exploitation. This includes family history, which is highly relevant for applications such as diagnosis and prognosis. OBJECTIVE This study aims to develop automatic strategies for annotating family history information in clinical notes, focusing not only on the extraction of relevant entities such as family members and disease mentions but also on the extraction of relations between the identified entities. METHODS This study extends a previous contribution for the 2019 track on family history extraction from national natural language processing clinical challenges by improving a previously developed rule-based engine, using deep learning (DL) approaches for the extraction of entities from clinical notes, and combining both approaches in a hybrid end-to-end system capable of successfully extracting family member and observation entities and the relations between those entities. Furthermore, this study analyzes the impact of factors such as the use of external resources and different types of embeddings in the performance of DL models. RESULTS The approaches developed were evaluated in a first task regarding entity extraction and in a second task concerning relation extraction. The proposed DL approach improved observation extraction, obtaining F<sub>1</sub> scores of 0.8688 and 0.7907 in the training and test sets, respectively. However, DL approaches have limitations in the extraction of family members. The rule-based engine was adjusted to have higher generalizing capability and achieved family member extraction F<sub>1</sub> scores of 0.8823 and 0.8092 in the training and test sets, respectively. The resulting hybrid system obtained F<sub>1</sub> scores of 0.8743 and 0.7979 in the training and test sets, respectively. For the second task, the original evaluator was adjusted to perform a more exact evaluation than the original one, and the hybrid system obtained F<sub>1</sub> scores of 0.6480 and 0.5082 in the training and test sets, respectively. CONCLUSIONS We evaluated the impact of several factors on the performance of DL models, and we present an end-to-end system for extracting family history information from clinical notes, which can help in the structuring and reuse of this type of information. The final hybrid solution is provided in a publicly available code repository.

scholarly journals Ascertaining Framingham heart failure phenotype from inpatient electronic health record data using natural language processing: a multicentre Atherosclerosis Risk in Communities (ARIC) validation study

BMJ Open ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. e047356
Author(s):  
Carlton R Moore ◽  
Saumya Jain ◽  
Stephanie Haas ◽  
Harish Yadav ◽  
Eric Whitsel ◽  
...  
Keyword(s):  
Language Processing ◽  
Validation Dataset ◽  
Free Text ◽  
Electronic Health Record Data ◽  
Atherosclerosis Risk In Communities ◽  
Clinical Notes ◽  
Atherosclerosis Risk ◽  
Record Data ◽  
Sensitivity Specificity ◽  
Aric Study

ObjectivesUsing free-text clinical notes and reports from hospitalised patients, determine the performance of natural language processing (NLP) ascertainment of Framingham heart failure (HF) criteria and phenotype.Study designA retrospective observational study design of patients hospitalised in 2015 from four hospitals participating in the Atherosclerosis Risk in Communities (ARIC) study was used to determine NLP performance in the ascertainment of Framingham HF criteria and phenotype.SettingFour ARIC study hospitals, each representing an ARIC study region in the USA.ParticipantsA stratified random sample of hospitalisations identified using a broad range of International Classification of Disease, ninth revision, diagnostic codes indicative of an HF event and occurring during 2015 was drawn for this study. A randomly selected set of 394 hospitalisations was used as the derivation dataset and 406 hospitalisations was used as the validation dataset.InterventionUse of NLP on free-text clinical notes and reports to ascertain Framingham HF criteria and phenotype.Primary and secondary outcome measuresNLP performance as measured by sensitivity, specificity, positive-predictive value (PPV) and agreement in ascertainment of Framingham HF criteria and phenotype. Manual medical record review by trained ARIC abstractors was used as the reference standard.ResultsOverall, performance of NLP ascertainment of Framingham HF phenotype in the validation dataset was good, with 78.8%, 81.7%, 84.4% and 80.0% for sensitivity, specificity, PPV and agreement, respectively.ConclusionsBy decreasing the need for manual chart review, our results on the use of NLP to ascertain Framingham HF phenotype from free-text electronic health record data suggest that validated NLP technology holds the potential for significantly improving the feasibility and efficiency of conducting large-scale epidemiologic surveillance of HF prevalence and incidence.

scholarly journals Measuring Adoption of Patient Priorities-Aligned Care Using Natural Language Processing

2020 ◽  
Vol 4 (Supplement_1) ◽  
pp. 183-183
Author(s):  
Javad Razjouyan ◽  
Jennifer Freytag ◽  
Edward Odom ◽  
Lilian Dindo ◽  
Aanand Naik
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Chart Review ◽  
Group Analysis ◽  
Intervention Group ◽  
Multiple Chronic Conditions ◽  
Free Text ◽  
Term Care

Abstract Patient Priorities Care (PPC) is a model of care that aligns health care recommendations with priorities of older adults with multiple chronic conditions. Social workers (SW), after online training, document PPC in the patient’s electronic health record (EHR). Our goal is to identify free-text notes with PPC language using a natural language processing (NLP) model and to measure PPC adoption and effect on long term services and support (LTSS) use. Free-text notes from the EHR produced by trained SWs passed through a hybrid NLP model that utilized rule-based and statistical machine learning. NLP accuracy was validated against chart review. Patients who received PPC were propensity matched with patients not receiving PPC (control) on age, gender, BMI, Charlson comorbidity index, facility and SW. The change in LTSS utilization 6-month intervals were compared by groups with univariate analysis. Chart review indicated that 491 notes out of 689 had PPC language and the NLP model reached to precision of 0.85, a recall of 0.90, an F1 of 0.87, and an accuracy of 0.91. Within group analysis shows that intervention group used LTSS 1.8 times more in the 6 months after the encounter compared to 6 months prior. Between group analysis shows that intervention group has significant higher number of LTSS utilization (p=0.012). An automated NLP model can be used to reliably measure the adaptation of PPC by SW. PPC seems to encourage use of LTSS that may delay time to long term care placement.

Beyond Deep Learning: An Econometric Example

2020 ◽  
Vol 28 (Supp01) ◽  
pp. 31-38 ◽  
Author(s):  
Ruofan Liao ◽  
Paravee Maneejuk ◽  
Songsak Sriboonchitta
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Prediction Models ◽  
Original Data ◽  
Parametric Model ◽  
Parametric Models ◽  
The Past ◽  
Currency Exchange ◽  
Currency Exchange Rate ◽  
Linear And Nonlinear

In the past, in many areas, the best prediction models were linear and nonlinear parametric models. In the last decade, in many application areas, deep learning has shown to lead to more accurate predictions than the parametric models. Deep learning-based predictions are reasonably accurate, but not perfect. How can we achieve better accuracy? To achieve this objective, we propose to combine neural networks with parametric model: namely, to train neural networks not on the original data, but on the differences between the actual data and the predictions of the parametric model. On the example of predicting currency exchange rate, we show that this idea indeed leads to more accurate predictions.

Sign in / Sign up

Export Citation Format

Share Document