BACKGROUND
Worldwide, the burden of chronic diseases is growing, necessitating novel approaches that complement and go beyond evidence-based medicine. In this respect a promising avenue is the secondary use of Electronic Health Records (EHR) data, where clinical data are analysed to conduct basic and clinical and translational research. Methods based on machine learning algorithms to process EHR are resulting in improved understanding of patients’ clinical trajectories and chronic disease risk prediction, creating a unique opportunity to derive previously unknown clinical insights. However, wealth of patients’ clinical history remains locked behind clinical narratives in free-form text. Consequently, unlocking the full potential of EHR data is contingent on development of Natural Language Processing (NLP) methods to automatically transform clinical text into structured clinical data that can be directly processed using machine learning algorithms.
OBJECTIVE
To provide a comprehensive overview of the development and uptake on NLP methods applied to free-text clinical notes related to chronic diseases, including investigation of challenges faced by NLP methodologies in understanding clinical narratives.
METHODS
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed and searches were conducted in 5 databases using “clinical notes”, “natural language processing” and “chronic disease” as keywords as well as their variations to maximise coverage of the articles.
RESULTS
Of the 2646 articles considered, 100 met the inclusion criteria. Review of the included papers resulted in identification of 42 chronic diseases, which were then further classified into 10 diseases categories using ICD-10. Majority of the studies focused on diseases of circulatory system (N=38) while endocrine and metabolic diseases were fewest (N=12). This was due to the structure of clinical records related to metabolic diseases that typically contain much more structured data than medical records for diseases of circulatory system, which focus more on unstructured data and consequently have seen a stronger focus of NLP. The review has shown that there is a significant increase in the use of machine learning methods compared to rule-based approaches, however deep learning methods remain emergent (N=3). Consequently, majority of works focus on classification of disease phenotype, while only a handful of papers concern the extraction of comorbidities from the free-text or the integration of clinical notes with structured data. There is a notable use of relatively simple methods, such as shallow classifiers (or combination with rule-based methods), due to the interpretability of predictions, which still represents a significant issue for more complex methods. Finally, scarcity of publicly available data may also have contributed to insufficient development of more advanced methods, such as extraction of word embeddings from clinical notes.
CONCLUSIONS
Efforts are needed to improve (1) progression of clinical NLP methods from extraction towards understanding; (2) recognition of relations among entities, rather than entities in isolation; (3) temporal extraction to understand past, current and future clinical events; (4) exploitation of alternative sources of clinical knowledge; and (5) availability of large-scale, de-identified clinical corpora.
Healthcare Consultancy System
