scholarly journals Can reproducibility be improved in clinical natural language processing? A study of 7 clinical NLP suites

2020 ◽  
Author(s):  
William Digan ◽  
Aurélie Névéol ◽  
Antoine Neuraz ◽  
Maxime Wack ◽  
David Baudoin ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Workflow Management ◽  
Lessons Learned ◽  
Secondary Use ◽  
Research Fields ◽  
Use Of Data ◽  
Public Repositories ◽  
Clinical Natural Language Processing

Abstract Background The increasing complexity of data streams and computational processes in modern clinical health information systems makes reproducibility challenging. Clinical natural language processing (NLP) pipelines are routinely leveraged for the secondary use of data. Workflow management systems (WMS) have been widely used in bioinformatics to handle the reproducibility bottleneck. Objective To evaluate if WMS and other bioinformatics practices could impact the reproducibility of clinical NLP frameworks. Materials and Methods Based on the literature across multiple research fields (NLP, bioinformatics and clinical informatics) we selected articles which (1) review reproducibility practices and (2) highlight a set of rules or guidelines to ensure tool or pipeline reproducibility. We aggregate insight from the literature to define reproducibility recommendations. Finally, we assess the compliance of 7 NLP frameworks to the recommendations. Results We identified 40 reproducibility features from 8 selected articles. Frameworks based on WMS match more than 50% of features (26 features for LAPPS Grid, 22 features for OpenMinted) compared to 18 features for current clinical NLP framework (cTakes, CLAMP) and 17 features for GATE, ScispaCy, and Textflows. Discussion 34 recommendations are endorsed by at least 2 articles from our selection. Overall, 15 features were adopted by every NLP Framework. Nevertheless, frameworks based on WMS had a better compliance with the features. Conclusion NLP frameworks could benefit from lessons learned from the bioinformatics field (eg, public repositories of curated tools and workflows or use of containers for shareability) to enhance the reproducibility in a clinical setting.

scholarly journals Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings

2017 ◽  
Vol 24 (5) ◽  
pp. 986-991 ◽  
Author(s):  
David S Carrell ◽  
Robert E Schoen ◽  
Daniel A Leffler ◽  
Michele Morris ◽  
Sherri Rose ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Electronic Health Record ◽  
Language Processing ◽  
Lessons Learned ◽  
Clinical Settings ◽  
Health Record ◽  
Widespread Application ◽  
Electronic Health ◽  
Clinical Natural Language Processing

Abstract Objective: Widespread application of clinical natural language processing (NLP) systems requires taking existing NLP systems and adapting them to diverse and heterogeneous settings. We describe the challenges faced and lessons learned in adapting an existing NLP system for measuring colonoscopy quality. Materials and Methods: Colonoscopy and pathology reports from 4 settings during 2013–2015, varying by geographic location, practice type, compensation structure, and electronic health record. Results: Though successful, adaptation required considerably more time and effort than anticipated. Typical NLP challenges in assembling corpora, diverse report structures, and idiosyncratic linguistic content were greatly magnified. Discussion: Strategies for addressing adaptation challenges include assessing site-specific diversity, setting realistic timelines, leveraging local electronic health record expertise, and undertaking extensive iterative development. More research is needed on how to make it easier to adapt NLP systems to new clinical settings. Conclusions: A key challenge in widespread application of NLP is adapting existing systems to new clinical settings.

scholarly journals Cohort Selection for Construction of a Clinical Natural Language Processing Corpus

2021 ◽  
pp. 100024
Author(s):  
Naga Lalitha Valli ALLA ◽  
Aipeng CHEN ◽  
Sean BATONGBACAL ◽  
Chandini NEKKANTTI ◽  
Hong-Jie Dai ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Cohort Selection ◽  
Selection For ◽  
Clinical Natural Language Processing

scholarly journals Natural language processing for web browsing analytics: Challenges, lessons learned, and opportunities

2021 ◽  
pp. 108357
Author(s):  
Daniel Perdices ◽  
Javier Ramos ◽  
José L. García-Dorado ◽  
Iván González ◽  
Jorge E. López de Vergara
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Lessons Learned ◽  
Web Browsing

scholarly journals Toward a clinical text encoder: pretraining for clinical natural language processing with applications to substance misuse

2019 ◽  
Vol 26 (11) ◽  
pp. 1272-1278 ◽  
Author(s):  
Dmitriy Dligach ◽  
Majid Afshar ◽  
Timothy Miller
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Substance Misuse ◽  
Research Direction ◽  
Clinical Text ◽  
Universal Properties ◽  
Classification Tasks ◽  
Performance Gains ◽  
Clinical Natural Language Processing

Abstract Objective Our objective is to develop algorithms for encoding clinical text into representations that can be used for a variety of phenotyping tasks. Materials and Methods Obtaining large datasets to take advantage of highly expressive deep learning methods is difficult in clinical natural language processing (NLP). We address this difficulty by pretraining a clinical text encoder on billing code data, which is typically available in abundance. We explore several neural encoder architectures and deploy the text representations obtained from these encoders in the context of clinical text classification tasks. While our ultimate goal is learning a universal clinical text encoder, we also experiment with training a phenotype-specific encoder. A universal encoder would be more practical, but a phenotype-specific encoder could perform better for a specific task. Results We successfully train several clinical text encoders, establish a new state-of-the-art on comorbidity data, and observe good performance gains on substance misuse data. Discussion We find that pretraining using billing codes is a promising research direction. The representations generated by this type of pretraining have universal properties, as they are highly beneficial for many phenotyping tasks. Phenotype-specific pretraining is a viable route for trading the generality of the pretrained encoder for better performance on a specific phenotyping task. Conclusions We successfully applied our approach to many phenotyping tasks. We conclude by discussing potential limitations of our approach.

scholarly journals Lessons Learned from Teaching Machine Learning and Natural Language Processing to High School Students

2020 ◽  
Vol 34 (09) ◽  
pp. 13397-13403
Author(s):  
Narges Norouzi ◽  
Snigdha Chaturvedi ◽  
Matthew Rutledge
Keyword(s):  
Machine Learning ◽  
High School ◽  
High School Students ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Teaching Machine ◽  
Lessons Learned ◽  
School Students ◽  
Gender Discrepancies

This paper describes an experience in teaching Machine Learning (ML) and Natural Language Processing (NLP) to a group of high school students over an intense one-month period. In this work, we provide an outline of an AI course curriculum we designed for high school students and then evaluate its effectiveness by analyzing student's feedback and student outcomes. After closely observing students, evaluating their responses to our surveys, and analyzing their contribution to the course project, we identified some possible impediments in teaching AI to high school students and propose some measures to avoid them. These measures include employing a combination of objectivist and constructivist pedagogies, reviewing/introducing basic programming concepts at the beginning of the course, and addressing gender discrepancies throughout the course.

scholarly journals CLAMP – a toolkit for efficiently building customized clinical natural language processing pipelines

2017 ◽  
Vol 25 (3) ◽  
pp. 331-336 ◽  
Author(s):  
Ergin Soysal ◽  
Jingqi Wang ◽  
Min Jiang ◽  
Yonghui Wu ◽  
Serguei Pakhomov ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
User Interface ◽  
Natural Language ◽  
Language Processing ◽  
High Performance ◽  
Smoking Status ◽  
Entity Recognition ◽  
Graphic User Interface ◽  
Clinical Text ◽  
Clinical Natural Language Processing

Abstract Existing general clinical natural language processing (NLP) systems such as MetaMap and Clinical Text Analysis and Knowledge Extraction System have been successfully applied to information extraction from clinical text. However, end users often have to customize existing systems for their individual tasks, which can require substantial NLP skills. Here we present CLAMP (Clinical Language Annotation, Modeling, and Processing), a newly developed clinical NLP toolkit that provides not only state-of-the-art NLP components, but also a user-friendly graphic user interface that can help users quickly build customized NLP pipelines for their individual applications. Our evaluation shows that the CLAMP default pipeline achieved good performance on named entity recognition and concept encoding. We also demonstrate the efficiency of the CLAMP graphic user interface in building customized, high-performance NLP pipelines with 2 use cases, extracting smoking status and lab test values. CLAMP is publicly available for research use, and we believe it is a unique asset for the clinical NLP community.

Sign in / Sign up

Export Citation Format

Share Document