scholarly journals Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach (Preprint)

2019 ◽  
Author(s):  
Irena Spasic ◽  
Dominik Krzeminski ◽  
Padraig Corcoran ◽  
Alexander Balinsky
Keyword(s):  
Clinical Trials ◽  
Language Processing ◽  
Medical Records ◽  
Training Data ◽  
Eligibility Criteria ◽  
Rule Based ◽  
Cohort Selection ◽  
Selection For ◽  
The Given ◽  
F Measure

BACKGROUND Clinical trials are an important step in introducing new interventions into clinical practice by generating data on their safety and efficacy. Clinical trials need to ensure that participants are similar so that the findings can be attributed to the interventions studied and not to some other factors. Therefore, each clinical trial defines eligibility criteria, which describe characteristics that must be shared by the participants. Unfortunately, the complexities of eligibility criteria may not allow them to be translated directly into readily executable database queries. Instead, they may require careful analysis of the narrative sections of medical records. Manual screening of medical records is time consuming, thus negatively affecting the timeliness of the recruitment process. OBJECTIVE Track 1 of the 2018 National Natural Language Processing Clinical Challenge focused on the task of cohort selection for clinical trials, aiming to answer the following question: Can natural language processing be applied to narrative medical records to identify patients who meet eligibility criteria for clinical trials? The task required the participating systems to analyze longitudinal patient records to determine if the corresponding patients met the given eligibility criteria. We aimed to describe a system developed to address this task. METHODS Our system consisted of 13 classifiers, one for each eligibility criterion. All classifiers used a bag-of-words document representation model. To prevent the loss of relevant contextual information associated with such representation, a pattern-matching approach was used to extract context-sensitive features. They were embedded back into the text as lexically distinguishable tokens, which were consequently featured in the bag-of-words representation. Supervised machine learning was chosen wherever a sufficient number of both positive and negative instances was available to learn from. A rule-based approach focusing on a small set of relevant features was chosen for the remaining criteria. RESULTS The system was evaluated using microaveraged F measure. Overall, 4 machine algorithms, including support vector machine, logistic regression, naïve Bayesian classifier, and gradient tree boosting (GTB), were evaluated on the training data using 10–fold cross-validation. Overall, GTB demonstrated the most consistent performance. Its performance peaked when oversampling was used to balance the training data. The final evaluation was performed on previously unseen test data. On average, the F measure of 89.04% was comparable to 3 of the top ranked performances in the shared task (91.11%, 90.28%, and 90.21%). With an F measure of 88.14%, we significantly outperformed these systems (81.03%, 78.50%, and 70.81%) in identifying patients with advanced coronary artery disease. CONCLUSIONS The holdout evaluation provides evidence that our system was able to identify eligible patients for the given clinical trial with high accuracy. Our approach demonstrates how rule-based knowledge infusion can improve the performance of machine learning algorithms even when trained on a relatively small dataset.

scholarly journals Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach

10.2196/15980 ◽  
2019 ◽  
Vol 7 (4) ◽  
pp. e15980 ◽  
Author(s):  
Irena Spasic ◽  
Dominik Krzeminski ◽  
Padraig Corcoran ◽  
Alexander Balinsky
Keyword(s):  
Clinical Trials ◽  
Language Processing ◽  
Medical Records ◽  
Training Data ◽  
Eligibility Criteria ◽  
Rule Based ◽  
Cohort Selection ◽  
Selection For ◽  
The Given ◽  
F Measure

Background Clinical trials are an important step in introducing new interventions into clinical practice by generating data on their safety and efficacy. Clinical trials need to ensure that participants are similar so that the findings can be attributed to the interventions studied and not to some other factors. Therefore, each clinical trial defines eligibility criteria, which describe characteristics that must be shared by the participants. Unfortunately, the complexities of eligibility criteria may not allow them to be translated directly into readily executable database queries. Instead, they may require careful analysis of the narrative sections of medical records. Manual screening of medical records is time consuming, thus negatively affecting the timeliness of the recruitment process. Objective Track 1 of the 2018 National Natural Language Processing Clinical Challenge focused on the task of cohort selection for clinical trials, aiming to answer the following question: Can natural language processing be applied to narrative medical records to identify patients who meet eligibility criteria for clinical trials? The task required the participating systems to analyze longitudinal patient records to determine if the corresponding patients met the given eligibility criteria. We aimed to describe a system developed to address this task. Methods Our system consisted of 13 classifiers, one for each eligibility criterion. All classifiers used a bag-of-words document representation model. To prevent the loss of relevant contextual information associated with such representation, a pattern-matching approach was used to extract context-sensitive features. They were embedded back into the text as lexically distinguishable tokens, which were consequently featured in the bag-of-words representation. Supervised machine learning was chosen wherever a sufficient number of both positive and negative instances was available to learn from. A rule-based approach focusing on a small set of relevant features was chosen for the remaining criteria. Results The system was evaluated using microaveraged F measure. Overall, 4 machine algorithms, including support vector machine, logistic regression, naïve Bayesian classifier, and gradient tree boosting (GTB), were evaluated on the training data using 10–fold cross-validation. Overall, GTB demonstrated the most consistent performance. Its performance peaked when oversampling was used to balance the training data. The final evaluation was performed on previously unseen test data. On average, the F measure of 89.04% was comparable to 3 of the top ranked performances in the shared task (91.11%, 90.28%, and 90.21%). With an F measure of 88.14%, we significantly outperformed these systems (81.03%, 78.50%, and 70.81%) in identifying patients with advanced coronary artery disease. Conclusions The holdout evaluation provides evidence that our system was able to identify eligible patients for the given clinical trial with high accuracy. Our approach demonstrates how rule-based knowledge infusion can improve the performance of machine learning algorithms even when trained on a relatively small dataset.

scholarly journals Clinical trial cohort selection based on multi-level rule-based natural language processing system

2019 ◽  
Vol 26 (11) ◽  
pp. 1218-1226 ◽  
Author(s):  
Long Chen ◽  
Yu Gu ◽  
Xin Ji ◽  
Chao Lou ◽  
Zhiyong Sun ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Rule Based ◽  
Language System ◽  
Unified Medical Language System ◽  
Rule Based System ◽  
Medical Language ◽  
Cohort Selection

Abstract Objective Identifying patients who meet selection criteria for clinical trials is typically challenging and time-consuming. In this article, we describe our clinical natural language processing (NLP) system to automatically assess patients’ eligibility based on their longitudinal medical records. This work was part of the 2018 National NLP Clinical Challenges (n2c2) Shared-Task and Workshop on Cohort Selection for Clinical Trials. Materials and Methods The authors developed an integrated rule-based clinical NLP system which employs a generic rule-based framework plugged in with lexical-, syntactic- and meta-level, task-specific knowledge inputs. In addition, the authors also implemented and evaluated a general clinical NLP (cNLP) system which is built with the Unified Medical Language System and Unstructured Information Management Architecture. Results and Discussion The systems were evaluated as part of the 2018 n2c2-1 challenge, and authors’ rule-based system obtained an F-measure of 0.9028, ranking fourth at the challenge and had less than 1% difference from the best system. While the general cNLP system didn’t achieve performance as good as the rule-based system, it did establish its own advantages and potential in extracting clinical concepts. Conclusion Our results indicate that a well-designed rule-based clinical NLP system is capable of achieving good performance on cohort selection even with a small training data set. In addition, the investigation of a Unified Medical Language System-based general cNLP system suggests that a hybrid system combining these 2 approaches is promising to surpass the state-of-the-art performance.

scholarly journals Cohort selection for clinical trials using hierarchical neural network

2019 ◽  
Vol 26 (11) ◽  
pp. 1203-1208 ◽  
Author(s):  
Ying Xiong ◽  
Xue Shi ◽  
Shuai Chen ◽  
Dehuan Jiang ◽  
Buzhou Tang ◽  
...  
Keyword(s):  
Neural Network ◽  
Clinical Trials ◽  
Language Processing ◽  
Short Term Memory ◽  
Imbalanced Data ◽  
Cohort Selection ◽  
Highway Network ◽  
Hierarchical Neural Network ◽  
Selection For ◽  
Fully Connected

Abstract Objective Cohort selection for clinical trials is a key step for clinical research. We proposed a hierarchical neural network to determine whether a patient satisfied selection criteria or not. Materials and Methods We designed a hierarchical neural network (denoted as CNN-Highway-LSTM or LSTM-Highway-LSTM) for the track 1 of the national natural language processing (NLP) clinical challenge (n2c2) on cohort selection for clinical trials in 2018. The neural network is composed of 5 components: (1) sentence representation using convolutional neural network (CNN) or long short-term memory (LSTM) network; (2) a highway network to adjust information flow; (3) a self-attention neural network to reweight sentences; (4) document representation using LSTM, which takes sentence representations in chronological order as input; (5) a fully connected neural network to determine whether each criterion is met or not. We compared the proposed method with its variants, including the methods only using the first component to represent documents directly and the fully connected neural network for classification (denoted as CNN-only or LSTM-only) and the methods without using the highway network (denoted as CNN-LSTM or LSTM-LSTM). The performance of all methods was measured by micro-averaged precision, recall, and F1 score. Results The micro-averaged F1 scores of CNN-only, LSTM-only, CNN-LSTM, LSTM-LSTM, CNN-Highway-LSTM, and LSTM-Highway-LSTM were 85.24%, 84.25%, 87.27%, 88.68%, 88.48%, and 90.21%, respectively. The highest micro-averaged F1 score is higher than our submitted 1 of 88.55%, which is 1 of the top-ranked results in the challenge. The results indicate that the proposed method is effective for cohort selection for clinical trials. Discussion Although the proposed method achieved promising results, some mistakes were caused by word ambiguity, negation, number analysis and incomplete dictionary. Moreover, imbalanced data was another challenge that needs to be tackled in the future. Conclusion In this article, we proposed a hierarchical neural network for cohort selection. Experimental results show that this method is good at selecting cohort.

scholarly journals Cohort Selection for Construction of a Clinical Natural Language Processing Corpus

2021 ◽  
pp. 100024
Author(s):  
Naga Lalitha Valli ALLA ◽  
Aipeng CHEN ◽  
Sean BATONGBACAL ◽  
Chandini NEKKANTTI ◽  
Hong-Jie Dai ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Cohort Selection ◽  
Selection For ◽  
Clinical Natural Language Processing

Artificial Intelligence Tool for Optimizing Eligibility Screening for Clinical Trials in a Large Community Cancer Center

2020 ◽  
pp. 50-59 ◽  
Author(s):  
J. Thaddeus Beck ◽  
Melissa Rammage ◽  
Gretchen P. Jackson ◽  
Anita M. Preininger ◽  
Irene Dankwa-Mullan ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Clinical Trial ◽  
Clinical Trials ◽  
Language Processing ◽  
Medical Records ◽  
Cancer Center ◽  
Patients With Cancer ◽  
Attribute Extraction ◽  
Time Required ◽  
Manual Review

PURPOSE Less than 5% of patients with cancer enroll in clinical trials, and 1 in 5 trials are stopped for poor accrual. We evaluated an automated clinical trial matching system that uses natural language processing to extract patient and trial characteristics from unstructured sources and machine learning to match patients to clinical trials. PATIENTS AND METHODS Medical records from 997 patients with breast cancer were assessed for trial eligibility at Highlands Oncology Group between May and August 2016. System and manual attribute extraction and eligibility determinations were compared using the percentage of agreement for 239 patients and 4 trials. Sensitivity and specificity of system-generated eligibility determinations were measured, and the time required for manual review and system-assisted eligibility determinations were compared. RESULTS Agreement between system and manual attribute extraction ranged from 64.3% to 94.0%. Agreement between system and manual eligibility determinations was 81%-96%. System eligibility determinations demonstrated specificities between 76% and 99%, with sensitivities between 91% and 95% for 3 trials and 46.7% for the 4th. Manual eligibility screening of 90 patients for 3 trials took 110 minutes; system-assisted eligibility determinations of the same patients for the same trials required 24 minutes. CONCLUSION In this study, the clinical trial matching system displayed a promising performance in screening patients with breast cancer for trial eligibility. System-assisted trial eligibility determinations were substantially faster than manual review, and the system reliably excluded ineligible patients for all trials and identified eligible patients for most trials.

Trial prospector: An automated clinical trials eligibility matching program.

2013 ◽  
Vol 31 (15_suppl) ◽  
pp. 6538-6538 ◽  
Author(s):  
Andrew J. Parchman ◽  
Guo-Qiang Zhang ◽  
Patrick Mergler ◽  
Jill Barnholtz-Sloan ◽  
Robert Lanese ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Language Processing ◽  
Clinical Data ◽  
Point Of Care ◽  
Effective Means ◽  
Data Entry ◽  
Cancer Center ◽  
Data Systems ◽  
Eligibility Criteria ◽  
Pathologic Diagnosis

6538 Background: Clinical trials are the evidence base for improving cancer treatment. Unfortunately, only approximately 5% of cancer patients (pts) take part in clinical research studies. Even in settings where clinical trials (CTs) are available, pt participation remains low. We hypothesize that the time required to identify appropriate studies for individual pts is a significant barrier to clinical trial accrual. Methods: Our multidisciplinary team developed Trial Prospector (TP), an innovative and flexible computer-based system that utilizes artificial intelligence and natural language processing to automatically extract information (e.g. demographics, pathologic diagnosis, stage, labs) from multiple clinical data systems and then match it to CT eligibility criteria. A user-friendly interface allows physician perusal of relevant CTs and eligibility checklists at the point of care without requiring manual data entry. We pilot tested TP in our cancer center GI oncology subspecialty clinics. TP was deployed for consecutive new pts, and oncologists (oncs) completed surveys after each visit to assess the usability and impact of TP. Results: Eleven medical oncologists (6 attendings and 5 fellows) participated in the pilot study. TP was deployed during 60 new pt visits. Of the 15 relevant GI/phase I CTs, TP identified a mean of 7 ± 2.7 eligible trials per patient. The most common reasons for ineligibility were pathologic diagnosis and labs. CTs were considered by the treating onc for 66.7% of the pts. 95% of participating oncs reviewed the TP output at the point of care with 70% spending 0-5 minutes assessing eligibility. Of the pts considered for CTs, a TP report was reviewed 72.5% of the time. Oncs reported that TP saved time identifying potential CTs during 57.1% of the visits. The reports were manually reviewed, and the TP matching algorithm was 100% accurate. 90.9% of the oncs recommended TP for CT screening. Conclusions: These results indicate that Trial Prospector is a feasible, accurate, and effective means to identify CTs for individual pts during a busy outpatient oncology clinic. Ongoing refinements will expand clinical data extraction and CT warehouse to improve precision and applicability across diseases.

scholarly journals AI4CoV: Matching COVID-19 Patients to Treatment Options Using Artificial Intelligence

2020 ◽  
Author(s):  
Andrew I. Hsu ◽  
Amber S. Yeh ◽  
Shao-Lang Chen ◽  
Jerry J. Yeh ◽  
DongQing Lv ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Language Processing ◽  
Cross Validation ◽  
Early Stage ◽  
Treatment Options ◽  
Clinical Manifestations ◽  
Ground Truth ◽  
English Text ◽  
Eligibility Criteria ◽  
Clinical Notes

AbstractWe developed AI4CoV, a novel AI system to match thousands of COVID-19 clinical trials to patients based on each patient’s eligibility to clinical trials in order to help physicians select treatment options for patients. AI4CoV leveraged Natural Language Processing (NLP) and Machine Learning to parse through eligibility criteria of trials and patients’ clinical manifestations in their clinical notes, both presented in English text, to accomplish 92.76% AUROC on a cross-validation test with 3,156 patient-trial pairs labeled with ground truth of suitability. Our retrospective multiple-site review shows that according to AI4CoV, severe patients of COVID-19 generally have less treatment options suitable for them than mild and moderate patients and that suitable and unsuitable treatment options are different for each patient. Our results show that the general approach of AI4CoV is useful during the early stage of a pandemic when the best treatments are still unknown.

scholarly journals A rule-based approach to identify patient eligibility criteria for clinical trials from narrative longitudinal records

JAMIA Open ◽  
2019 ◽  
Vol 2 (4) ◽  
pp. 521-527
Author(s):  
George Karystianis ◽  
Oscar Florez-Vargas ◽  
Tony Butler ◽  
Goran Nenadic
Keyword(s):  
Clinical Trial ◽  
Clinical Trials ◽  
Eligibility Criteria ◽  
Rule Based ◽  
Time Period ◽  
Mining Methods ◽  
Artery Disease ◽  
Rule Based Approach ◽  
Clinical Records ◽  
Advanced Coronary Artery Disease

Abstract Objective Achieving unbiased recognition of eligible patients for clinical trials from their narrative longitudinal clinical records can be time consuming. We describe and evaluate a knowledge-driven method that identifies whether a patient meets a selected set of 13 eligibility clinical trial criteria from their longitudinal clinical records, which was one of the tasks of the 2018 National NLP Clinical Challenges. Materials and Methods The approach developed uses rules combined with manually crafted dictionaries that characterize the domain. The rules are based on common syntactical patterns observed in text indicating or describing explicitly a criterion. Certain criteria were classified as “met” only when they occurred within a designated time period prior to the most recent narrative of a patient record and were dealt through their position in text. Results The system was applied to an evaluation set of 86 unseen clinical records and achieved a microaverage F1-score of 89.1% (with a micro F1-score of 87.0% and 91.2% for the patients that met and did not meet the criteria, respectively). Most criteria returned reliable results (drug abuse, 92.5%; Hba1c, 91.3%) while few (eg, advanced coronary artery disease, 72.0%; myocardial infarction within 6 months of the most recent narrative, 47.5%) proved challenging enough. Conclusion Overall, the results are encouraging and indicate that automated text mining methods can be used to process clinical records to recognize whether a patient meets a set of clinical trial criteria and could be leveraged to reduce the workload of humans screening patients for trials.

scholarly journals Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

2020 ◽  
Vol 10 (17) ◽  
pp. 5758
Author(s):  
Injy Sarhan ◽  
Marco Spruit
Keyword(s):  
Information Extraction ◽  
Transfer Learning ◽  
Language Processing ◽  
Relation Extraction ◽  
Training Data ◽  
Open Information Extraction ◽  
Comparable Performance ◽  
Medical Domain ◽  
Inductive Transfer Learning ◽  
F Measure

Various tasks in natural language processing (NLP) suffer from lack of labelled training data, which deep neural networks are hungry for. In this paper, we relied upon features learned to generate relation triples from the open information extraction (OIE) task. First, we studied how transferable these features are from one OIE domain to another, such as from a news domain to a bio-medical domain. Second, we analyzed their transferability to a semantically related NLP task, namely, relation extraction (RE). We thereby contribute to answering the question: can OIE help us achieve adequate NLP performance without labelled data? Our results showed comparable performance when using inductive transfer learning in both experiments by relying on a very small amount of the target data, wherein promising results were achieved. When transferring to the OIE bio-medical domain, we achieved an F-measure of 78.0%, only 1% lower when compared to traditional learning. Additionally, transferring to RE using an inductive approach scored an F-measure of 67.2%, which was 3.8% lower than training and testing on the same task. Hereby, our analysis shows that OIE can act as a reliable source task.

scholarly journals A clinical trials corpus annotated with UMLS entities to enhance the access to evidence-based medicine

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Leonardo Campillos-Llanos ◽  
Ana Valverde-Mateos ◽  
Adrián Capllonch-Carrión ◽  
Antonio Moreno-Sandoval
Keyword(s):  
Clinical Trials ◽  
Language Processing ◽  
Evidence Based Medicine ◽  
Relevant Information ◽  
Entity Recognition ◽  
Evidence Based ◽  
Use Case ◽  
Neural Network Models ◽  
Based Medicine ◽  
F Measure

Abstract Background The large volume of medical literature makes it difficult for healthcare professionals to keep abreast of the latest studies that support Evidence-Based Medicine. Natural language processing enhances the access to relevant information, and gold standard corpora are required to improve systems. To contribute with a new dataset for this domain, we collected the Clinical Trials for Evidence-Based Medicine in Spanish (CT-EBM-SP) corpus. Methods We annotated 1200 texts about clinical trials with entities from the Unified Medical Language System semantic groups: anatomy (ANAT), pharmacological and chemical substances (CHEM), pathologies (DISO), and lab tests, diagnostic or therapeutic procedures (PROC). We doubly annotated 10% of the corpus and measured inter-annotator agreement (IAA) using F-measure. As use case, we run medical entity recognition experiments with neural network models. Results This resource contains 500 abstracts of journal articles about clinical trials and 700 announcements of trial protocols (292 173 tokens). We annotated 46 699 entities (13.98% are nested entities). Regarding IAA agreement, we obtained an average F-measure of 85.65% (±4.79, strict match) and 93.94% (±3.31, relaxed match). In the use case experiments, we achieved recognition results ranging from 80.28% (±00.99) to 86.74% (±00.19) of average F-measure. Conclusions Our results show that this resource is adequate for experiments with state-of-the-art approaches to biomedical named entity recognition. It is freely distributed at: http://www.lllf.uam.es/ESP/nlpmedterm_en.html. The methods are generalizable to other languages with similar available sources.

Sign in / Sign up

Export Citation Format

Share Document