Radiology reports automated annotation performance: rule-based machine learning vs deep learning

2021 ◽  
Author(s):  
A. Sahl ◽  
S. Hasan
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Rule Based ◽  
Automated Annotation ◽  
Radiology Reports

scholarly journals Comparison of rule-based and neural network models for negation detection in radiology reports

2020 ◽  
pp. 1-22 ◽  
Author(s):  
D. Sykes ◽  
A. Grivas ◽  
C. Grover ◽  
R. Tobin ◽  
C. Sudlow ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Language Processing ◽  
Network Models ◽  
Neural Network Models ◽  
Test Set ◽  
Rule Based ◽  
Radiology Reports ◽  
The Neural Network ◽  
Negation Detection

Abstract Using natural language processing, it is possible to extract structured information from raw text in the electronic health record (EHR) at reasonably high accuracy. However, the accurate distinction between negated and non-negated mentions of clinical terms remains a challenge. EHR text includes cases where diseases are stated not to be present or only hypothesised, meaning a disease can be mentioned in a report when it is not being reported as present. This makes tasks such as document classification and summarisation more difficult. We have developed the rule-based EdIE-R-Neg, part of an existing text mining pipeline called EdIE-R (Edinburgh Information Extraction for Radiology reports), developed to process brain imaging reports, (https://www.ltg.ed.ac.uk/software/edie-r/) and two machine learning approaches; one using a bidirectional long short-term memory network and another using a feedforward neural network. These were developed on data from the Edinburgh Stroke Study (ESS) and tested on data from routine reports from NHS Tayside (Tayside). Both datasets consist of written reports from medical scans. These models are compared with two existing rule-based models: pyConText (Harkema et al. 2009. Journal of Biomedical Informatics42(5), 839–851), a python implementation of a generalisation of NegEx, and NegBio (Peng et al. 2017. NegBio: A high-performance tool for negation and uncertainty detection in radiology reports. arXiv e-prints, p. arXiv:1712.05898), which identifies negation scopes through patterns applied to a syntactic representation of the sentence. On both the test set of the dataset from which our models were developed, as well as the largely similar Tayside test set, the neural network models and our custom-built rule-based system outperformed the existing methods. EdIE-R-Neg scored highest on F1 score, particularly on the test set of the Tayside dataset, from which no development data were used in these experiments, showing the power of custom-built rule-based systems for negation detection on datasets of this size. The performance gap of the machine learning models to EdIE-R-Neg on the Tayside test set was reduced through adding development Tayside data into the ESS training set, demonstrating the adaptability of the neural network models.

scholarly journals Deep Learning Technique to Predict Heart Disease using IoT Based ECG Data

2019 ◽  
Vol 9 (2) ◽  
pp. 2559-2562
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Deep Learning ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Rule Based ◽  
Learning Techniques ◽  
Learning Technique ◽  
Logistic Regression Algorithm ◽  
Target Output

Scientific Knowledge and Electronic devices are growing day by day. In this aspect, many expert systems are involved in the healthcare industry using machine learning algorithms. Deep neural networks beat the machine learning techniques and often take raw data i.e., unrefined data to calculate the target output. Deep learning or feature learning is used to focus on features which is very important and gives a complete understanding of the model generated. Existing methodology used data mining technique like rule based classification algorithm and machine learning algorithm like hybrid logistic regression algorithm to preprocess data and extract meaningful insights of data. This is, however a supervised data. The proposed work is based on unsupervised data that is there is no labelled data and deep neural techniques is deployed to get the target output. Machine learning algorithms are compared with proposed deep learning techniques using TensorFlow and Keras in the aspect of accuracy. Deep learning methodology outfits the existing rule based classification and hybrid logistic regression algorithm in terms of accuracy. The designed methodology is tested on the public MIT-BIH arrhythmia database, classifying four kinds of abnormal beats. The proposed approach based on deep learning technique offered a better performance, improving the results when compared to machine learning approaches of the state-of-the-art

scholarly journals Evaluating shallow and deep learning strategies for the 2018 n2c2 shared task on clinical text classification

2019 ◽  
Vol 26 (11) ◽  
pp. 1247-1254 ◽  
Author(s):  
Michel Oleynik ◽  
Amila Kugic ◽  
Zdenko Kasáč ◽  
Markus Kreuzthaler
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Learning Strategies ◽  
Short Term Memory ◽  
Shared Task ◽  
Short Term ◽  
Rule Based ◽  
Long Short Term Memory ◽  
Rule Based Classifier ◽  
Clinical Phenotyping

Abstract Objective Automated clinical phenotyping is challenging because word-based features quickly turn it into a high-dimensional problem, in which the small, privacy-restricted, training datasets might lead to overfitting. Pretrained embeddings might solve this issue by reusing input representation schemes trained on a larger dataset. We sought to evaluate shallow and deep learning text classifiers and the impact of pretrained embeddings in a small clinical dataset. Materials and Methods We participated in the 2018 National NLP Clinical Challenges (n2c2) Shared Task on cohort selection and received an annotated dataset with medical narratives of 202 patients for multilabel binary text classification. We set our baseline to a majority classifier, to which we compared a rule-based classifier and orthogonal machine learning strategies: support vector machines, logistic regression, and long short-term memory neural networks. We evaluated logistic regression and long short-term memory using both self-trained and pretrained BioWordVec word embeddings as input representation schemes. Results Rule-based classifier showed the highest overall micro F1 score (0.9100), with which we finished first in the challenge. Shallow machine learning strategies showed lower overall micro F1 scores, but still higher than deep learning strategies and the baseline. We could not show a difference in classification efficiency between self-trained and pretrained embeddings. Discussion Clinical context, negation, and value-based criteria hindered shallow machine learning approaches, while deep learning strategies could not capture the term diversity due to the small training dataset. Conclusion Shallow methods for clinical phenotyping can still outperform deep learning methods in small imbalanced data, even when supported by pretrained embeddings.

scholarly journals Identifying Diagnosis Evidence of Liver Cancer in Chinese Radiology Reports Using BERT-based Deep Learning Method (Preprint)

2020 ◽  
Author(s):  
Hui Chen ◽  
Honglei Liu ◽  
Ni Wang ◽  
Yanqun Huang ◽  
Zhiqiang Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Liver Cancer ◽  
Language Processing ◽  
Cancer Diagnosis ◽  
Data Driven ◽  
Learning Method ◽  
Rule Based ◽  
Radiological Features ◽  
Radiology Reports ◽  
Computer Aided

BACKGROUND Liver cancer remains to be a substantial disease burden in China. As one of the primary diagnostic means for liver cancer, the dynamic enhanced computed tomography (CT) scan provides detailed diagnosis evidence that is recorded in the free-text radiology reports. OBJECTIVE In this study, we combined knowledge-driven deep learning methods and data-driven natural language processing (NLP) methods to extract the radiological features from these reports, and designed a computer-aided liver cancer diagnosis framework.In this study, we combined knowledge-driven deep learning methods and data-driven natural language processing (NLP) methods to extract the radiological features from these reports, and designed a computer-aided liver cancer diagnosis framework. METHODS We collected 1089 CT radiology reports in Chinese. We proposed a pre-trained fine-tuning BERT (Bidirectional Encoder Representations from Transformers) language model for word embedding. The embedding served as the inputs for BiLSTM (Bidirectional Long Short-Term Memory) and CRF (Conditional Random Field) model (BERT-BiLSTM-CRF) to extract features of hyperintense enhancement in the arterial phase (APHE) and hypointense in the portal and delayed phases (PDPH). Furthermore, we also extracted features using the traditional rule-based NLP method based on the content of radiology reports. We then applied random forest for liver cancer diagnosis and calculated the Gini impurity for the identification of diagnosis evidence. RESULTS The BERT-BiLSTM-CRF predicted the features of APHE and PDPH with an F1 score of 98.40% and 90.67%, respectively. The prediction model using combined features had a higher performance (F1 score, 88.55%) than those using the single kind of features obtained by BERT-BiLSTM-CRF (84.88%) or traditional rule-based NLP method (83.52%). The features of APHE and PDPH were the top two essential features for the liver cancer diagnosis. CONCLUSIONS We proposed a BERT-based deep learning method for diagnosis evidence extraction based on clinical knowledge. With the recognized features of APHE and PDPH, the liver cancer diagnosis could get a high performance, which was further increased by combining with the radiological features obtained by the traditional rule-based NLP method. The BERT-BiLSTM-CRF had achieved the state-of-the-art performance in this study, which could be extended to other kinds of Chinese clinical texts. CLINICALTRIAL None

scholarly journals A Demand-Side Load Event Detection Algorithm Based on Wide-Deep Neural Networks and Randomized Sparse Backpropagation

2021 ◽  
Vol 9 ◽  
Author(s):  
Chen Li ◽  
Gaoqi Liang ◽  
Huan Zhao ◽  
Guo Chen
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Event Detection ◽  
Detection Algorithm ◽  
Demand Side ◽  
Rule Based ◽  
Detection Algorithms ◽  
Deep Model ◽  
Conventional Machine

Event detection is an important application in demand-side management. Precise event detection algorithms can improve the accuracy of non-intrusive load monitoring (NILM) and energy disaggregation models. Existing event detection algorithms can be divided into four categories: rule-based, statistics-based, conventional machine learning, and deep learning. The rule-based approach entails hand-crafted feature engineering and carefully calibrated thresholds; the accuracies of statistics-based and conventional machine learning methods are inferior to the deep learning algorithms due to their limited ability to extract complex features. Deep learning models require a long training time and are hard to interpret. This paper proposes a novel algorithm for load event detection in smart homes based on wide and deep learning that combines the convolutional neural network (CNN) and the soft-max regression (SMR). The deep model extracts the power time series patterns and the wide model utilizes the percentile information of the power time series. A randomized sparse backpropagation (RSB) algorithm for weight filters is proposed to improve the robustness of the standard wide-deep model. Compared to the standard wide-deep, pure CNN, and SMR models, the hybrid wide-deep model powered by RSB demonstrates its superiority in terms of accuracy, convergence speed, and robustness.

scholarly journals A Comparison of Machine Learning and Deep Learning Methods with Rule Based Features for Mixed Emotion Analysis

2021 ◽  
Vol 14 (1) ◽  
pp. 42-53
Author(s):  
Christy Daniel ◽  
◽  
Shyamala Loganathan ◽  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Learning Model ◽  
Support Vector ◽  
Rule Based ◽  
Mixed Emotions ◽  
Abstract Noun ◽  
Emotion Words ◽  
Complex Sentences ◽  
Abstract Nouns

Multi-class classification of sentiments from text data still remains a challenging task to detect the sentiments hidden behind the sentences because of the probable existence of multiple meanings for some of the texts in the dataset. To overcome this, the proposed rule based modified Convolutional neural network-Global Vectors (RCNN-GloVe) and rule-based modified Support Vector Machine - Global Vectors (RSVM-GloVe) were developed for classifying the twitter complex sentences at twelve different levels focusing on mixed emotions by targeting abstract nouns and adjective emotion words. To execute this, three proposed algorithms were developed such as the optimized abstract noun algorithm (OABNA) to identify the abstract noun emotion words, optimized complex sentences algorithm (OCSA) to extract all the complex sentences in a tweet precisely and adjective searching algorithm (ADJSA) to retrieve all the sentences with adjectives. The results of this study indicates that our proposed RCNNGloVe method used in the sentiment analysis was able to classify the mixed emotions accurately from the twitter dataset with the highest accuracy level of 92.02% in abstract nouns and 88.93% in adjectives. It is distinctly evident from the research that the proposed deep learning model (RCNN-GloVe) had an edge over the machine learning model (RSVM-GloVe).

scholarly journals Machine learning-based identification and rule-based normalization of adverse drug reactions in drug labels

2019 ◽  
Vol 20 (S21) ◽  
Author(s):  
Mert Tiftikci ◽  
Arzucan Özgür ◽  
Yongqun He ◽  
Junguk Hur
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Adverse Drug Reactions ◽  
Safety Concern ◽  
Challenge Test ◽  
Entity Recognition ◽  
Drug Reactions ◽  
Data Set ◽  
Rule Based ◽  
Rule Based System

Abstract Background Use of medication can cause adverse drug reactions (ADRs), unwanted or unexpected events, which are a major safety concern. Drug labels, or prescribing information or package inserts, describe ADRs. Therefore, systematically identifying ADR information from drug labels is critical in multiple aspects; however, this task is challenging due to the nature of the natural language of drug labels. Results In this paper, we present a machine learning- and rule-based system for the identification of ADR entity mentions in the text of drug labels and their normalization through the Medical Dictionary for Regulatory Activities (MedDRA) dictionary. The machine learning approach is based on a recently proposed deep learning architecture, which integrates bi-directional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network (CNN), and Conditional Random Fields (CRF) for entity recognition. The rule-based approach, used for normalizing the identified ADR mentions to MedDRA terms, is based on an extension of our in-house text-mining system, SciMiner. We evaluated our system on the Text Analysis Conference (TAC) Adverse Drug Reaction 2017 challenge test data set, consisting of 200 manually curated US FDA drug labels. Our ML-based system achieved 77.0% F1 score on the task of ADR mention recognition and 82.6% micro-averaged F1 score on the task of ADR normalization, while rule-based system achieved 67.4 and 77.6% F1 scores, respectively. Conclusion Our study demonstrates that a system composed of a deep learning architecture for entity recognition and a rule-based model for entity normalization is a promising approach for ADR extraction from drug labels.

scholarly journals Fast Machine Learning Annotation in the Medical Domain: A Semi-Automated Video Annotation Tool for Gastroenterologists

2021 ◽  
Author(s):  
Adrian Krenzer ◽  
Kevin Makowski ◽  
Amar Hekalo ◽  
Daniel Fitting ◽  
Joel Troya ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
State Of The Art ◽  
Video Annotation ◽  
Successful Implementation ◽  
Annotation Tool ◽  
Domain Expert ◽  
Domain Experts ◽  
Automated Annotation ◽  
Medical Domain

Abstract Background: Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all of the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g. visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between. Results: Using this framework we were able to reduce work load of domain experts on average by a factor of 20. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated pre-annotation model enhances the annotation speed further. Through a study with 10 participants we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool. Conclusion: In summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source.

Sign in / Sign up

Export Citation Format

Share Document