Neural translation and automated recognition of ICD-10 medical entities from natural language (Preprint)

2020 ◽  
Author(s):  
Louis Falissard ◽  
Claire Morgand ◽  
Walid Ghosn ◽  
Claire Imbaud ◽  
Karim Bounebache ◽  
...  
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
State Of The Art ◽  
Entity Recognition ◽  
Limiting Factor ◽  
Current State ◽  
Automated Coding ◽  
Icd 10 ◽  
Modelling Problem ◽  
F Measure

BACKGROUND The recognition of medical entities from natural language is an ubiquitous problem in the medical field, with applications ranging from medical act coding to the analysis of electronic health data for public health. It is however a complex task usually requiring human expert intervention, thus making it expansive and time consuming. The recent advances in artificial intelligence, specifically the raise of deep learning methods, has enabled computers to make efficient decisions on a number of complex problems, with the notable example of neural sequence models and their powerful applications in natural language processing. They however require a considerable amount of data to learn from, which is typically their main limiting factor. However, the CépiDc stores an exhaustive database of death certificates at the French national scale, amounting to several millions of natural language examples provided with their associated human coded medical entities available to the machine learning practitioner. OBJECTIVE This article investigates the applications of deep neural sequence models to the medical entity recognition from natural language problem. METHODS The investigated dataset is based on every French death certificate from 2011 to 2016, containing information such as the subject’s age, gender, and the chain of events leading to his or her death both in French and encoded as ICD-10 medical entities, for a total of around 3 million observations. The task of automatically recognizing ICD-10 medical entities from the French natural language based chain of event is then formulated as a type of predictive modelling problem known as a sequence-to-sequence modelling problem. A deep neural network based model known as the Transformer is then slightly adapted and fit to the dataset. Its performance is then assessed on an exterior dataset and compared to the current state of the art. Confidence intervals for derived measurements are derived via bootstrap. RESULTS The proposed approach resulted in a test F-measure of .952 [.946, .957], which constitutes a significant improvement on the current state of the art and its previously reported 82.5 F-measure assessed on a comparable dataset. Such an improvement opens a whole field of new applications, from nosologist level automated coding to temporal harmonization of death statistics. CONCLUSIONS This article shows that deep artificial neural network can directly learn from voluminous datasets complex relationships between natural language and medical entities, without any explicit prior knowledge. Although not entirely free from mistakes, the derived model constitutes a powerful tool for automated coding of medical entities from medical language with promising potential applications.

scholarly journals Automatic ICD-10 Coding and Training System: Deep Neural Network Based on Supervised Learning

10.2196/23230 ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. e23230
Author(s):  
Pei-Fu Chen ◽  
Ssu-Ming Wang ◽  
Wei-Chih Liao ◽  
Lu-Cheng Kuo ◽  
Kuan-Chih Chen ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Deep Neural Network ◽  
University Hospital ◽  
Classification Model ◽  
Icd 10 ◽  
And Training

Background The International Classification of Diseases (ICD) code is widely used as the reference in medical system and billing purposes. However, classifying diseases into ICD codes still mainly relies on humans reading a large amount of written material as the basis for coding. Coding is both laborious and time-consuming. Since the conversion of ICD-9 to ICD-10, the coding task became much more complicated, and deep learning– and natural language processing–related approaches have been studied to assist disease coders. Objective This paper aims at constructing a deep learning model for ICD-10 coding, where the model is meant to automatically determine the corresponding diagnosis and procedure codes based solely on free-text medical notes to improve accuracy and reduce human effort. Methods We used diagnosis records of the National Taiwan University Hospital as resources and apply natural language processing techniques, including global vectors, word to vectors, embeddings from language models, bidirectional encoder representations from transformers, and single head attention recurrent neural network, on the deep neural network architecture to implement ICD-10 auto-coding. Besides, we introduced the attention mechanism into the classification model to extract the keywords from diagnoses and visualize the coding reference for training freshmen in ICD-10. Sixty discharge notes were randomly selected to examine the change in the F1-score and the coding time by coders before and after using our model. Results In experiments on the medical data set of National Taiwan University Hospital, our prediction results revealed F1-scores of 0.715 and 0.618 for the ICD-10 Clinical Modification code and Procedure Coding System code, respectively, with a bidirectional encoder representations from transformers embedding approach in the Gated Recurrent Unit classification model. The well-trained models were applied on the ICD-10 web service for coding and training to ICD-10 users. With this service, coders can code with the F1-score significantly increased from a median of 0.832 to 0.922 (P<.05), but not in a reduced interval. Conclusions The proposed model significantly improved the F1-score but did not decrease the time consumed in coding by disease coders.

scholarly journals Deep Transfer Learning for Source Code Modeling

2020 ◽  
Vol 30 (05) ◽  
pp. 649-668
Author(s):  
Yasir Hussain ◽  
Zhiqiu Huang ◽  
Yu Zhou ◽  
Senzhang Wang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Transfer Learning ◽  
Related Problem ◽  
State Of The Art ◽  
Source Code ◽  
Traditional Learning ◽  
Learning Models ◽  
Modeling And Analysis ◽  
F Measure

In recent years, deep learning models have shown great potential in source code modeling and analysis. Generally, deep learning-based approaches are problem-specific and data-hungry. A challenging issue of these approaches is that they require training from scratch for a different related problem. In this work, we propose a transfer learning-based approach that significantly improves the performance of deep learning-based source code models. In contrast to traditional learning paradigms, transfer learning can transfer the knowledge learned in solving one problem into another related problem. First, we present two recurrent neural network-based models RNN and GRU for the purpose of transfer learning in the domain of source code modeling. Next, via transfer learning, these pre-trained (RNN and GRU) models are used as feature extractors. Then, these extracted features are combined into attention learner for different downstream tasks. The attention learner leverages from the learned knowledge of pre-trained models and fine-tunes them for a specific downstream task. We evaluate the performance of the proposed approach with extensive experiments with the source code suggestion task. The results indicate that the proposed approach outperforms the state-of-the-art models in terms of accuracy, precision, recall and F-measure without training the models from scratch.

scholarly journals Regularized Evolution for Image Classifier Architecture Search

2019 ◽  
Vol 33 ◽  
pp. 4780-4789 ◽  
Author(s):  
Esteban Real ◽  
Alok Aggarwal ◽  
Yanping Huang ◽  
Quoc V. Le
Keyword(s):  
Neural Network ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Simple Method ◽  
Current State ◽  
Network Topologies ◽  
Comparable Accuracy ◽  
Tournament Selection ◽  
Complex Architecture ◽  
First Time

The effort devoted to hand-crafting neural network image classifiers has motivated the use of architecture search to discover them automatically. Although evolutionary algorithms have been repeatedly applied to neural network topologies, the image classifiers thus discovered have remained inferior to human-crafted ones. Here, we evolve an image classifier— AmoebaNet-A—that surpasses hand-designs for the first time. To do this, we modify the tournament selection evolutionary algorithm by introducing an age property to favor the younger genotypes. Matching size, AmoebaNet-A has comparable accuracy to current state-of-the-art ImageNet models discovered with more complex architecture-search methods. Scaled to larger size, AmoebaNet-A sets a new state-of-theart 83.9% top-1 / 96.6% top-5 ImageNet accuracy. In a controlled comparison against a well known reinforcement learning algorithm, we give evidence that evolution can obtain results faster with the same hardware, especially at the earlier stages of the search. This is relevant when fewer compute resources are available. Evolution is, thus, a simple method to effectively discover high-quality architectures.

Named entity recognition for Polish

2019 ◽  
Vol 55 (2) ◽  
pp. 239-269
Author(s):  
Michał Marcińczuk ◽  
Aleksander Wawer
Keyword(s):  
Open Source ◽  
State Of The Art ◽  
Proper Names ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Coarse Grained ◽  
Named Entity ◽  
Current State ◽  
Annotated Corpora ◽  
Available Resources

Abstract In this article we discuss the current state-of-the-art for named entity recognition for Polish. We present publicly available resources and open-source tools for named entity recognition. The overview includes various kind of resources, i.e. guidelines, annotated corpora (NKJP, KPWr, CEN, PST) and lexicons (NELexiconS, PNET, Gazetteer). We present the major NER tools for Polish (Sprout, NERF, Liner2, Parallel LSTM-CRFs and PolDeepNer) and discuss their performance on the reference datasets. In the article we cover identification of named entity mentions in the running text, local and global entity categorization, fine- and coarse-grained categorization and lemmatization of proper names.

scholarly journals Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

2018 ◽  
Vol 61 ◽  
pp. 65-170 ◽  
Author(s):  
Albert Gatt ◽  
Emiel Krahmer
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
Natural Language Generation ◽  
Data Driven ◽  
Research Topics ◽  
Language Generation ◽  
The Past ◽  
Current State ◽  
Linguistic Input ◽  
New Applications

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past two decades, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of NLP, with an emphasis on different evaluation methods and the relationships between them.

scholarly journals InferNER: an attentive model leveraging the sentence-level information for Named Entity Recognition in Microblogs

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Moemmur Shahzad ◽  
Ayesha Amin ◽  
Diego Esteves ◽  
Axel-Cyrille Ngonga Ngomo
Keyword(s):  
Social Media ◽  
Visual Information ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Current State ◽  
Sentence Level ◽  
Level Information ◽  
External Sources

We investigate the problem of named entity recognition in the user-generated text such as social media posts. This task is rendered particularly difficult by the restricted length and limited grammatical coherence of this data type. Current state-of-the-art approaches rely on external sources such as gazetteers to alleviate some of these restrictions. We present a neural model able to outperform state of the art on this task without recurring to gazetteers or similar external sources of information. Our approach relies on word-, character-, and sentence-level information for NER in short-text. Social media posts like tweets often have associated images that may provide auxiliary context relevant to understand these texts. Hence, we also incorporate visual information and introduce an attention component which computes attention weight probabilities over textual and text-relevant visual contexts separately. Our model outperforms the current state of the art on various NER datasets. On WNUT 2016 and 2017, our model achieved 53.48\% and 50.52\% F1 score, respectively. With Multimodal model, our system also outperforms the current SOTA with an F1 score of 74\% on the multimodal dataset. Our evaluation further suggests that our model also goes beyond the current state-of-the-art on newswire data, hence corroborating its suitability for various NER tasks.

scholarly journals Self-normalizing learning on biomedical ontologies using a deep Siamese neural network

2020 ◽  
Author(s):  
Fatima Zohra Smaili ◽  
Xin Gao ◽  
Robert Hoehndorf
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Language Processing ◽  
Research Group ◽  
Prediction Method ◽  
Entity Recognition ◽  
Superior Performance ◽  
Biomedical Ontologies ◽  
Sources Of Information ◽  
Structured Information

AbstractMotivationOntologies are widely used in biomedicine for the annotation and standardization of data. One of the main roles of ontologies is to provide structured background knowledge within a domain as well as a set of labels, synonyms, and definitions for the classes within a domain. The two types of information provided by ontologies have been extensively exploited in natural language processing and machine learning applications. However, they are commonly used separately, and thus it is unknown if joining the two sources of information can further benefit data analysis tasks.ResultsWe developed a novel method that applies named entity recognition and normalization methods on texts to connect the structured information in biomedical ontologies with the information contained in natural language. We apply this normalization both to literature and to the natural language information contained within ontologies themselves. The normalized ontologies and text are then used to generate embeddings, and relations between entities are predicted using a deep Siamese neural network model that takes these embeddings as input. We demonstrate that our novel embedding and prediction method using self-normalized biomedical ontologies significantly outperforms the state-of-the-art methods in embedding ontologies on two benchmark tasks: prediction of interactions between proteins and prediction of gene–disease associations. Our method also allows us to apply ontology-based annotations and axioms to the prediction of toxicological effects of chemicals where our method shows superior performance. Our method is generic and can be applied in scenarios where ontologies consisting of both structured information and natural language labels or synonyms are used.Availabilityhttps://github.com/bio-ontology-research-group/[email protected] and [email protected]

scholarly journals Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

2018 ◽  
Vol 232 ◽  
pp. 01061
Author(s):  
Danhua Li ◽  
Xiaofeng Di ◽  
Xuan Qu ◽  
Yunfei Zhao ◽  
Honggang Kong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Deep Convolutional Neural Network ◽  
High Quality ◽  
Low Level ◽  
Features Fusion ◽  
Current State ◽  
High Level

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

scholarly journals Natural language interfaces to databases – an introduction

1995 ◽  
Vol 1 (1) ◽  
pp. 29-81 ◽  
Author(s):  
I. Androutsopoulos ◽  
G.D. Ritchie ◽  
P. Thanisch
Keyword(s):  
Natural Language ◽  
Computational Linguistics ◽  
State Of The Art ◽  
Query Languages ◽  
Natural Language Interfaces ◽  
Advantages And Disadvantages ◽  
Current State ◽  
Database Updates ◽  
Graphical Interfaces ◽  
History Of

AbstractThis paper is an introduction to natural language interfaces to databases (NLIDBS). A brief overview of the history of NLIDBS is first given. Some advantages and disadvantages of NLIDBS are then discussed, comparing NLIDBS to formal query languages, form-based interfaces, and graphical interfaces. An introduction to some of the linguistic problems NLIDBS have to confront follows, for the benefit of readers less familiar with computational linguistics. The discussion then moves on to NLIDB architectures, portability issues, restricted natural language input systems (including menu-based NLIDBS), and NLIDBS with reasoning capabilities. Some less explored areas of NLIDB research are then presented, namely database updates, meta-knowledge questions, temporal questions, and multi-modal NLIDBS. The paper ends with reflections on the current state of the art.

scholarly journals Analyzing Depthwise Convolution Based Neural Network: Study Case in Ship Detection and Land Cover Classification

2019 ◽  
Vol 12 (2) ◽  
pp. 103
Author(s):  
Kuntoro Adi Nugroho ◽  
Yudi Eko Windarto
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
State Of The Art ◽  
Learning Algorithms ◽  
Satellite Image ◽  
Feature Learning ◽  
Feature Extraction Method ◽  
Current State ◽  
Study Case ◽  
Unsupervised Feature Extraction

Various methods are available to perform feature extraction on satellite images. Among the available alternatives, deep convolutional neural network (ConvNet) is the state of the art method. Although previous studies have reported successful attempts on developing and implementing ConvNet on remote sensing application, several issues are not well explored, such as the use of depthwise convolution, final pooling layer size, and comparison between grayscale and RGB settings. The objective of this study is to perform analysis to address these issues. Two feature learning algorithms were proposed, namely ConvNet as the current state of the art for satellite image classification and Gray Level Co-occurence Matrix (GLCM) which represents a classic unsupervised feature extraction method. The experiment demonstrated consistent result with previous studies that ConvNet is superior in most cases compared to GLCM, especially with 3x3xn final pooling. The performance of the learning algorithms are much higher on features from RGB channels, except for ConvNet with relatively small number of features.

Sign in / Sign up

Export Citation Format

Share Document