Hierarchical Relation Extraction Based On Bidirectional Long Short-Term Memory Networks

AbstractObjectiveIdentification of drugs, associated medication entities, and interactions among them are crucial to prevent unwanted effects of drug therapy, known as adverse drug events. This article describes our participation to the n2c2 shared-task in extracting relations between medication-related entities in electronic health records.Materials and MethodsWe proposed an ensemble approach for relation extraction and classification between drugs and medication-related entities. We incorporated state-of-the-art named-entity recognition (NER) models based on bidirectional long short-term memory (BiLSTM) networks and conditional random fields (CRF) for end-to-end extraction. We additionally developed separate models for intra- and inter-sentence relation extraction and combined them using an ensemble method. The intra-sentence models rely on bidirectional long short-term memory networks and attention mechanisms and are able to capture dependencies between multiple related pairs in the same sentence. For the inter-sentence relations, we adopted a neural architecture that utilizes the Transformer network to improve performance in longer sequences.ResultsOur team ranked third with a micro-averaged F1 score of 94.72% and 87.65% for relation and end-to-end relation extraction, respectively (Tracks 2 and 3). Our ensemble effectively takes advantages from our proposed models. Analysis of the reported results indicated that our proposed approach is more generalizable than the top-performing system, which employs additional training data- and corpus-driven processing techniques.ConclusionsWe proposed a relation extraction system to identify relations between drugs and medication-related entities. The proposed approach is independent of external syntactic tools. Analysis showed that by using latent Drug-Drug interactions we were able to significantly improve the performance of non–Drug-Drug pairs in EHRs.

Download Full-text

A Customized Attention-Based Long Short-Term Memory Network for Distant Supervised Relation Extraction

Neural Computation ◽

10.1162/neco_a_00970 ◽

2017 ◽

Vol 29 (7) ◽

pp. 1964-1985 ◽

Cited By ~ 1

Author(s):

Dengchao He ◽

Hongjun Zhang ◽

Wenning Hao ◽

Rui Zhang ◽

Kai Cheng

Keyword(s):

False Positive ◽

Short Term Memory ◽

Relation Extraction ◽

Short Term ◽

Training Corpus ◽

Term Memory ◽

Distant Supervision ◽

Positive Data ◽

Memory Network ◽

Long Short Term Memory

Distant supervision, a widely applied approach in the field of relation extraction can automatically generate large amounts of labeled training corpus with minimal manual effort. However, the labeled training corpus may have many false-positive data, which would hurt the performance of relation extraction. Moreover, in traditional feature-based distant supervised approaches, extraction models adopt human design features with natural language processing. It may also cause poor performance. To address these two shortcomings, we propose a customized attention-based long short-term memory network. Our approach adopts word-level attention to achieve better data representation for relation extraction without manually designed features to perform distant supervision instead of fully supervised relation extraction, and it utilizes instance-level attention to tackle the problem of false-positive data. Experimental results demonstrate that our proposed approach is effective and achieves better performance than traditional methods.

Download Full-text

Extracting entities with attributes in clinical text via joint deep learning

Journal of the American Medical Informatics Association ◽

10.1093/jamia/ocz158 ◽

2019 ◽

Vol 26 (12) ◽

pp. 1584-1591 ◽

Cited By ~ 1

Author(s):

Xue Shi ◽

Yingping Yi ◽

Ying Xiong ◽

Buzhou Tang ◽

Qingcai Chen ◽

...

Keyword(s):

Deep Learning ◽

Language Processing ◽

Short Term Memory ◽

Conditional Random Field ◽

Relation Extraction ◽

Entity Recognition ◽

Short Term ◽

Term Memory ◽

Clinical Text ◽

Long Short Term Memory

Abstract Objective Extracting clinical entities and their attributes is a fundamental task of natural language processing (NLP) in the medical domain. This task is typically recognized as 2 sequential subtasks in a pipeline, clinical entity or attribute recognition followed by entity-attribute relation extraction. One problem of pipeline methods is that errors from entity recognition are unavoidably passed to relation extraction. We propose a novel joint deep learning method to recognize clinical entities or attributes and extract entity-attribute relations simultaneously. Materials and Methods The proposed method integrates 2 state-of-the-art methods for named entity recognition and relation extraction, namely bidirectional long short-term memory with conditional random field and bidirectional long short-term memory, into a unified framework. In this method, relation constraints between clinical entities and attributes and weights of the 2 subtasks are also considered simultaneously. We compare the method with other related methods (ie, pipeline methods and other joint deep learning methods) on an existing English corpus from SemEval-2015 and a newly developed Chinese corpus. Results Our proposed method achieves the best F1 of 74.46% on entity recognition and the best F1 of 50.21% on relation extraction on the English corpus, and 89.32% and 88.13% on the Chinese corpora, respectively, which outperform the other methods on both tasks. Conclusions The joint deep learning–based method could improve both entity recognition and relation extraction from clinical text in both English and Chinese, indicating that the approach is promising.

Download Full-text

Discovering microbe-disease associations from the literature using a hierarchical long short-term memory network and an ensemble parser model

Scientific Reports ◽

10.1038/s41598-021-83966-8 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Yesol Park ◽

Joohong Lee ◽

Heesang Moon ◽

Yong Suk Choi ◽

Mina Rho

Keyword(s):

Language Processing ◽

Large Scale ◽

Short Term Memory ◽

Relation Extraction ◽

Parse Tree ◽

Short Term ◽

Term Memory ◽

Disease Associations ◽

Memory Network ◽

Long Short Term Memory

AbstractWith recent advances in biotechnology and sequencing technology, the microbial community has been intensively studied and discovered to be associated with many chronic as well as acute diseases. Even though a tremendous number of studies describing the association between microbes and diseases have been published, text mining methods that focus on such associations have been rarely studied. We propose a framework that combines machine learning and natural language processing methods to analyze the association between microbes and diseases. A hierarchical long short-term memory network was used to detect sentences that describe the association. For the sentences determined, two different parse tree-based search methods were combined to find the relation-describing word. The ensemble model of constituency parsing for structural pattern matching and dependency-based relation extraction improved the prediction accuracy. By combining deep learning and parse tree-based extractions, our proposed framework could extract the microbe-disease association with higher accuracy. The evaluation results showed that our system achieved an F-score of 0.8764 and 0.8524 in binary decisions and extracting relation words, respectively. As a case study, we performed a large-scale analysis of the association between microbes and diseases. Additionally, a set of common microbes shared by multiple diseases were also identified in this study. This study could provide valuable information for the major microbes that were studied for a specific disease. The code and data are available at https://github.com/DMnBI/mdi_predictor.

Download Full-text