scholarly journals Document-Level Biomedical Relation Extraction Leveraging Pretrained Self-Attention Structure and Entity Replacement: Algorithm and Pretreatment Method Validation Study

10.2196/17644 ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. e17644
Author(s):  
Xiaofeng Liu ◽  
Jianye Fan ◽  
Shoubin Dong
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Biomedical Literature ◽  
Long Distance ◽  
Replacement Method ◽  
Replacement Algorithm ◽  
Additional Noise ◽  
The Relationship ◽  
Document Level ◽  
Biomedical Relation Extraction

Background The most current methods applied for intrasentence relation extraction in the biomedical literature are inadequate for document-level relation extraction, in which the relationship may cross sentence boundaries. Hence, some approaches have been proposed to extract relations by splitting the document-level datasets through heuristic rules and learning methods. However, these approaches may introduce additional noise and do not really solve the problem of intersentence relation extraction. It is challenging to avoid noise and extract cross-sentence relations. Objective This study aimed to avoid errors by dividing the document-level dataset, verify that a self-attention structure can extract biomedical relations in a document with long-distance dependencies and complex semantics, and discuss the relative benefits of different entity pretreatment methods for biomedical relation extraction. Methods This paper proposes a new data preprocessing method and attempts to apply a pretrained self-attention structure for document biomedical relation extraction with an entity replacement method to capture very long-distance dependencies and complex semantics. Results Compared with state-of-the-art approaches, our method greatly improved the precision. The results show that our approach increases the F1 value, compared with state-of-the-art methods. Through experiments of biomedical entity pretreatments, we found that a model using an entity replacement method can improve performance. Conclusions When considering all target entity pairs as a whole in the document-level dataset, a pretrained self-attention structure is suitable to capture very long-distance dependencies and learn the textual context and complicated semantics. A replacement method for biomedical entities is conducive to biomedical relation extraction, especially to document-level relation extraction.

scholarly journals Document-Level Biomedical Relation Extraction Leveraging Pretrained Self-Attention Structure and Entity Replacement: Algorithm and Pretreatment Method Validation Study (Preprint)

2019 ◽  
Author(s):  
Xiaofeng Liu ◽  
Jianye Fan ◽  
Shoubin Dong
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Biomedical Literature ◽  
Long Distance ◽  
Replacement Method ◽  
Replacement Algorithm ◽  
Additional Noise ◽  
The Relationship ◽  
Document Level ◽  
Biomedical Relation Extraction

BACKGROUND The most current methods applied for intrasentence relation extraction in the biomedical literature are inadequate for document-level relation extraction, in which the relationship may cross sentence boundaries. Hence, some approaches have been proposed to extract relations by splitting the document-level datasets through heuristic rules and learning methods. However, these approaches may introduce additional noise and do not really solve the problem of intersentence relation extraction. It is challenging to avoid noise and extract cross-sentence relations. OBJECTIVE This study aimed to avoid errors by dividing the document-level dataset, verify that a self-attention structure can extract biomedical relations in a document with long-distance dependencies and complex semantics, and discuss the relative benefits of different entity pretreatment methods for biomedical relation extraction. METHODS This paper proposes a new data preprocessing method and attempts to apply a pretrained self-attention structure for document biomedical relation extraction with an entity replacement method to capture very long-distance dependencies and complex semantics. RESULTS Compared with state-of-the-art approaches, our method greatly improved the precision. The results show that our approach increases the F1 value, compared with state-of-the-art methods. Through experiments of biomedical entity pretreatments, we found that a model using an entity replacement method can improve performance. CONCLUSIONS When considering all target entity pairs as a whole in the document-level dataset, a pretrained self-attention structure is suitable to capture very long-distance dependencies and learn the textual context and complicated semantics. A replacement method for biomedical entities is conducive to biomedical relation extraction, especially to document-level relation extraction.

scholarly journals Full-Abstract Biomedical Relation Extraction with Keyword-Attentive Domain Knowledge Infusion

2021 ◽  
Vol 11 (16) ◽  
pp. 7318
Author(s):  
Xian Zhu ◽  
Lele Zhang ◽  
Jiangnan Du ◽  
Zhifeng Xiao
Keyword(s):  
Language Processing ◽  
Domain Knowledge ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Relation Extraction ◽  
Text Representation ◽  
Upper Level ◽  
Document Level ◽  
Semantic Interaction ◽  
Biomedical Relation Extraction

Relation extraction (RE) is an essential task in natural language processing. Given a context, RE aims to classify an entity-mention pair into a set of pre-defined relations. In the biomedical field, building an efficient and accurate RE system is critical for the construction of a domain knowledge base to support upper-level applications. Recent advances have witnessed a focus shift from sentence to document-level RE problems, which are more challenging due to the need for inter- and intra-sentence semantic reasoning. This type of distant dependency is difficult to understand and capture for a learning algorithm. To address the challenge, prior efforts either attempted to improve the cross sentence text representation or infuse domain or local knowledge into the model. Both strategies demonstrated efficacy on various datasets. In this paper, a keyword-attentive knowledge infusion strategy is proposed and integrated into BioBERT. A domain keyword collection mechanism is developed to discover the most relation-suggestive word tokens for bio-entities in a given context. By manipulating the attention masks, the model can be guided to focus on the semantic interaction between bio-entities linked by the keywords. We validated the proposed method on the Biocreative V Chemical Disease Relation dataset with an F1 of 75.6%, outperforming the state-of-the-art by 5.6%.

scholarly journals Biomedical Relation Extraction Using Distant Supervision

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Nada Boudjellal ◽  
Huaping Zhang ◽  
Asif Khan ◽  
Arshad Ahmad
Keyword(s):  
Big Data ◽  
Information Extraction ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Structured Data ◽  
Distant Supervision ◽  
Future Challenges ◽  
Unstructured Information ◽  
Biomedical Relation Extraction

With the accelerating growth of big data, especially in the healthcare area, information extraction is more needed currently than ever, for it can convey unstructured information into an easily interpretable structured data. Relation extraction is the second of the two important tasks of relation extraction. This study presents an overview of relation extraction using distant supervision, providing a generalized architecture of this task based on the state-of-the-art work that proposed this method. Besides, it surveys the methods used in the literature targeting this topic with a description of different knowledge bases used in the process along with the corpora, which can be helpful for beginner practitioners seeking knowledge on this subject. Moreover, the limitations of the proposed approaches and future challenges were highlighted, and possible solutions were proposed.

scholarly journals PPPred: Classifying Protein-phenotype Co-mentions Extracted from Biomedical Literature

2019 ◽  
Author(s):  
Morteza Pourreza Shahri ◽  
Mandi M. Roe ◽  
Gillian Reynolds ◽  
Indika Kahanda
Keyword(s):  
Relation Extraction ◽  
Biomedical Literature ◽  
Supervised Machine Learning ◽  
Human Phenotype ◽  
Unstructured Text ◽  
Gold Standard Dataset ◽  
Sentence Level ◽  
Machine Learning Approach ◽  
Human Proteins ◽  
Biomedical Relation Extraction

ABSTRACTThe MEDLINE database provides an extensive source of scientific articles and heterogeneous biomedical information in the form of unstructured text. One of the most important knowledge present within articles are the relations between human proteins and their phenotypes, which can stay hidden due to the exponential growth of publications. This has presented a range of opportunities for the development of computational methods to extract these biomedical relations from the articles. However, currently, no such method exists for the automated extraction of relations involving human proteins and human phenotype ontology (HPO) terms. In our previous work, we developed a comprehensive database composed of all co-mentions of proteins and phenotypes. In this study, we present a supervised machine learning approach called PPPred (Protein-Phenotype Predictor) for classifying the validity of a given sentence-level co-mention. Using an in-house developed gold standard dataset, we demonstrate that PPPred significantly outperforms several baseline methods. This two-step approach of co-mention extraction and classification constitutes a complete biomedical relation extraction pipeline for extracting protein-phenotype relations.CCS CONCEPTS•Computing methodologies → Information extraction; Supervised learning by classification; •Applied computing →Bioinformatics;

scholarly journals Document-level Relation Extraction as Semantic Segmentation

2021 ◽  
Author(s):  
Ningyu Zhang ◽  
Xiang Chen ◽  
Xin Xie ◽  
Shumin Deng ◽  
Chuanqi Tan ◽  
...  
Keyword(s):  
Computer Vision ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Segmentation ◽  
Experimental Results ◽  
Context Information ◽  
Global Information ◽  
Benchmark Datasets ◽  
Segmentation Task ◽  
Document Level

Document-level relation extraction aims to extract relations among multiple entity pairs from a document. Previously proposed graph-based or transformer-based models utilize the entities independently, regardless of global information among relational triples. This paper approaches the problem by predicting an entity-level relation matrix to capture local and global information, parallel to the semantic segmentation task in computer vision. Herein, we propose a Document U-shaped Network for document-level relation extraction. Specifically, we leverage an encoder module to capture the context information of entities and a U-shaped segmentation module over the image-style feature map to capture global interdependency among triples. Experimental results show that our approach can obtain state-of-the-art performance on three benchmark datasets DocRED, CDR, and GDA.

scholarly journals Kernelized Hashcode Representations for Relation Extraction

2019 ◽  
Vol 33 ◽  
pp. 6431-6440
Author(s):  
Sahil Garg ◽  
Aram Galstyan ◽  
Greg Ver Steeg ◽  
Irina Rish ◽  
Guillermo Cecchi ◽  
...  
Keyword(s):  
Kernel Methods ◽  
State Of The Art ◽  
Computational Cost ◽  
Relation Extraction ◽  
Locality Sensitive Hashing ◽  
Classification Methods ◽  
Classification Problems ◽  
Random Subspaces ◽  
Class Labels ◽  
Biomedical Relation Extraction

Kernel methods have produced state-of-the-art results for a number of NLP tasks such as relation extraction, but suffer from poor scalability due to the high cost of computing kernel similarities between natural language structures. A recently proposed technique, kernelized locality-sensitive hashing (KLSH), can significantly reduce the computational cost, but is only applicable to classifiers operating on kNN graphs. Here we propose to use random subspaces of KLSH codes for efficiently constructing an explicit representation of NLP structures suitable for general classification methods. Further, we propose an approach for optimizing the KLSH model for classification problems by maximizing an approximation of mutual information between the KLSH codes (feature vectors) and the class labels. We evaluate the proposed approach on biomedical relation extraction datasets, and observe significant and robust improvements in accuracy w.r.t. state-ofthe-art classifiers, along with drastic (orders-of-magnitude) speedup compared to conventional kernel methods.

Sign in / Sign up

Export Citation Format

Share Document