scholarly journals Simultaneously Linking Entities and Extracting Relations from Biomedical Text without Mention-Level Supervision

2020 ◽  
Vol 34 (05) ◽  
pp. 7407-7414
Author(s):  
Trapit Bansal ◽  
Pat Verga ◽  
Neha Choudhary ◽  
Andrew McCallum
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Training Data ◽  
Biomedical Text ◽  
Entity Linking ◽  
Distant Supervision ◽  
Entity Relationships ◽  
And Training

Understanding the meaning of text often involves reasoning about entities and their relationships. This requires identifying textual mentions of entities, linking them to a canonical concept, and discerning their relationships. These tasks are nearly always viewed as separate components within a pipeline, each requiring a distinct model and training data. While relation extraction can often be trained with readily available weak or distant supervision, entity linkers typically require expensive mention-level supervision – which is not available in many domains. Instead, we propose a model which is trained to simultaneously produce entity linking and relation decisions while requiring no mention-level annotations. This approach avoids cascading errors that arise from pipelined methods and more accurately predicts entity relationships from text. We show that our model outperforms a state-of-the art entity linking and relation extraction pipeline on two biomedical datasets and can drastically improve the overall recall of the system.

scholarly journals Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction

2019 ◽  
Vol 33 ◽  
pp. 419-426 ◽  
Author(s):  
Yujin Yuan ◽  
Liyuan Liu ◽  
Siliang Tang ◽  
Zhongfei Zhang ◽  
Yueting Zhuang ◽  
...  
Keyword(s):  
Selective Attention ◽  
Supervised Learning ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Distant Supervision ◽  
Sentence Level ◽  
Noise Robust

Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations. However, the generated training data typically contain massive noise, and may result in poor performances with the vanilla supervised learning. In this paper, we propose to conduct multi-instance learning with a novel Cross-relation Cross-bag Selective Attention (C2SA), which leads to noise-robust training for distant supervised relation extractor. Specifically, we employ the sentence-level selective attention to reduce the effect of noisy or mismatched sentences, while the correlation among relations were captured to improve the quality of attention weights. Moreover, instead of treating all entity-pairs equally, we try to pay more attention to entity-pairs with a higher quality. Similarly, we adopt the selective attention mechanism to achieve this goal. Experiments with two types of relation extractor demonstrate the superiority of the proposed approach over the state-of-the-art, while further ablation studies verify our intuitions and demonstrate the effectiveness of our proposed two techniques.

scholarly journals Bias Modeling for Distantly Supervised Relation Extraction

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Yang Xiang ◽  
Yaoyun Zhang ◽  
Xiaolong Wang ◽  
Yang Qin ◽  
Wenying Han
Keyword(s):  
Language Processing ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Free Text ◽  
Distant Supervision ◽  
Annotation Process ◽  
Noise Tolerant

Distant supervision (DS) automatically annotates free text with relation mentions from existing knowledge bases (KBs), providing a way to alleviate the problem of insufficient training data for relation extraction in natural language processing (NLP). However, the heuristic annotation process does not guarantee the correctness of the generated labels, promoting a hot research issue on how to efficiently make use of the noisy training data. In this paper, we model two types of biases to reduce noise: (1)bias-distto model the relative distance between points (instances) and classes (relation centers); (2)bias-rewardto model the possibility of each heuristically generated label being incorrect. Based on the biases, we propose three noise tolerant models:MIML-dist,MIML-dist-classify, andMIML-reward, building on top of a state-of-the-art distantly supervised learning algorithm. Experimental evaluations compared with three landmark methods on the KBP dataset validate the effectiveness of the proposed methods.

scholarly journals Extraction of causal relations based on SBEL and BERT model

Database ◽  
2021 ◽  
Vol 2021 ◽  
Author(s):  
Yifan Shao ◽  
Haoru Li ◽  
Jinghang Gu ◽  
Longhua Qian ◽  
Guodong Zhou
Keyword(s):  
State Of The Art ◽  
Causal Relation ◽  
Relation Extraction ◽  
The Other ◽  
Biomedical Text ◽  
Intermediate Form ◽  
Biomedical Text Mining ◽  
Causal Relations ◽  
The One ◽  
Stage 1

Abstract Extraction of causal relations between biomedical entities in the form of Biological Expression Language (BEL) poses a new challenge to the community of biomedical text mining due to the complexity of BEL statements. We propose a simplified form of BEL statements [Simplified Biological Expression Language (SBEL)] to facilitate BEL extraction and employ BERT (Bidirectional Encoder Representation from Transformers) to improve the performance of causal relation extraction (RE). On the one hand, BEL statement extraction is transformed into the extraction of an intermediate form—SBEL statement, which is then further decomposed into two subtasks: entity RE and entity function detection. On the other hand, we use a powerful pretrained BERT model to both extract entity relations and detect entity functions, aiming to improve the performance of two subtasks. Entity relations and functions are then combined into SBEL statements and finally merged into BEL statements. Experimental results on the BioCreative-V Track 4 corpus demonstrate that our method achieves the state-of-the-art performance in BEL statement extraction with F1 scores of 54.8% in Stage 2 evaluation and of 30.1% in Stage 1 evaluation, respectively. Database URL: https://github.com/grapeff/SBEL_datasets

scholarly journals Learning to Transfer Relational Representations through Analogy

2019 ◽  
Vol 33 ◽  
pp. 10015-10016
Author(s):  
Gaetano Rossiello ◽  
Alfio Gliozzo ◽  
Michael Glass
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
The State ◽  
Large Set ◽  
Relational Information ◽  
Siamese Network ◽  
Distant Supervision ◽  
Novel Approach ◽  
Art Methods

We propose a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same relation, then those two pairs are analogous. We collect a large set of analogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. This dataset is adopted to train a hierarchical siamese network in order to learn entity-entity embeddings which encode relational information through the different linguistic paraphrasing expressing the same relation. The model can be used to generate pre-trained embeddings which provide a valuable signal when integrated into an existing neural-based model by outperforming the state-of-the-art methods on a relation extraction task.

scholarly journals Biomedical Relation Extraction Using Distant Supervision

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Nada Boudjellal ◽  
Huaping Zhang ◽  
Asif Khan ◽  
Arshad Ahmad
Keyword(s):  
Big Data ◽  
Information Extraction ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Structured Data ◽  
Distant Supervision ◽  
Future Challenges ◽  
Unstructured Information ◽  
Biomedical Relation Extraction

With the accelerating growth of big data, especially in the healthcare area, information extraction is more needed currently than ever, for it can convey unstructured information into an easily interpretable structured data. Relation extraction is the second of the two important tasks of relation extraction. This study presents an overview of relation extraction using distant supervision, providing a generalized architecture of this task based on the state-of-the-art work that proposed this method. Besides, it surveys the methods used in the literature targeting this topic with a description of different knowledge bases used in the process along with the corpora, which can be helpful for beginner practitioners seeking knowledge on this subject. Moreover, the limitations of the proposed approaches and future challenges were highlighted, and possible solutions were proposed.

scholarly journals Examining Deep Learning Architectures for Crime Classification and Prediction

Forecasting ◽  
2021 ◽  
Vol 3 (4) ◽  
pp. 741-762
Author(s):  
Panagiotis Stalidis ◽  
Theodoros Semertzidis ◽  
Petros Daras
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Open Data ◽  
Training Data ◽  
Crime Prediction ◽  
Crime Types ◽  
Improved Performance ◽  
Learning Architectures ◽  
And Training ◽  
Crime Classification

In this paper, a detailed study on crime classification and prediction using deep learning architectures is presented. We examine the effectiveness of deep learning algorithms in this domain and provide recommendations for designing and training deep learning systems for predicting crime areas, using open data from police reports. Having time-series of crime types per location as training data, a comparative study of 10 state-of-the-art methods against 3 different deep learning configurations is conducted. In our experiments with 5 publicly available datasets, we demonstrate that the deep learning-based methods consistently outperform the existing best-performing methods. Moreover, we evaluate the effectiveness of different parameters in the deep learning architectures and give insights for configuring them to achieve improved performance in crime classification and finally crime prediction.

scholarly journals Using distant supervision to augment manually annotated data for relation extraction

2019 ◽  
Author(s):  
Peng Su ◽  
Gang Li ◽  
Cathy Wu ◽  
K. Vijay-Shanker
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Relation Extraction ◽  
Biomedical Literature ◽  
Training Data ◽  
Distant Supervision ◽  
Large Size ◽  
Domain Expertise

AbstractSignificant progress has been made in applying deep learning on natural language processing tasks recently. However, deep learning models typically require a large amount of annotated training data while often only small labeled datasets are available for many natural language processing tasks in biomedical literature. Building large-size datasets for deep learning is expensive since it involves considerable human effort and usually requires domain expertise in specialized fields. In this work, we consider augmenting manually annotated data with large amounts of data using distant supervision. However, data obtained by distant supervision is often noisy, we first apply some heuristics to remove some of the incorrect annotations. Then using methods inspired from transfer learning, we show that the resulting models outperform models trained on the original manually annotated sets.

scholarly journals Distant Supervision for Relation Extraction with Sentence Selection and Interaction Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Tiantian Chen ◽  
Nianbin Wang ◽  
Hongbin Wang ◽  
Haomin Zhan
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Features ◽  
Distant Supervision ◽  
Word Level ◽  
Proposed Model ◽  
Relation Prediction ◽  
Better Than

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large-scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word-level attention mechanism-based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1-score on a widely used dataset, which proves that our model performs significantly better than the state-of-the-art RE systems.

scholarly journals SpanBERT: Improving Pre-training by Representing and Predicting Spans

2020 ◽  
Vol 8 ◽  
pp. 64-77 ◽  
Author(s):  
Mandar Joshi ◽  
Danqi Chen ◽  
Yinhan Liu ◽  
Daniel S. Weld ◽  
Luke Zettlemoyer ◽  
...  
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Relation Extraction ◽  
Training Data ◽  
Coreference Resolution ◽  
Single Model ◽  
Model Size ◽  
Boundary Representations ◽  
Selection Tasks ◽  
The Individual

We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the span boundary representations to predict the entire content of the masked span, without relying on the individual token representations within it. SpanBERT consistently outperforms BERT and our better-tuned baselines, with substantial gains on span selection tasks such as question answering and coreference resolution. In particular, with the same training data and model size as BERTlarge, our single model obtains 94.6% and 88.7% F1 on SQuAD 1.1 and 2.0 respectively. We also achieve a new state of the art on the OntoNotes coreference resolution task (79.6% F1), strong performance on the TACRED relation extraction benchmark, and even gains on GLUE. 1

scholarly journals PEDL: extracting protein–protein associations using deep language models and distant supervision

2020 ◽  
Vol 36 (Supplement_1) ◽  
pp. i490-i498
Author(s):  
Leon Weber ◽  
Kirsten Thobe ◽  
Oscar Arturo Migueles Lozano ◽  
Jana Wolf ◽  
Ulf Leser
Keyword(s):  
Relation Extraction ◽  
Training Data ◽  
Language Models ◽  
Supplementary Information ◽  
Functional Protein ◽  
Major Pathway ◽  
Distant Supervision ◽  
Order Of Magnitude ◽  
Biomedical Publications ◽  
Pathway Databases

Abstract Motivation A significant portion of molecular biology investigates signalling pathways and thus depends on an up-to-date and complete resource of functional protein–protein associations (PPAs) that constitute such pathways. Despite extensive curation efforts, major pathway databases are still notoriously incomplete. Relation extraction can help to gather such pathway information from biomedical publications. Current methods for extracting PPAs typically rely exclusively on rare manually labelled data which severely limits their performance. Results We propose PPA Extraction with Deep Language (PEDL), a method for predicting PPAs from text that combines deep language models and distant supervision. Due to the reliance on distant supervision, PEDL has access to an order of magnitude more training data than methods solely relying on manually labelled annotations. We introduce three different datasets for PPA prediction and evaluate PEDL for the two subtasks of predicting PPAs between two proteins, as well as identifying the text spans stating the PPA. We compared PEDL with a recently published state-of-the-art model and found that on average PEDL performs better in both tasks on all three datasets. An expert evaluation demonstrates that PEDL can be used to predict PPAs that are missing from major pathway databases and that it correctly identifies the text spans supporting the PPA. Availability and implementation PEDL is freely available at https://github.com/leonweber/pedl. The repository also includes scripts to generate the used datasets and to reproduce the experiments from this article. Supplementary information Supplementary data are available at Bioinformatics online.

Sign in / Sign up

Export Citation Format

Share Document