scholarly journals Bias Modeling for Distantly Supervised Relation Extraction

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Yang Xiang ◽  
Yaoyun Zhang ◽  
Xiaolong Wang ◽  
Yang Qin ◽  
Wenying Han
Keyword(s):  
Language Processing ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Free Text ◽  
Distant Supervision ◽  
Annotation Process ◽  
Noise Tolerant

Distant supervision (DS) automatically annotates free text with relation mentions from existing knowledge bases (KBs), providing a way to alleviate the problem of insufficient training data for relation extraction in natural language processing (NLP). However, the heuristic annotation process does not guarantee the correctness of the generated labels, promoting a hot research issue on how to efficiently make use of the noisy training data. In this paper, we model two types of biases to reduce noise: (1)bias-distto model the relative distance between points (instances) and classes (relation centers); (2)bias-rewardto model the possibility of each heuristically generated label being incorrect. Based on the biases, we propose three noise tolerant models:MIML-dist,MIML-dist-classify, andMIML-reward, building on top of a state-of-the-art distantly supervised learning algorithm. Experimental evaluations compared with three landmark methods on the KBP dataset validate the effectiveness of the proposed methods.

scholarly journals Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction

2019 ◽  
Vol 33 ◽  
pp. 419-426 ◽  
Author(s):  
Yujin Yuan ◽  
Liyuan Liu ◽  
Siliang Tang ◽  
Zhongfei Zhang ◽  
Yueting Zhuang ◽  
...  
Keyword(s):  
Selective Attention ◽  
Supervised Learning ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Distant Supervision ◽  
Sentence Level ◽  
Noise Robust

Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations. However, the generated training data typically contain massive noise, and may result in poor performances with the vanilla supervised learning. In this paper, we propose to conduct multi-instance learning with a novel Cross-relation Cross-bag Selective Attention (C2SA), which leads to noise-robust training for distant supervised relation extractor. Specifically, we employ the sentence-level selective attention to reduce the effect of noisy or mismatched sentences, while the correlation among relations were captured to improve the quality of attention weights. Moreover, instead of treating all entity-pairs equally, we try to pay more attention to entity-pairs with a higher quality. Similarly, we adopt the selective attention mechanism to achieve this goal. Experiments with two types of relation extractor demonstrate the superiority of the proposed approach over the state-of-the-art, while further ablation studies verify our intuitions and demonstrate the effectiveness of our proposed two techniques.

scholarly journals Learning to Transfer Relational Representations through Analogy

2019 ◽  
Vol 33 ◽  
pp. 10015-10016
Author(s):  
Gaetano Rossiello ◽  
Alfio Gliozzo ◽  
Michael Glass
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
The State ◽  
Large Set ◽  
Relational Information ◽  
Siamese Network ◽  
Distant Supervision ◽  
Novel Approach ◽  
Art Methods

We propose a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same relation, then those two pairs are analogous. We collect a large set of analogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. This dataset is adopted to train a hierarchical siamese network in order to learn entity-entity embeddings which encode relational information through the different linguistic paraphrasing expressing the same relation. The model can be used to generate pre-trained embeddings which provide a valuable signal when integrated into an existing neural-based model by outperforming the state-of-the-art methods on a relation extraction task.

scholarly journals Biomedical Relation Extraction Using Distant Supervision

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Nada Boudjellal ◽  
Huaping Zhang ◽  
Asif Khan ◽  
Arshad Ahmad
Keyword(s):  
Big Data ◽  
Information Extraction ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Structured Data ◽  
Distant Supervision ◽  
Future Challenges ◽  
Unstructured Information ◽  
Biomedical Relation Extraction

With the accelerating growth of big data, especially in the healthcare area, information extraction is more needed currently than ever, for it can convey unstructured information into an easily interpretable structured data. Relation extraction is the second of the two important tasks of relation extraction. This study presents an overview of relation extraction using distant supervision, providing a generalized architecture of this task based on the state-of-the-art work that proposed this method. Besides, it surveys the methods used in the literature targeting this topic with a description of different knowledge bases used in the process along with the corpora, which can be helpful for beginner practitioners seeking knowledge on this subject. Moreover, the limitations of the proposed approaches and future challenges were highlighted, and possible solutions were proposed.

scholarly journals Using distant supervision to augment manually annotated data for relation extraction

2019 ◽  
Author(s):  
Peng Su ◽  
Gang Li ◽  
Cathy Wu ◽  
K. Vijay-Shanker
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Relation Extraction ◽  
Biomedical Literature ◽  
Training Data ◽  
Distant Supervision ◽  
Large Size ◽  
Domain Expertise

AbstractSignificant progress has been made in applying deep learning on natural language processing tasks recently. However, deep learning models typically require a large amount of annotated training data while often only small labeled datasets are available for many natural language processing tasks in biomedical literature. Building large-size datasets for deep learning is expensive since it involves considerable human effort and usually requires domain expertise in specialized fields. In this work, we consider augmenting manually annotated data with large amounts of data using distant supervision. However, data obtained by distant supervision is often noisy, we first apply some heuristics to remove some of the incorrect annotations. Then using methods inspired from transfer learning, we show that the resulting models outperform models trained on the original manually annotated sets.

scholarly journals Simultaneously Linking Entities and Extracting Relations from Biomedical Text without Mention-Level Supervision

2020 ◽  
Vol 34 (05) ◽  
pp. 7407-7414
Author(s):  
Trapit Bansal ◽  
Pat Verga ◽  
Neha Choudhary ◽  
Andrew McCallum
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Training Data ◽  
Biomedical Text ◽  
Entity Linking ◽  
Distant Supervision ◽  
Entity Relationships ◽  
And Training

Understanding the meaning of text often involves reasoning about entities and their relationships. This requires identifying textual mentions of entities, linking them to a canonical concept, and discerning their relationships. These tasks are nearly always viewed as separate components within a pipeline, each requiring a distinct model and training data. While relation extraction can often be trained with readily available weak or distant supervision, entity linkers typically require expensive mention-level supervision – which is not available in many domains. Instead, we propose a model which is trained to simultaneously produce entity linking and relation decisions while requiring no mention-level annotations. This approach avoids cascading errors that arise from pipelined methods and more accurately predicts entity relationships from text. We show that our model outperforms a state-of-the art entity linking and relation extraction pipeline on two biomedical datasets and can drastically improve the overall recall of the system.

scholarly journals Full-Abstract Biomedical Relation Extraction with Keyword-Attentive Domain Knowledge Infusion

2021 ◽  
Vol 11 (16) ◽  
pp. 7318
Author(s):  
Xian Zhu ◽  
Lele Zhang ◽  
Jiangnan Du ◽  
Zhifeng Xiao
Keyword(s):  
Language Processing ◽  
Domain Knowledge ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Relation Extraction ◽  
Text Representation ◽  
Upper Level ◽  
Document Level ◽  
Semantic Interaction ◽  
Biomedical Relation Extraction

Relation extraction (RE) is an essential task in natural language processing. Given a context, RE aims to classify an entity-mention pair into a set of pre-defined relations. In the biomedical field, building an efficient and accurate RE system is critical for the construction of a domain knowledge base to support upper-level applications. Recent advances have witnessed a focus shift from sentence to document-level RE problems, which are more challenging due to the need for inter- and intra-sentence semantic reasoning. This type of distant dependency is difficult to understand and capture for a learning algorithm. To address the challenge, prior efforts either attempted to improve the cross sentence text representation or infuse domain or local knowledge into the model. Both strategies demonstrated efficacy on various datasets. In this paper, a keyword-attentive knowledge infusion strategy is proposed and integrated into BioBERT. A domain keyword collection mechanism is developed to discover the most relation-suggestive word tokens for bio-entities in a given context. By manipulating the attention masks, the model can be guided to focus on the semantic interaction between bio-entities linked by the keywords. We validated the proposed method on the Biocreative V Chemical Disease Relation dataset with an F1 of 75.6%, outperforming the state-of-the-art by 5.6%.

scholarly journals A Recurrent Model for Collective Entity Linking with Adaptive Features

2020 ◽  
Vol 34 (01) ◽  
pp. 329-336
Author(s):  
Xiaoling Zhou ◽  
Yukai Miao ◽  
Wei Wang ◽  
Jianbin Qin
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
State Of The Art ◽  
Learning To Rank ◽  
Parameter Tuning ◽  
Knowledge Bases ◽  
The State ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Free Text

The vast amount of web data enables us to build knowledge bases with unprecedented quality and coverage. Named Entity Disambiguation (NED) is an important task that automatically resolves ambiguous mentions in free text to correct target entries in the knowledge base. Traditional machine learning based methods for NED were outperformed and made obsolete by the state-of-the-art deep learning based models. However, deep learning models are more complex, requiring large amount of training data and lengthy training and parameter tuning time. In this paper, we revisit traditional machine learning techniques and propose a light-weight, tuneable and time-efficient method without using deep learning or deep learning generated features. We propose novel adaptive features that focus on extracting discriminative features to better model similarities between candidate entities and the mention's context. We learn a local ranking model based on traditional and the new adaptive features based on the learning-to-rank framework. While arriving at linking decisions individually via the local model, our method also takes into consideration the correlation between decisions by running multiple recurrent global models, which can be deemed as a learned local search method. Our method attains performances comparable to the state-of-the-art deep learning-based methods on NED benchmark datasets while being significantly faster to train.

scholarly journals Fine-Grained Named Entity Typing over Distantly Supervised Data Based on Refined Representations

2020 ◽  
Vol 34 (05) ◽  
pp. 7391-7398
Author(s):  
Muhammad Asif Ali ◽  
Yifang Sun ◽  
Bing Li ◽  
Wei Wang
Keyword(s):  
Language Processing ◽  
Training Data ◽  
Specific Context ◽  
Fine Grained ◽  
Named Entity ◽  
Distant Supervision ◽  
Proposed Model ◽  
Wide Range ◽  
Relative Score ◽  
Noisy Labels

Fine-Grained Named Entity Typing (FG-NET) is a key component in Natural Language Processing (NLP). It aims at classifying an entity mention into a wide range of entity types. Due to a large number of entity types, distant supervision is used to collect training data for this task, which noisily assigns type labels to entity mentions irrespective of the context. In order to alleviate the noisy labels, existing approaches on FG-NET analyze the entity mentions entirely independent of each other and assign type labels solely based on mention's sentence-specific context. This is inadequate for highly overlapping and/or noisy type labels as it hinders information passing across sentence boundaries. For this, we propose an edge-weighted attentive graph convolution network that refines the noisy mention representations by attending over corpus-level contextual clues prior to the end classification. Experimental evaluation shows that the proposed model outperforms the existing research by a relative score of upto 10.2% and 8.3% for macro-f1 and micro-f1 respectively.

Adapting SVM for data sparseness and imbalance: a case study in information extraction

2009 ◽  
Vol 15 (2) ◽  
pp. 241-271 ◽  
Author(s):  
YAOYONG LI ◽  
KALINA BONTCHEVA ◽  
HAMISH CUNNINGHAM
Keyword(s):  
Active Learning ◽  
Language Learning ◽  
Information Extraction ◽  
Language Processing ◽  
Learning Algorithm ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Support Vector ◽  
Passive Learning ◽  
Wide Range

AbstractSupport Vector Machines (SVM) have been used successfully in many Natural Language Processing (NLP) tasks. The novel contribution of this paper is in investigating two techniques for making SVM more suitable for language learning tasks. Firstly, we propose an SVM with uneven margins (SVMUM) model to deal with the problem of imbalanced training data. Secondly, SVM active learning is employed in order to alleviate the difficulty in obtaining labelled training data. The algorithms are presented and evaluated on several Information Extraction (IE) tasks, where they achieved better performance than the standard SVM and the SVM with passive learning, respectively. Moreover, by combining SVMUM with the active learning algorithm, we achieve the best reported results on the seminars and jobs corpora, which are benchmark data sets used for evaluation and comparison of machine learning algorithms for IE. In addition, we also evaluate the token based classification framework for IE with three different entity tagging schemes. In comparison to previous methods dealing with the same problems, our methods are both effective and efficient, which are valuable features for real-world applications. Due to the similarity in the formulation of the learning problem for IE and for other NLP tasks, the two techniques are likely to be beneficial in a wide range of applications1.

scholarly journals Towards relation extraction from Arabic text: a review

2019 ◽  
Vol 5 (5) ◽  
pp. 212-215
Author(s):  
Abeer AlArfaj
Keyword(s):  
Language Processing ◽  
Question Answering ◽  
State Of The Art ◽  
Relation Extraction ◽  
Arabic Language ◽  
Extraction Methods ◽  
The State ◽  
Semantic Relations ◽  
Arabic Text ◽  
Taxonomic Relation

Semantic relation extraction is an important component of ontologies that can support many applications e.g. text mining, question answering, and information extraction. However, extracting semantic relations between concepts is not trivial and one of the main challenges in Natural Language Processing (NLP) Field. The Arabic language has complex morphological, grammatical, and semantic aspects since it is a highly inflectional and derivational language, which makes task even more challenging. In this paper, we present a review of the state of the art for relation extraction from texts, addressing the progress and difficulties in this field. We discuss several aspects related to this task, considering the taxonomic and non-taxonomic relation extraction methods. Majority of relation extraction approaches implement a combination of statistical and linguistic techniques to extract semantic relations from text. We also give special attention to the state of the work on relation extraction from Arabic texts, which need further progress.

Sign in / Sign up

Export Citation Format

Share Document