scholarly journals Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embedding

2019 ◽  
Vol 33 ◽  
pp. 7418-7425 ◽  
Author(s):  
Changsen Yuan ◽  
Heyan Huang ◽  
Chong Feng ◽  
Xiao Liu ◽  
Xiaochi Wei
Keyword(s):  
Mutual Information ◽  
Efficient Method ◽  
Relation Extraction ◽  
Benchmark Dataset ◽  
Labor Costs ◽  
Distant Supervision ◽  
Complex Information ◽  
Novel Approach ◽  
Independent And Identically Distributed

Distant supervision for relation extraction is an efficient method to reduce labor costs and has been widely used to seek novel relational facts in large corpora, which can be identified as a multi-instance multi-label problem. However, existing distant supervision methods suffer from selecting important words in the sentence and extracting valid sentences in the bag. Towards this end, we propose a novel approach to address these problems in this paper. Firstly, we propose a linear attenuation simulation to reflect the importance of words in the sentence with respect to the distances between entities and words. Secondly, we propose a non-independent and identically distributed (non-IID) relevance embedding to capture the relevance of sentences in the bag. Our method can not only capture complex information of words about hidden relations, but also express the mutual information of instances in the bag. Extensive experiments on a benchmark dataset have well-validated the effectiveness of the proposed method.

scholarly journals Learning to Transfer Relational Representations through Analogy

2019 ◽  
Vol 33 ◽  
pp. 10015-10016
Author(s):  
Gaetano Rossiello ◽  
Alfio Gliozzo ◽  
Michael Glass
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
The State ◽  
Large Set ◽  
Relational Information ◽  
Siamese Network ◽  
Distant Supervision ◽  
Novel Approach ◽  
Art Methods

We propose a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same relation, then those two pairs are analogous. We collect a large set of analogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. This dataset is adopted to train a hierarchical siamese network in order to learn entity-entity embeddings which encode relational information through the different linguistic paraphrasing expressing the same relation. The model can be used to generate pre-trained embeddings which provide a valuable signal when integrated into an existing neural-based model by outperforming the state-of-the-art methods on a relation extraction task.

scholarly journals Improving Neural Relation Extraction with Positive and Unlabeled Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 7927-7934
Author(s):  
Zhengqiu He ◽  
Wenliang Chen ◽  
Yuyi Wang ◽  
Wei Zhang ◽  
Guanchun Wang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Relation Extraction ◽  
Experimental Results ◽  
New Approach ◽  
Distant Supervision ◽  
Novel Approach ◽  
Pu Learning ◽  
Positive And Unlabeled Learning

We present a novel approach to improve the performance of distant supervision relation extraction with Positive and Unlabeled (PU) Learning. This approach first applies reinforcement learning to decide whether a sentence is positive to a given relation, and then positive and unlabeled bags are constructed. In contrast to most previous studies, which mainly use selected positive instances only, we make full use of unlabeled instances and propose two new representations for positive and unlabeled bags. These two representations are then combined in an appropriate way to make bag-level prediction. Experimental results on a widely used real-world dataset demonstrate that this new approach indeed achieves significant and consistent improvements as compared to several competitive baselines.

scholarly journals Utilizing Entity-Based Gated Convolution and Multilevel Sentence Attention to Improve Distantly Supervised Relation Extraction

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Qian Yi ◽  
Guixuan Zhang ◽  
Shuwu Zhang
Keyword(s):  
Selective Attention ◽  
Large Scale ◽  
State Of The Art ◽  
Relation Extraction ◽  
Benchmark Dataset ◽  
Experimental Results ◽  
Fine Grained ◽  
Convolution Operation ◽  
Distant Supervision ◽  
Extraction Model

Distant supervision is an effective method to automatically collect large-scale datasets for relation extraction (RE). Automatically constructed datasets usually comprise two types of noise: the intrasentence noise and the wrongly labeled noisy sentence. To address issues caused by the above two types of noise and improve distantly supervised relation extraction, this paper proposes a novel distantly supervised relation extraction model, which consists of an entity-based gated convolution sentence encoder and a multilevel sentence selective attention (Matt) module. Specifically, we first apply an entity-based gated convolution operation to force the sentence encoder to extract entity-pair-related features and filter out useless intrasentence noise information. Furthermore, the multilevel attention schema fuses the bag information to obtain a fine-grained bag-specific query vector, which can better identify valid sentences and reduce the influence of wrongly labeled sentences. Experimental results on a large-scale benchmark dataset show that our model can effectively reduce the influence of the above two types of noise and achieves state-of-the-art performance in relation extraction.

Distant Supervision for Relation Extraction with Matrix Completion

2014 ◽  
Author(s):  
Miao Fan ◽  
Deli Zhao ◽  
Qiang Zhou ◽  
Zhiyuan Liu ◽  
Thomas Fang Zheng ◽  
...  
Keyword(s):  
Matrix Completion ◽  
Relation Extraction ◽  
Distant Supervision

Bootstrapped Multi-level Distant Supervision for Relation Extraction

2018 ◽  
pp. 408-423 ◽  
Author(s):  
Ying He ◽  
Zhixu Li ◽  
Guanfeng Liu ◽  
Fangfei Cao ◽  
Zhigang Chen ◽  
...  
Keyword(s):  
Relation Extraction ◽  
Distant Supervision ◽  
Multi Level

scholarly journals Neuron Interaction Based Representation Composition for Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8204-8211
Author(s):  
Jian Li ◽  
Xing Wang ◽  
Baosong Yang ◽  
Shuming Shi ◽  
Michael R. Lyu ◽  
...  
Keyword(s):  
Machine Translation ◽  
Semantic Information ◽  
Low Rank ◽  
Strong Interactions ◽  
Low Rank Approximation ◽  
Neural Machine Translation ◽  
First Order ◽  
Complex Information ◽  
Novel Approach ◽  
Rank Approximation

Recent NLP studies reveal that substantial linguistic information can be attributed to single neurons, i.e., individual dimensions of the representation vectors. We hypothesize that modeling strong interactions among neurons helps to better capture complex information by composing the linguistic properties embedded in individual neurons. Starting from this intuition, we propose a novel approach to compose representations learned by different components in neural machine translation (e.g., multi-layer networks or multi-head attention), based on modeling strong interactions among neurons in the representation vectors. Specifically, we leverage bilinear pooling to model pairwise multiplicative interactions among individual neurons, and a low-rank approximation to make the model computationally feasible. We further propose extended bilinear pooling to incorporate first-order representations. Experiments on WMT14 English⇒German and English⇒French translation tasks show that our model consistently improves performances over the SOTA Transformer baseline. Further analyses demonstrate that our approach indeed captures more syntactic and semantic information as expected.

Sign in / Sign up

Export Citation Format

Share Document