scholarly journals ATFE: A Two-dimensional Feature Encoding-based Sentence-level Attention Model for Distant Supervised Relation Extraction

2021 ◽  
Author(s):  
Shiyang Li
Keyword(s):  
Relation Extraction ◽  
Two Dimensional ◽  
Attention Model ◽  
Sentence Level ◽  
Feature Encoding

scholarly journals Effective Deep Memory Networks for Distant Supervised Relation Extraction

2017 ◽  
Author(s):  
Xiaocheng Feng ◽  
Jiang Guo ◽  
Bing Qin ◽  
Ting Liu ◽  
Yongjie Liu
Keyword(s):  
Relation Extraction ◽  
Training Data ◽  
Context Word ◽  
Neural Models ◽  
Attention Model ◽  
Memory Experiment ◽  
Feature Based ◽  
Real World Datasets ◽  
Major Attention ◽  
Better Than

Distant supervised relation extraction (RE) has been an effective way of finding novel relational facts from text without labeled training data. Typically it can be formalized as a multi-instance multi-label problem.In this paper, we introduce a novel neural approach for distant supervised (RE) with specific focus on attention mechanisms.Unlike the feature-based logistic regression model and compositional neural models such as CNN, our approach includes two major attention-based memory components, which is capable of explicitly capturing the importance of each context word for modeling the representation of the entity pair, as well as the intrinsic dependencies between relations.Such importance degree and dependency relationship are calculated with multiple computational layers, each of which is a neural attention model over an external memory. Experiment on real-world datasets shows that our approach performs significantly and consistently better than various baselines.

scholarly journals Dual CNN for Relation Extraction with Knowledge-Based Attention and Word Embeddings

2019 ◽  
Vol 2019 ◽  
pp. 1-10 ◽  
Author(s):  
Jun Li ◽  
Guimin Huang ◽  
Jianheng Chen ◽  
Yabing Wang
Keyword(s):  
Knowledge Base ◽  
Relation Extraction ◽  
Background Knowledge ◽  
Word Embedding ◽  
Entity Recognition ◽  
Instance Selection ◽  
Attention Model ◽  
Training Tool ◽  
Knowledge Based ◽  
Proposed Model

Relation extraction is the underlying critical task of textual understanding. However, the existing methods currently have defects in instance selection and lack background knowledge for entity recognition. In this paper, we propose a knowledge-based attention model, which can make full use of supervised information from a knowledge base, to select an entity. We also design a method of dual convolutional neural networks (CNNs) considering the word embedding of each word is restricted by using a single training tool. The proposed model combines a CNN with an attention mechanism. The model inserts the word embedding and supervised information from the knowledge base into the CNN, performs convolution and pooling, and combines the knowledge base and CNN in the full connection layer. Based on these processes, the model not only obtains better entity representations but also improves the performance of relation extraction with the help of rich background knowledge. The experimental results demonstrate that the proposed model achieves competitive performance.

scholarly journals PPPred: Classifying Protein-phenotype Co-mentions Extracted from Biomedical Literature

2019 ◽  
Author(s):  
Morteza Pourreza Shahri ◽  
Mandi M. Roe ◽  
Gillian Reynolds ◽  
Indika Kahanda
Keyword(s):  
Relation Extraction ◽  
Biomedical Literature ◽  
Supervised Machine Learning ◽  
Human Phenotype ◽  
Unstructured Text ◽  
Gold Standard Dataset ◽  
Sentence Level ◽  
Machine Learning Approach ◽  
Human Proteins ◽  
Biomedical Relation Extraction

ABSTRACTThe MEDLINE database provides an extensive source of scientific articles and heterogeneous biomedical information in the form of unstructured text. One of the most important knowledge present within articles are the relations between human proteins and their phenotypes, which can stay hidden due to the exponential growth of publications. This has presented a range of opportunities for the development of computational methods to extract these biomedical relations from the articles. However, currently, no such method exists for the automated extraction of relations involving human proteins and human phenotype ontology (HPO) terms. In our previous work, we developed a comprehensive database composed of all co-mentions of proteins and phenotypes. In this study, we present a supervised machine learning approach called PPPred (Protein-Phenotype Predictor) for classifying the validity of a given sentence-level co-mention. Using an in-house developed gold standard dataset, we demonstrate that PPPred significantly outperforms several baseline methods. This two-step approach of co-mention extraction and classification constitutes a complete biomedical relation extraction pipeline for extracting protein-phenotype relations.CCS CONCEPTS•Computing methodologies → Information extraction; Supervised learning by classification; •Applied computing →Bioinformatics;

scholarly journals MRE: A Military Relation Extraction Model Based on BiGRU and Multi-Head Attention

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1742
Author(s):  
Yiwei Lu ◽  
Ruopeng Yang ◽  
Xuping Jiang ◽  
Dan Zhou ◽  
Changshen Yin ◽  
...  
Keyword(s):  
Comprehensive Evaluation ◽  
Language Model ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Semantic Features ◽  
Chinese Word Segmentation ◽  
Conceptual Foundation ◽  
Attention Model ◽  
Operational Information ◽  
Military Relations

A great deal of operational information exists in the form of text. Therefore, extracting operational information from unstructured military text is of great significance for assisting command decision making and operations. Military relation extraction is one of the main tasks of military information extraction, which aims at identifying the relation between two named entities from unstructured military texts. However, the traditional methods of extracting military relations cannot easily resolve problems such as inadequate manual features and inaccurate Chinese word segmentation in military fields, failing to make full use of symmetrical entity relations in military texts. With our approach, based on the pre-trained language model, we present a Chinese military relation extraction method, which combines the bi-directional gate recurrent unit (BiGRU) and multi-head attention mechanism (MHATT). More specifically, the conceptual foundation of our method lies in constructing an embedding layer and combining word embedding with position embedding, based on the pre-trained language model; the output vectors of BiGRU neural networks are symmetrically spliced to learn the semantic features of context, and they fuse the multi-head attention mechanism to improve the ability of expressing semantic information. On the military text corpus that we have built, we conduct extensive experiments. We demonstrate the superiority of our method over the traditional non-attention model, attention model, and improved attention model, and the comprehensive evaluation value F1-score of the model is improved by about 4%.

Direction-sensitive relation extraction using Bi-SDP attention model

2020 ◽  
Vol 198 ◽  
pp. 105928 ◽  
Author(s):  
Hailin Wang ◽  
Ke Qin ◽  
Guoming Lu ◽  
Guangchun Luo ◽  
Guisong Liu
Keyword(s):  
Relation Extraction ◽  
Attention Model

scholarly journals Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction

2019 ◽  
Vol 33 ◽  
pp. 419-426 ◽  
Author(s):  
Yujin Yuan ◽  
Liyuan Liu ◽  
Siliang Tang ◽  
Zhongfei Zhang ◽  
Yueting Zhuang ◽  
...  
Keyword(s):  
Selective Attention ◽  
Supervised Learning ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Distant Supervision ◽  
Sentence Level ◽  
Noise Robust

Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations. However, the generated training data typically contain massive noise, and may result in poor performances with the vanilla supervised learning. In this paper, we propose to conduct multi-instance learning with a novel Cross-relation Cross-bag Selective Attention (C2SA), which leads to noise-robust training for distant supervised relation extractor. Specifically, we employ the sentence-level selective attention to reduce the effect of noisy or mismatched sentences, while the correlation among relations were captured to improve the quality of attention weights. Moreover, instead of treating all entity-pairs equally, we try to pay more attention to entity-pairs with a higher quality. Similarly, we adopt the selective attention mechanism to achieve this goal. Experiments with two types of relation extractor demonstrate the superiority of the proposed approach over the state-of-the-art, while further ablation studies verify our intuitions and demonstrate the effectiveness of our proposed two techniques.

scholarly journals Using Sentence-Level Neural Network Models for Multiple-Choice Reading Comprehension Tasks

2018 ◽  
Vol 2018 ◽  
pp. 1-8 ◽  
Author(s):  
Yuanlong Wang ◽  
Ru Li ◽  
Hu Zhang ◽  
Hongyan Tan ◽  
Qinghua Chai
Keyword(s):  
Neural Network ◽  
Reading Comprehension ◽  
Sentence Comprehension ◽  
Multiple Choice ◽  
Network Models ◽  
Choice Task ◽  
Chinese Reading ◽  
Neural Network Models ◽  
Attention Model ◽  
Sentence Level

Comprehending unstructured text is a challenging task for machines because it involves understanding texts and answering questions. In this paper, we study the multiple-choice task for reading comprehension based on MC Test datasets and Chinese reading comprehension datasets, among which Chinese reading comprehension datasets which are built by ourselves. Observing the above-mentioned training sets, we find that “sentence comprehension” is more important than “word comprehension” in multiple-choice task, and therefore we propose sentence-level neural network models. Our model firstly uses LSTM network and a composition model to learn compositional vector representation for sentences and then trains a sentence-level attention model for obtaining the sentence-level attention between the sentence embedding in documents and the optional sentences embedding by dot product. Finally, a consensus attention is gained by merging individual attention with the merging function. Experimental results show that our model outperforms various state-of-the-art baselines significantly for both the multiple-choice reading comprehension datasets.

scholarly journals Temporal enhanced sentence‐level attention model for hashtag recommendation

2018 ◽  
Vol 3 (2) ◽  
pp. 95-100 ◽  
Author(s):  
Jun Ma ◽  
Chong Feng ◽  
Ge Shi ◽  
Xuewen Shi ◽  
Heyang Huang
Keyword(s):  
Attention Model ◽  
Sentence Level
Sign in / Sign up

Export Citation Format

Share Document