scholarly journals Syntax-Informed Self-Attention Network for Span-Based Joint Entity and Relation Extraction

2021 ◽  
Vol 11 (4) ◽  
pp. 1480
Author(s):  
Haiyang Zhang ◽  
Guanqun Zhang ◽  
Ricardo Ma
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Local Context ◽  
Semantic Features ◽  
Global Context ◽  
Contextual Features ◽  
Current State ◽  
Syntactic Features ◽  
Dependency Trees ◽  
Depth Learning

Current state-of-the-art joint entity and relation extraction framework is based on span-level entity classification and relation identification between pairs of entity mentions. However, while maintaining an efficient exhaustive search on spans, the importance of syntactic features is not taken into consideration. It will lead to a problem that the prediction of a relation between two entities is related based on corresponding entity types, but in fact they are not related in the sentence. In addition, although previous works have proven that extract local context is beneficial for the task, it still lacks in-depth learning of contextual features in local context. In this paper, we propose to incorporate syntax knowledge into multi-head self-attention by employing part of heads to focus on syntactic parents of each token from pruned dependency trees, and we use it to model the global context to fuse syntactic and semantic features. In addition, in order to get richer contextual features from the local context, we apply local focus mechanism on entity pairs and corresponding context. Based on applying the two strategies, we perform joint entity and relation extraction on span-level. Experimental results show that our model achieves significant improvements on both Conll04 and SciERC dataset compared to strong competitors.

Multi-Graph Cooperative Learning Towards Distant Supervised Relation Extraction

2021 ◽  
Vol 12 (5) ◽  
pp. 1-21
Author(s):  
Changsen Yuan ◽  
Heyan Huang ◽  
Chong Feng
Keyword(s):  
Cooperative Learning ◽  
State Of The Art ◽  
Relation Extraction ◽  
Sentence Length ◽  
Universal Relation ◽  
Dependency Parsing ◽  
Convolutional Network ◽  
Syntactic Features ◽  
Use Dependency

The Graph Convolutional Network (GCN) is a universal relation extraction method that can predict relations of entity pairs by capturing sentences’ syntactic features. However, existing GCN methods often use dependency parsing to generate graph matrices and learn syntactic features. The quality of the dependency parsing will directly affect the accuracy of the graph matrix and change the whole GCN’s performance. Because of the influence of noisy words and sentence length in the distant supervised dataset, using dependency parsing on sentences causes errors and leads to unreliable information. Therefore, it is difficult to obtain credible graph matrices and relational features for some special sentences. In this article, we present a Multi-Graph Cooperative Learning model (MGCL), which focuses on extracting the reliable syntactic features of relations by different graphs and harnessing them to improve the representations of sentences. We conduct experiments on a widely used real-world dataset, and the experimental results show that our model achieves the state-of-the-art performance of relation extraction.

scholarly journals Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge

2021 ◽  
Author(s):  
Shengqiong Wu ◽  
Hao Fei ◽  
Yafeng Ren ◽  
Donghong Ji ◽  
Jingye Li
Keyword(s):  
State Of The Art ◽  
Boundary Detection ◽  
High Order ◽  
Experimental Results ◽  
Convolutional Network ◽  
Syntactic Knowledge ◽  
Current State ◽  
Syntactic Features ◽  
Benchmark Datasets

In this paper, we propose to enhance the pair-wise aspect and opinion terms extraction (PAOTE) task by incorporating rich syntactic knowledge. We first build a syntax fusion encoder for encoding syntactic features, including a label-aware graph convolutional network (LAGCN) for modeling the dependency edges and labels, as well as the POS tags unifiedly, and a local-attention module encoding POS tags for better term boundary detection. During pairing, we then adopt Biaffine and Triaffine scoring for high-order aspect-opinion term pairing, in the meantime re-harnessing the syntax-enriched representations in LAGCN for syntactic-aware scoring. Experimental results on four benchmark datasets demonstrate that our model outperforms current state-of-the-art baselines, meanwhile yielding explainable predictions with syntactic knowledge.

scholarly journals Evaluating semantometrics from computer science publications

2020 ◽  
Vol 125 (3) ◽  
pp. 2915-2954
Author(s):  
Christin Katharina Kreutz ◽  
Premtim Sahitaj ◽  
Ralf Schenkel
Keyword(s):  
State Of The Art ◽  
Binary Classification ◽  
Citation Network ◽  
Distance Measures ◽  
Semantic Features ◽  
Citation Networks ◽  
Current State ◽  
Publication Time ◽  
Citation Practices ◽  
Vector Representations

AbstractIdentification of important works and assessment of importance of publications in vast scientific corpora are challenging yet common tasks subjected by many research projects. While the influence of citations in finding seminal papers has been analysed thoroughly, citation-based approaches come with several problems. Their impracticality when confronted with new publications which did not yet receive any citations, area-dependent citation practices and different reasons for citing are only a few drawbacks of them. Methods relying on more than citations, for example semantic features such as words or topics contained in publications of citation networks, are regarded with less vigour while providing promising preliminary results. In this work we tackle the issue of classifying publications with their respective referenced and citing papers as either seminal, survey or uninfluential by utilising semantometrics. We use distance measures over words, semantics, topics and publication years of papers in their citation network to engineer features on which we predict the class of a publication. We present the SUSdblp dataset consisting of 1980 labelled entries to provide a means of evaluating this approach. A classification accuracy of up to .9247 was achieved when combining multiple types of features using semantometrics. This is +.1232 compared to the current state of the art (SOTA) which uses binary classification to identify papers from classes seminal and survey. The utilisation of one-vector representations for the ternary classification task resulted in an accuracy of .949 which is +.1475 compared to the binary SOTA. Classification based on information available at publication time derived with semantometrics resulted in an accuracy of .8152 while an accuracy of .9323 could be achieved when using one-vector representations.

scholarly journals LCF: A Local Context Focus Mechanism for Aspect-Based Sentiment Classification

2019 ◽  
Vol 9 (16) ◽  
pp. 3389 ◽  
Author(s):  
Biqing Zeng ◽  
Heng Yang ◽  
Ruyang Xu ◽  
Wu Zhou ◽  
Xuli Han
Keyword(s):  
State Of The Art ◽  
Sentiment Classification ◽  
Experimental Results ◽  
Local Context ◽  
Baseline Model ◽  
Global Context ◽  
Benchmark Datasets ◽  
Art Performance ◽  
Context Features

Aspect-based sentiment classification (ABSC) aims to predict sentiment polarities of different aspects within sentences or documents. Many previous studies have been conducted to solve this problem, but previous works fail to notice the correlation between the aspect’s sentiment polarity and the local context. In this paper, a Local Context Focus (LCF) mechanism is proposed for aspect-based sentiment classification based on Multi-head Self-Attention (MHSA). This mechanism is called LCF design, and utilizes the Context features Dynamic Mask (CDM) and Context Features Dynamic Weighted (CDW) layers to pay more attention to the local context words. Moreover, a BERT-shared layer is adopted to LCF design to capture internal long-term dependencies of local context and global context. Experiments are conducted on three common ABSC datasets: the laptop and restaurant datasets of SemEval-2014 and the ACL twitter dataset. Experimental results demonstrate that the LCF baseline model achieves considerable performance. In addition, we conduct ablation experiments to prove the significance and effectiveness of LCF design. Especially, by incorporating with BERT-shared layer, the LCF-BERT model refreshes state-of-the-art performance on all three benchmark datasets.

scholarly journals Distant Supervision for Relation Extraction with Sentence Selection and Interaction Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Tiantian Chen ◽  
Nianbin Wang ◽  
Hongbin Wang ◽  
Haomin Zhan
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Features ◽  
Distant Supervision ◽  
Word Level ◽  
Proposed Model ◽  
Relation Prediction ◽  
Better Than

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large-scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word-level attention mechanism-based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1-score on a widely used dataset, which proves that our model performs significantly better than the state-of-the-art RE systems.

scholarly journals Multi-View Consistency for Relation Extraction via Mutual Information and Structure Prediction

2020 ◽  
Vol 34 (05) ◽  
pp. 9106-9113
Author(s):  
Amir Veyseh ◽  
Franck Dernoncourt ◽  
My Thai ◽  
Dejing Dou ◽  
Thien Nguyen
Keyword(s):  
Structure Prediction ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Relations ◽  
General Strategy ◽  
Novel Method ◽  
Dependency Trees ◽  
The Common ◽  
The One

Relation Extraction (RE) is one of the fundamental tasks in Information Extraction. The goal of this task is to find the semantic relations between entity mentions in text. It has been shown in many previous work that the structure of the sentences (i.e., dependency trees) can provide important information/features for the RE models. However, the common limitation of the previous work on RE is the reliance on some external parsers to obtain the syntactic trees for the sentence structures. On the one hand, it is not guaranteed that the independent external parsers can offer the optimal sentence structures for RE and the customized structures for RE might help to further improve the performance. On the other hand, the quality of the external parsers might suffer when applied to different domains, thus also affecting the performance of the RE models on such domains. In order to overcome this issue, we introduce a novel method for RE that simultaneously induces the structures and predicts the relations for the input sentences, thus avoiding the external parsers and potentially leading to better sentence structures for RE. Our general strategy to learn the RE-specific structures is to apply two different methods to infer the structures for the input sentences (i.e., two views). We then introduce several mechanisms to encourage the structure and semantic consistencies between these two views so the effective structure and semantic representations for RE can emerge. We perform extensive experiments on the ACE 2005 and SemEval 2010 datasets to demonstrate the advantages of the proposed method, leading to the state-of-the-art performance on such datasets.

scholarly journals Semantic Enhanced Distantly Supervised Relation Extraction via Graph Attention Network

Information ◽  
2020 ◽  
Vol 11 (11) ◽  
pp. 528
Author(s):  
Xiaoye Ouyang ◽  
Shudong Chen ◽  
Rong Wang
Keyword(s):  
New York ◽  
New York Times ◽  
Relation Extraction ◽  
Extraction Methods ◽  
Semantic Features ◽  
Attention Networks ◽  
Word Position ◽  
Supervised Methods ◽  
Dependency Trees ◽  
Type Information

Distantly Supervised relation extraction methods can automatically extract the relation between entity pairs, which are essential for the construction of a knowledge graph. However, the automatically constructed datasets comprise amounts of low-quality sentences and noisy words, and the current Distantly Supervised methods ignore these noisy data, resulting in unacceptable accuracy. To mitigate this problem, we present a novel Distantly Supervised approach SEGRE (Semantic Enhanced Graph attention networks Relation Extraction) for improved relation extraction. Our model first uses word position and entity type information to provide abundant local features and background knowledge. Then it builds the dependency trees to remove noisy words that are irrelevant to relations and employs Graph Attention Networks (GATs) to encode syntactic information, which also captures the important semantic features of relational words in each instance. Furthermore, to make our model more robust against noisy words, the intra-bag attention module is used to weight the bag representation and mitigate noise in the bag. Through extensive experiments on Riedel New York Times (NYT) and Google IISc Distantly Supervised (GIDS) datasets, we demonstrate SEGRE’s effectiveness.

scholarly journals Towards Making the Most of Context in Neural Machine Translation

2020 ◽  
Author(s):  
Zaixiang Zheng ◽  
Xiang Yue ◽  
Shujian Huang ◽  
Jiajun Chen ◽  
Alexandra Birch
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Local Context ◽  
Unified Approach ◽  
Neural Machine Translation ◽  
Global Context ◽  
Level Data ◽  
Sentence Level ◽  
Target Languages ◽  
Document Level

Document-level machine translation manages to outperform sentence level models by a small margin, but have failed to be widely adopted. We argue that previous research did not make a clear use of the global context, and propose a new document-level NMT framework that deliberately models the local context of each sentence with the awareness of the global context of the document in both source and target languages. We specifically design the model to be able to deal with documents containing any number of sentences, including single sentences. This unified approach allows our model to be trained elegantly on standard datasets without needing to train on sentence and document level data separately. Experimental results demonstrate that our model outperforms Transformer baselines and previous document-level NMT models with substantial margins of up to 2.1 BLEU on state-of-the-art baselines. We also provide analyses which show the benefit of context far beyond the neighboring two or three sentences, which previous studies have typically incorporated.

Sign in / Sign up

Export Citation Format

Share Document