scholarly journals Graph Convolutional Networks with Bidirectional Attention for Aspect-Based Sentiment Classification

2021 ◽  
Vol 11 (4) ◽  
pp. 1528
Author(s):  
Jie Liu ◽  
Peiyu Liu ◽  
Zhenfang Zhu ◽  
Xiaowen Li ◽  
Guangtao Xu
Keyword(s):  
Attention Mechanism ◽  
Sentiment Classification ◽  
Long Distance ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Syntactic Information ◽  
Position Coding ◽  
Novel Method

Aspect-based sentiment classification aims at determining the corresponding sentiment of a particular aspect. Many sophisticated approaches, such as attention mechanisms and Graph Convolutional Networks, have been widely used to address this challenge. However, most of the previous methods have not well analyzed the role of words and long-distance dependencies, and the interaction between context and aspect terms is not well realized, which greatly limits the effectiveness of the model. In this paper, we propose an effective and novel method using attention mechanism and graph convolutional network (ATGCN). Firstly, we make full use of multi-head attention and point-wise convolution transformation to obtain the hidden state. Secondly, we introduce position coding in the model, and use Graph Convolutional Networks to obtain syntactic information and long-distance dependencies. Finally, the interaction between context and aspect terms is further realized by bidirectional attention. Experiments on three benchmarking collections indicate the effectiveness of ATGCN.

scholarly journals Attention-Enhanced Graph Convolutional Networks for Aspect-Based Sentiment Classification with Multi-Head Attention

2021 ◽  
Vol 11 (8) ◽  
pp. 3640
Author(s):  
Guangtao Xu ◽  
Peiyu Liu ◽  
Zhenfang Zhu ◽  
Jie Liu ◽  
Fuyong Xu
Keyword(s):  
Sentiment Classification ◽  
Experimental Results ◽  
Sentence Structure ◽  
Convolutional Network ◽  
Structure Information ◽  
Convolutional Networks ◽  
Dependency Tree ◽  
Syntactic Information ◽  
Benchmark Datasets ◽  
Reasonable Use

The purpose of aspect-based sentiment classification is to identify the sentiment polarity of each aspect in a sentence. Recently, due to the introduction of Graph Convolutional Networks (GCN), more and more studies have used sentence structure information to establish the connection between aspects and opinion words. However, the accuracy of these methods is limited by noise information and dependency tree parsing performance. To solve this problem, we proposed an attention-enhanced graph convolutional network (AEGCN) for aspect-based sentiment classification with multi-head attention (MHA). Our proposed method can better combine semantic and syntactic information by introducing MHA and GCN. We also added an attention mechanism to GCN to enhance its performance. In order to verify the effectiveness of our proposed method, we conducted a lot of experiments on five benchmark datasets. The experimental results show that our proposed method can make more reasonable use of semantic and syntactic information, and further improve the performance of GCN.

scholarly journals Targeted Sentiment Classification Based on Attentional Encoding and Graph Convolutional Networks

2020 ◽  
Vol 10 (3) ◽  
pp. 957 ◽  
Author(s):  
Luwei Xiao ◽  
Xiaohui Hu ◽  
Yinong Chen ◽  
Yun Xue ◽  
Donghong Gu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Semantic Information ◽  
Sentiment Classification ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Dependency Tree ◽  
Proposed Model ◽  
Syntactic Information ◽  
Specific Goal ◽  
State Of Art

Targeted sentiment classification aims to predict the emotional trend of a specific goal. Currently, most methods (e.g., recurrent neural networks and convolutional neural networks combined with an attention mechanism) are not able to fully capture the semantic information of the context and they also lack a mechanism to explain the relevant syntactical constraints and long-range word dependencies. Therefore, syntactically irrelevant context words may mistakenly be recognized as clues to predict the target sentiment. To tackle these problems, this paper considers that the semantic information, syntactic information, and their interaction information are very crucial to targeted sentiment analysis, and propose an attentional-encoding-based graph convolutional network (AEGCN) model. Our proposed model is mainly composed of multi-head attention and an improved graph convolutional network built over the dependency tree of a sentence. Pre-trained BERT is applied to this task, and new state-of-art performance is achieved. Experiments on five datasets show the effectiveness of the model proposed in this paper compared with a series of the latest models.

scholarly journals Sentence Compression Using BERT and Graph Convolutional Networks

2021 ◽  
Vol 11 (21) ◽  
pp. 9910
Author(s):  
Yo-Han Park ◽  
Gyong-Ho Lee ◽  
Yong-Seok Choi ◽  
Kong-Joo Lee
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Compression Model ◽  
Sentence Compression ◽  
Dependency Tree ◽  
Proposed Model ◽  
Syntactic Information ◽  
Input Sentence

Sentence compression is a natural language-processing task that produces a short paraphrase of an input sentence by deleting words from the input sentence while ensuring grammatical correctness and preserving meaningful core information. This study introduces a graph convolutional network (GCN) into a sentence compression task to encode syntactic information, such as dependency trees. As we upgrade the GCN to activate a directed edge, the compression model with the GCN layers can distinguish between parent and child nodes in a dependency tree when aggregating adjacent nodes. Furthermore, by increasing the number of GCN layers, the model can gradually collect high-order information of a dependency tree when propagating node information through the layers. We implement a sentence compression model for Korean and English, respectively. This model consists of three components: pre-trained BERT model, GCN layers, and a scoring layer. The scoring layer can determine whether a word should remain in a compressed sentence by relying on the word vector containing contextual and syntactic information encoded by BERT and GCN layers. To train and evaluate the proposed model, we used the Google sentence compression dataset for English and a Korean sentence compression corpus containing about 140,000 sentence pairs for Korean. The experimental results demonstrate that the proposed model achieves state-of-the-art performance for English. To the best of our knowledge, this sentence compression model based on the deep learning model trained with a large-scale corpus is the first attempt for Korean.

scholarly journals Augmenting Paraphrase Generation with Syntax Information Using Graph Convolutional Networks

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 566
Author(s):  
Xiaoqiang Chi ◽  
Yang Xiang
Keyword(s):  
Language Processing ◽  
Neural Nets ◽  
Linguistic Knowledge ◽  
Convolutional Network ◽  
Tree Structures ◽  
Convolutional Networks ◽  
Sequence Modeling ◽  
Syntactic Information ◽  
Dependency Trees ◽  
Paraphrase Generation

Paraphrase generation is an important yet challenging task in natural language processing. Neural network-based approaches have achieved remarkable success in sequence-to-sequence learning. Previous paraphrase generation work generally ignores syntactic information regardless of its availability, with the assumption that neural nets could learn such linguistic knowledge implicitly. In this work, we make an endeavor to probe into the efficacy of explicit syntactic information for the task of paraphrase generation. Syntactic information can appear in the form of dependency trees, which could be easily acquired from off-the-shelf syntactic parsers. Such tree structures could be conveniently encoded via graph convolutional networks to obtain more meaningful sentence representations, which could improve generated paraphrases. Through extensive experiments on four paraphrase datasets with different sizes and genres, we demonstrate the utility of syntactic information in neural paraphrase generation under the framework of sequence-to-sequence modeling. Specifically, our graph convolutional network-enhanced models consistently outperform their syntax-agnostic counterparts using multiple evaluation metrics.

scholarly journals Attention-Based Graph Convolutional Network for Zero-Shot Learning with Pre-Training

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Xuefei Wu ◽  
Mingjiang Liu ◽  
Bo Xin ◽  
Zhangqing Zhu ◽  
Gang Wang
Keyword(s):  
Loss Function ◽  
Attention Mechanism ◽  
Propagation Model ◽  
Knowledge Graph ◽  
Learning Paradigm ◽  
Convolutional Network ◽  
Dense Graph ◽  
Convolutional Networks ◽  
Central Node ◽  
Graph Propagation

Zero-shot learning (ZSL) is a powerful and promising learning paradigm for classifying instances that have not been seen in training. Although graph convolutional networks (GCNs) have recently shown great potential for the ZSL tasks, these models cannot adjust the constant connection weights between the nodes in knowledge graph and the neighbor nodes contribute equally to classify the central node. In this study, we apply an attention mechanism to adjust the connection weights adaptively to learn more important information for classifying unseen target nodes. First, we propose an attention graph convolutional network for zero-shot learning (AGCNZ) by integrating the attention mechanism and GCN directly. Then, in order to prevent the dilution of knowledge from distant nodes, we apply the dense graph propagation (DGP) model for the ZSL tasks and propose an attention dense graph propagation model for zero-shot learning (ADGPZ). Finally, we propose a modified loss function with a relaxation factor to further improve the performance of the learned classifier. Experimental results under different pre-training settings verified the effectiveness of the proposed attention-based models for ZSL.

Spatial Relational Attention Using Fully Convolutional Networks for Image Caption Generation

2020 ◽  
Vol 19 (02) ◽  
pp. 2050011
Author(s):  
Teng Jiang ◽  
Liang Gong ◽  
Yupu Yang
Keyword(s):  
Spatial Relations ◽  
Image Features ◽  
Attention Mechanism ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
Benchmark Datasets ◽  
Visual Concepts ◽  
Image Caption Generation ◽  
Image Caption

Attention-based encoder–decoder framework has greatly improved image caption generation tasks. The attention mechanism plays a transitional role by transforming static image features into sequential captions. To generate reasonable captions, it is of great significance to detect spatial characteristics of images. In this paper, we propose a spatial relational attention approach to consider spatial positions and attributes. Image features are firstly weighted by the attention mechanism. Then they are concatenated with contextual features to form a spatial–visual tensor. The tensor is feature extracted by a fully convolutional network to produce visual concepts for the decoder network. The fully convolutional layers maintain spatial topology of images. Experiments conducted on the three benchmark datasets, namely Flickr8k, Flickr30k and MSCOCO, demonstrate the effectiveness of our proposed approach. Captions generated by the spatial relational attention method precisely capture spatial relations of objects.

scholarly journals An Attention-Enhanced Multi-Scale and Dual Sign Language Recognition Network Based on a Graph Convolution Network

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1120
Author(s):  
Lu Meng ◽  
Ronghui Li
Keyword(s):  
Sign Language ◽  
Attention Mechanism ◽  
Language Recognition ◽  
Sign Language Recognition ◽  
Accuracy Rate ◽  
Long Distance ◽  
Convolutional Network ◽  
Attention Network ◽  
Multi Scale ◽  
Human Finger

Sign language is the most important way of communication for hearing-impaired people. Research on sign language recognition can help normal people understand sign language. We reviewed the classic methods of sign language recognition, and the recognition accuracy is not high enough because of redundant information, human finger occlusion, motion blurring, the diversified signing styles of different people, and so on. To overcome these shortcomings, we propose a multi-scale and dual sign language recognition Network (SLR-Net) based on a graph convolutional network (GCN). The original input data was RGB videos. We first extracted the skeleton data from them and then used the skeleton data for sign language recognition. SLR-Net is mainly composed of three sub-modules: multi-scale attention network (MSA), multi-scale spatiotemporal attention network (MSSTA) and attention enhanced temporal convolution network (ATCN). MSA allows the GCN to learn the dependencies between long-distance vertices; MSSTA can directly learn the spatiotemporal features; ATCN allows the GCN network to better learn the long temporal dependencies. The three different attention mechanisms, multi-scale attention mechanism, spatiotemporal attention mechanism, and temporal attention mechanism, are proposed to further improve the robustness and accuracy. Besides, a keyframe extraction algorithm is proposed, which can greatly improve efficiency by sacrificing a little accuracy. Experimental results showed that our method can reach 98.08% accuracy rate in the CSL-500 dataset with a 500-word vocabulary. Even on the challenging dataset DEVISIGN-L with a 2000-word vocabulary, it also reached a 64.57% accuracy rate, outperforming other state-of-the-art sign language recognition methods.

scholarly journals KGGCN: Knowledge-Guided Graph Convolutional Networks for Distantly Supervised Relation Extraction

2021 ◽  
Vol 11 (16) ◽  
pp. 7734
Author(s):  
Ningyi Mao ◽  
Wenti Huang ◽  
Hai Zhong
Keyword(s):  
Prior Knowledge ◽  
Structural Information ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Knowledge Graph ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Lexical Resource ◽  
Sentence Level ◽  
The Impact

Distantly supervised relation extraction is the most popular technique for identifying semantic relation between two entities. Most prior models only focus on the supervision information present in training sentences. In addition to training sentences, external lexical resource and knowledge graphs often contain other relevant prior knowledge. However, relation extraction models usually ignore such readily available information. Moreover, previous works only utilize a selective attention mechanism over sentences to alleviate the impact of noise, they lack the consideration of the implicit interaction between sentences with relation facts. In this paper, (1) a knowledge-guided graph convolutional network is proposed based on the word-level attention mechanism to encode the sentences. It can capture the key words and cue phrases to generate expressive sentence-level features by attending to the relation indicators obtained from the external lexical resource. (2) A knowledge-guided sentence selector is proposed, which explores the semantic and structural information of triples from knowledge graph as sentence-level knowledge attention to distinguish the importance of each individual sentence. Experimental results on two widely used datasets, NYT-FB and GDS, show that our approach is able to efficiently use the prior knowledge from the external lexical resource and knowledge graph to enhance the performance of distantly supervised relation extraction.

scholarly journals Hyperspectral Images Classification Based on Dense Convolutional Networks with Spectral-Wise Attention Mechanism

2019 ◽  
Vol 11 (2) ◽  
pp. 159 ◽  
Author(s):  
Bei Fang ◽  
Ying Li ◽  
Haokui Zhang ◽  
Jonathan Chan
Keyword(s):  
Classification Performance ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Hyperspectral Images ◽  
Spectral Features ◽  
Competitive Performance ◽  
Feature Maps ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Spatial Features

Hyperspectral images (HSIs) data that is typically presented in 3-D format offers an opportunity for 3-D networks to extract spectral and spatial features simultaneously. In this paper, we propose a novel end-to-end 3-D dense convolutional network with spectral-wise attention mechanism (MSDN-SA) for HSI classification. The proposed MSDN-SA exploits 3-D dilated convolutions to simultaneously capture the spectral and spatial features at different scales, and densely connects all 3-D feature maps with each other. In addition, a spectral-wise attention mechanism is introduced to enhance the distinguishability of spectral features, which improves the classification performance of the trained models. Experimental results on three HSI datasets demonstrate that our MSDN-SA achieves competitive performance for HSI classification.

Lexical attention and aspect-oriented graph convolutional networks for aspect-based sentiment analysis

2021 ◽  
pp. 1-12
Author(s):  
Wenwen Li ◽  
Shiqun Yin ◽  
Ting Pu
Keyword(s):  
Sentiment Analysis ◽  
Oriented Graph ◽  
Attention Mechanism ◽  
Experimental Results ◽  
The Other ◽  
Convolutional Networks ◽  
Other Hand ◽  
Syntactic Information ◽  
Benchmark Datasets ◽  
Syntactic Dependencies

 The purpose of aspect-based sentiment analysis is to predict the sentiment polarity of different aspects in a text. In previous work, while attention has been paid to the use of Graph Convolutional Networks (GCN) to encode syntactic dependencies in order to exploit syntactic information, previous models have tended to confuse opinion words from different aspects due to the complexity of language and the diversity of aspects. On the other hand, the effect of word lexicality on aspects’ sentiment polarity judgments has not been considered in previous studies. In this paper, we propose lexical attention and aspect-oriented GCN to solve the above problems. First, we construct an aspect-oriented dependency-parsed tree by analyzing and pruning the dependency-parsed tree of the sentence, then use the lexical attention mechanism to focus on the features of the lexical properties that play a key role in determining the sentiment polarity, and finally extract the aspect-oriented lexical weighted features by a GCN.Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach.

Sign in / Sign up

Export Citation Format

Share Document