scholarly journals Graph convolutional networks: analysis, improvements and results

2021 ◽  
Author(s):  
Ihsan Ullah ◽  
Mario Manzo ◽  
Mitul Shah ◽  
Michael G. Madden
Keyword(s):  
Machine Learning Algorithms ◽  
Complex Structures ◽  
Convolutional Network ◽  
Topological Information ◽  
Computational Costs ◽  
Convolutional Networks ◽  
Complex Organization ◽  
Benchmark Datasets ◽  
Structural Redesign ◽  
Networks Analysis

AbstractA graph can represent a complex organization of data in which dependencies exist between multiple entities or activities. Such complex structures create challenges for machine learning algorithms, particularly when combined with the high dimensionality of data in current applications. Graph convolutional networks were introduced to adopt concepts from deep convolutional networks (i.e. the convolutional operations/layers) that have shown good results. In this context, we propose two major enhancements to two of the existing graph convolutional network frameworks: (1) topological information enrichment through clustering coefficients; and (2) structural redesign of the network through the addition of dense layers. Furthermore, we propose minor enhancements using convex combinations of activation functions and hyper-parameter optimization. We present extensive results on four state-of-art benchmark datasets. We show that our approach achieves competitive results for three of the datasets and state-of-the-art results for the fourth dataset while having lower computational costs compared to competing methods.

scholarly journals Topology Optimization based Graph Convolutional Network

2019 ◽  
Author(s):  
Liang Yang ◽  
Zesheng Kang ◽  
Xiaochun Cao ◽  
Di Jin ◽  
Bo Yang ◽  
...  
Keyword(s):  
Topology Optimization ◽  
Network Topology ◽  
State Of The Art ◽  
Convolutional Network ◽  
Topological Information ◽  
Convolutional Networks ◽  
The Past ◽  
Attributed Network ◽  
Fully Connected ◽  
The Given

In the past few years, semi-supervised node classification in attributed network has been developed rapidly. Inspired by the success of deep learning, researchers adopt the convolutional neural network to develop the Graph Convolutional Networks (GCN), and they have achieved surprising classification accuracy by considering the topological information and employing the fully connected network (FCN). However, the given network topology may also induce a performance degradation if it is directly employed in classification, because it may possess high sparsity and certain noises. Besides, the lack of learnable filters in GCN also limits the performance. In this paper, we propose a novel Topology Optimization based Graph Convolutional Networks (TO-GCN) to fully utilize the potential information by jointly refining the network topology and learning the parameters of the FCN. According to our derivations, TO-GCN is more flexible than GCN, in which the filters are fixed and only the classifier can be updated during the learning process. Extensive experiments on real attributed networks demonstrate the superiority of the proposed TO-GCN against the state-of-the-art approaches.

scholarly journals Encoding Text Information with Graph Convolutional Networks for Personality Recognition

2020 ◽  
Vol 10 (12) ◽  
pp. 4081
Author(s):  
Zhe Wang ◽  
Chun-Hua Wu ◽  
Qing-Biao Li ◽  
Bo Yan ◽  
Kang-Feng Zheng
Keyword(s):  
Personality Traits ◽  
Social Engineering ◽  
Big Five Personality ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Wide Range ◽  
Benchmark Datasets ◽  
Text Information ◽  
Class Labels ◽  
Personality Recognition

Personality recognition is a classic and important problem in social engineering. Due to the small number and particularity of personality recognition databases, only limited research has explored convolutional neural networks for this task. In this paper, we explore the use of graph convolutional network techniques for inferring a user’s personality traits from their Facebook status updates or essay information. Since the basic five personality traits (such as openness) and their aspects (such as status information) are related to a wide range of text features, this work takes the Big Five personality model as the core of the study. We construct a single user personality graph for the corpus based on user-document relations, document-word relations, and word co-occurrence and then learn the personality graph convolutional networks (personality GCN) for the user. The parameters or the inputs of our personality GCN are initialized with a one-hot representation for users, words and documents; then, under the supervision of users and documents with known class labels, it jointly learns the embeddings for users, words, and documents. We used feature information sharing to incorporate the correlation between the five personality traits into personality recognition to perfect the personality GCN. Our experimental results on two public and authoritative benchmark datasets show that the general personality GCN without any external word embeddings or knowledge is superior to the state-of-the-art methods for personality recognition. The personality GCN method is efficient on small datasets, and the average F1-score and accuracy of personality recognition are improved by up to approximately 3.6% and 2.4–2.57%, respectively.

Spatial Relational Attention Using Fully Convolutional Networks for Image Caption Generation

2020 ◽  
Vol 19 (02) ◽  
pp. 2050011
Author(s):  
Teng Jiang ◽  
Liang Gong ◽  
Yupu Yang
Keyword(s):  
Spatial Relations ◽  
Image Features ◽  
Attention Mechanism ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
Benchmark Datasets ◽  
Visual Concepts ◽  
Image Caption Generation ◽  
Image Caption

Attention-based encoder–decoder framework has greatly improved image caption generation tasks. The attention mechanism plays a transitional role by transforming static image features into sequential captions. To generate reasonable captions, it is of great significance to detect spatial characteristics of images. In this paper, we propose a spatial relational attention approach to consider spatial positions and attributes. Image features are firstly weighted by the attention mechanism. Then they are concatenated with contextual features to form a spatial–visual tensor. The tensor is feature extracted by a fully convolutional network to produce visual concepts for the decoder network. The fully convolutional layers maintain spatial topology of images. Experiments conducted on the three benchmark datasets, namely Flickr8k, Flickr30k and MSCOCO, demonstrate the effectiveness of our proposed approach. Captions generated by the spatial relational attention method precisely capture spatial relations of objects.

scholarly journals Gaze Gesture Recognition by Graph Convolutional Networks

2021 ◽  
Vol 8 ◽  
Author(s):  
Lei Shi ◽  
Cosmin Copot ◽  
Steve Vanlanduit
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Eye Tracking ◽  
Gesture Recognition ◽  
Machine Learning Algorithms ◽  
Machine Learning Techniques ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Research Areas ◽  
Artificial Neural Network Ann

Gaze gestures are extensively used in the interactions with agents/computers/robots. Either remote eye tracking devices or head-mounted devices (HMDs) have the advantage of hands-free during the interaction. Previous studies have demonstrated the success of applying machine learning techniques for gaze gesture recognition. More recently, graph neural networks (GNNs) have shown great potential applications in several research areas such as image classification, action recognition, and text classification. However, GNNs are less applied in eye tracking researches. In this work, we propose a graph convolutional network (GCN)–based model for gaze gesture recognition. We train and evaluate the GCN model on the HideMyGaze! dataset. The results show that the accuracy, precision, and recall of the GCN model are 97.62%, 97.18%, and 98.46%, respectively, which are higher than the other compared conventional machine learning algorithms, the artificial neural network (ANN) and the convolutional neural network (CNN).

A gating context-aware text classification model with BERT and graph convolutional networks

2021 ◽  
pp. 1-13
Author(s):  
Weiqi Gao ◽  
Hao Huang
Keyword(s):  
Text Classification ◽  
Structural Data ◽  
Classification Model ◽  
Context Aware ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Gating Mechanism ◽  
Document Frequency ◽  
Proposed Model ◽  
Benchmark Datasets

Graph convolutional networks (GCNs), which are capable of effectively processing graph-structural data, have been successfully applied in text classification task. Existing studies on GCN based text classification model largely concerns with the utilization of word co-occurrence and Term Frequency-Inverse Document Frequency (TF–IDF) information for graph construction, which to some extent ignore the context information of the texts. To solve this problem, we propose a gating context-aware text classification model with Bidirectional Encoder Representations from Transformers (BERT) and graph convolutional network, named as Gating Context GCN (GC-GCN). More specifically, we integrates the graph embedding with BERT embedding by using a GCN with gating mechanism enables the acquisition of context coding. We carry out text classification experiments to show the effectiveness of the proposed model. Experimental results shown our model has respectively obtained 0.19%, 0.57%, 1.05% and 1.17% improvements over the Text-GCN baseline on the 20NG, R8, R52, and Ohsumed benchmark datasets. Furthermore, to overcome the problem that word co-occurrence and TF–IDF are not suitable for graph construction for short texts, Euclidean distance is used to combine with word co-occurrence and TF–IDF information. We obtain an improvement by 1.38% on the MR dataset compared to Text-GCN baseline.

scholarly journals Context Modulated Dynamic Networks for Actor and Action Video Segmentation with Language Queries

2020 ◽  
Vol 34 (07) ◽  
pp. 12152-12159
Author(s):  
Hao Wang ◽  
Cheng Deng ◽  
Fan Ma ◽  
Yi Yang
Keyword(s):  
Video Segmentation ◽  
State Of The Art ◽  
Dynamic Networks ◽  
Specific Region ◽  
Visual Features ◽  
Convolutional Network ◽  
Fine Grained ◽  
Convolutional Networks ◽  
Benchmark Datasets ◽  
Context Features

Actor and action video segmentation with language queries aims to segment out the expression referred objects in the video. This process requires comprehensive language reasoning and fine-grained video understanding. Previous methods mainly leverage dynamic convolutional networks to match visual and semantic representations. However, the dynamic convolution neglects spatial context when processing each region in the frame and is thus challenging to segment similar objects in the complex scenarios. To address such limitation, we construct a context modulated dynamic convolutional network. Specifically, we propose a context modulated dynamic convolutional operation in the proposed framework. The kernels for the specific region are generated from both language sentences and surrounding context features. Moreover, we devise a temporal encoder to incorporate motions into the visual features to further match the query descriptions. Extensive experiments on two benchmark datasets, Actor-Action Dataset Sentences (A2D Sentences) and J-HMDB Sentences, demonstrate that our proposed approach notably outperforms state-of-the-art methods.

scholarly journals Attention-Enhanced Graph Convolutional Networks for Aspect-Based Sentiment Classification with Multi-Head Attention

2021 ◽  
Vol 11 (8) ◽  
pp. 3640
Author(s):  
Guangtao Xu ◽  
Peiyu Liu ◽  
Zhenfang Zhu ◽  
Jie Liu ◽  
Fuyong Xu
Keyword(s):  
Sentiment Classification ◽  
Experimental Results ◽  
Sentence Structure ◽  
Convolutional Network ◽  
Structure Information ◽  
Convolutional Networks ◽  
Dependency Tree ◽  
Syntactic Information ◽  
Benchmark Datasets ◽  
Reasonable Use

The purpose of aspect-based sentiment classification is to identify the sentiment polarity of each aspect in a sentence. Recently, due to the introduction of Graph Convolutional Networks (GCN), more and more studies have used sentence structure information to establish the connection between aspects and opinion words. However, the accuracy of these methods is limited by noise information and dependency tree parsing performance. To solve this problem, we proposed an attention-enhanced graph convolutional network (AEGCN) for aspect-based sentiment classification with multi-head attention (MHA). Our proposed method can better combine semantic and syntactic information by introducing MHA and GCN. We also added an attention mechanism to GCN to enhance its performance. In order to verify the effectiveness of our proposed method, we conducted a lot of experiments on five benchmark datasets. The experimental results show that our proposed method can make more reasonable use of semantic and syntactic information, and further improve the performance of GCN.

scholarly journals Part-Level Graph Convolutional Network for Skeleton-Based Action Recognition

2020 ◽  
Vol 34 (07) ◽  
pp. 11045-11052
Author(s):  
Linjiang Huang ◽  
Yan Huang ◽  
Wanli Ouyang ◽  
Liang Wang
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
The State ◽  
Body Parts ◽  
Convolutional Network ◽  
Joint Level ◽  
Convolutional Networks ◽  
Level Information ◽  
Benchmark Datasets ◽  
High Level

Recently, graph convolutional networks have achieved remarkable performance for skeleton-based action recognition. In this work, we identify a problem posed by the GCNs for skeleton-based action recognition, namely part-level action modeling. To address this problem, a novel Part-Level Graph Convolutional Network (PL-GCN) is proposed to capture part-level information of skeletons. Different from previous methods, the partition of body parts is learnable rather than manually defined. We propose two part-level blocks, namely Part Relation block (PR block) and Part Attention block (PA block), which are achieved by two differentiable operations, namely graph pooling operation and graph unpooling operation. The PR block aims at learning high-level relations between body parts while the PA block aims at highlighting the important body parts in the action. Integrating the original GCN with the two blocks, the PL-GCN can learn both part-level and joint-level information of the action. Extensive experiments on two benchmark datasets show the state-of-the-art performance on skeleton-based action recognition and demonstrate the effectiveness of the proposed method.

scholarly journals Efficient End-to-End Sentence-Level Lipreading with Temporal Convolutional Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 6975
Author(s):  
Tao Zhang ◽  
Lun He ◽  
Xudong Li ◽  
Guoqing Feng
Keyword(s):  
Performance Improvement ◽  
State Of The Art ◽  
Error Rates ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Sentence Level ◽  
End To End ◽  
High Level ◽  
Improved Accuracy ◽  
Talking Face

Lipreading aims to recognize sentences being spoken by a talking face. In recent years, the lipreading method has achieved a high level of accuracy on large datasets and made breakthrough progress. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data and have the defects of disappearing training gradient and slow convergence. To overcome these problems, we proposed an efficient end-to-end sentence-level lipreading model, using an encoder based on a 3D convolutional network, ResNet50, Temporal Convolutional Network (TCN), and a CTC objective function as the decoder. More importantly, the proposed architecture incorporates TCN as a feature learner to decode feature. It can partly eliminate the defects of RNN (LSTM, GRU) gradient disappearance and insufficient performance, and this yields notable performance improvement as well as faster convergence. Experiments show that the training and convergence speed are 50% faster than the state-of-the-art method, and improved accuracy by 2.4% on the GRID dataset.

scholarly journals An IoT-Focused Intrusion Detection System Approach Based on Preprocessing Characterization for Cybersecurity Datasets

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 656
Author(s):  
Xavier Larriva-Novo ◽  
Víctor A. Villagrá ◽  
Mario Vega-Barbas ◽  
Diego Rivera ◽  
Mario Sanz Rodrigo
Keyword(s):  
Machine Learning ◽  
Intrusion Detection ◽  
High Performance ◽  
Learning Algorithm ◽  
Detection System ◽  
Machine Learning Algorithms ◽  
Statistical Characteristics ◽  
Detection Techniques ◽  
Traffic Characteristics ◽  
Benchmark Datasets

Security in IoT networks is currently mandatory, due to the high amount of data that has to be handled. These systems are vulnerable to several cybersecurity attacks, which are increasing in number and sophistication. Due to this reason, new intrusion detection techniques have to be developed, being as accurate as possible for these scenarios. Intrusion detection systems based on machine learning algorithms have already shown a high performance in terms of accuracy. This research proposes the study and evaluation of several preprocessing techniques based on traffic categorization for a machine learning neural network algorithm. This research uses for its evaluation two benchmark datasets, namely UGR16 and the UNSW-NB15, and one of the most used datasets, KDD99. The preprocessing techniques were evaluated in accordance with scalar and normalization functions. All of these preprocessing models were applied through different sets of characteristics based on a categorization composed by four groups of features: basic connection features, content characteristics, statistical characteristics and finally, a group which is composed by traffic-based features and connection direction-based traffic characteristics. The objective of this research is to evaluate this categorization by using various data preprocessing techniques to obtain the most accurate model. Our proposal shows that, by applying the categorization of network traffic and several preprocessing techniques, the accuracy can be enhanced by up to 45%. The preprocessing of a specific group of characteristics allows for greater accuracy, allowing the machine learning algorithm to correctly classify these parameters related to possible attacks.

Sign in / Sign up

Export Citation Format

Share Document