scholarly journals Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective

2019 ◽  
Author(s):  
Kaidi Xu ◽  
Hongge Chen ◽  
Sijia Liu ◽  
Pin-Yu Chen ◽  
Tsui-Wei Weng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Classification Performance ◽  
Original Graph ◽  
Graph Data ◽  
Gradient Based ◽  
Significant Performance ◽  
Adversarial Training ◽  
Graph Neural Networks ◽  
Attack And Defense ◽  
Discrete Graph

Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification. However, only few work has addressed the adversarial robustness of GNNs. In this paper, we first present a novel gradient-based attack method that facilitates the difficulty of tackling discrete graph data. When comparing to current adversarial attacks on GNNs, the results show that by only perturbing a small number of edge perturbations, including addition and deletion, our optimization-based attack can lead to a noticeable decrease in classification performance. Moreover, leveraging our gradient-based attack, we propose the first optimization-based adversarial training for GNNs. Our method yields higher robustness against both different gradient based and greedy attack methods without sacrifice classification accuracy on original graph.

scholarly journals Robust Graph Neural Networks Against Adversarial Attacks via Jointly Adversarial Training

2020 ◽  
Vol 53 (5) ◽  
pp. 420-425
Author(s):  
Hu Tian ◽  
Bowei Ye ◽  
Xiaolong Zheng ◽  
Desheng Dash Wu
Keyword(s):  
Neural Networks ◽  
Adversarial Training ◽  
Graph Neural Networks

scholarly journals Multi-View Attribute Graph Convolution Networks for Clustering

2020 ◽  
Author(s):  
Jiafeng Cheng ◽  
Qianqian Wang ◽  
Zhiqiang Tao ◽  
Deyan Xie ◽  
Quanxue Gao
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Graph Embedding ◽  
Structured Data ◽  
Attention Networks ◽  
Graph Data ◽  
Graph Reconstruction ◽  
Node Attributes ◽  
Graph Neural Networks ◽  
Geometric Relationship

Graph neural networks (GNNs) have made considerable achievements in processing graph-structured data. However, existing methods can not allocate learnable weights to different nodes in the neighborhood and lack of robustness on account of neglecting both node attributes and graph reconstruction. Moreover, most of multi-view GNNs mainly focus on the case of multiple graphs, while designing GNNs for solving graph-structured data of multi-view attributes is still under-explored. In this paper, we propose a novel Multi-View Attribute Graph Convolution Networks (MAGCN) model for the clustering task. MAGCN is designed with two-pathway encoders that map graph embedding features and learn the view-consistency information. Specifically, the first pathway develops multi-view attribute graph attention networks to reduce the noise/redundancy and learn the graph embedding features for each multi-view graph data. The second pathway develops consistent embedding encoders to capture the geometric relationship and probability distribution consistency among different views, which adaptively finds a consistent clustering embedding space for multi-view attributes. Experiments on three benchmark graph datasets show the superiority of our method compared with several state-of-the-art algorithms.

scholarly journals Active Learning for Node Classification: An Evaluation

Entropy ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. 1164
Author(s):  
Kaushalya Madhawa ◽  
Tsuyoshi Murata
Keyword(s):  
Neural Networks ◽  
Active Learning ◽  
State Of The Art ◽  
Learning Algorithms ◽  
Classification Performance ◽  
Data Types ◽  
Neural Network Models ◽  
Attributed Graph ◽  
Attributed Graphs ◽  
Graph Neural Networks

Current breakthroughs in the field of machine learning are fueled by the deployment of deep neural network models. Deep neural networks models are notorious for their dependence on large amounts of labeled data for training them. Active learning is being used as a solution to train classification models with less labeled instances by selecting only the most informative instances for labeling. This is especially important when the labeled data are scarce or the labeling process is expensive. In this paper, we study the application of active learning on attributed graphs. In this setting, the data instances are represented as nodes of an attributed graph. Graph neural networks achieve the current state-of-the-art classification performance on attributed graphs. The performance of graph neural networks relies on the careful tuning of their hyperparameters, usually performed using a validation set, an additional set of labeled instances. In label scarce problems, it is realistic to use all labeled instances for training the model. In this setting, we perform a fair comparison of the existing active learning algorithms proposed for graph neural networks as well as other data types such as images and text. With empirical results, we demonstrate that state-of-the-art active learning algorithms designed for other data types do not perform well on graph-structured data. We study the problem within the framework of the exploration-vs.-exploitation trade-off and propose a new count-based exploration term. With empirical evidence on multiple benchmark graphs, we highlight the importance of complementing uncertainty-based active learning models with an exploration term.

scholarly journals Privacy Issues Regarding the Application of DNNs to Activity-Recognition using Wearables and Its Countermeasures by Use of Adversarial Training

2017 ◽  
Author(s):  
Yusuke Iwasawa ◽  
Kotaro Nakayama ◽  
Ikuko Yairi ◽  
Yutaka Matsuo
Keyword(s):  
Neural Networks ◽  
Activity Recognition ◽  
Information Disclosure ◽  
Deep Neural Networks ◽  
Recognition Performance ◽  
Empirical Validation ◽  
Privacy Concerns ◽  
Proposed Model ◽  
Significant Performance ◽  
Adversarial Training

Deep neural networks have been successfully applied to activity recognition with wearables in terms of recognition performance. However, the black-box nature of neural networks could lead to privacy concerns. Namely, generally it is hard to expect what neural networks learn from data, and so they possibly learn features that highly discriminate user-information unintentionally, which increases the risk of information-disclosure. In this study, we analyzed the features learned by conventional deep neural networks when applied to data of wearables to confirm this phenomenon.Based on the results of our analysis, we propose the use of an adversarial training framework to suppress the risk of sensitive/unintended information disclosure. Our proposed model considers both an adversarial user classifier and a regular activity-classifier during training, which allows the model to learn representations that help the classifier to distinguish the activities but which, at the same time, prevents it from accessing user-discriminative information. This paper provides an empirical validation of the privacy issue and efficacy of the proposed method using three activity recognition tasks based on data of wearables. The empirical validation shows that our proposed method suppresses the concerns without any significant performance degradation, compared to conventional deep nets on all three tasks.

scholarly journals Graph-Free Knowledge Distillation for Graph Neural Networks

2021 ◽  
Author(s):  
Xiang Deng ◽  
Zhongfei Zhang
Keyword(s):  
Neural Networks ◽  
Back Propagation ◽  
Multinomial Distribution ◽  
Large Data ◽  
Training Data ◽  
Grid Data ◽  
Continuous Space ◽  
Graph Data ◽  
Knowledge Distillation ◽  
Graph Neural Networks

Knowledge distillation (KD) transfers knowledge from a teacher network to a student by enforcing the student to mimic the outputs of the pretrained teacher on training data. However, data samples are not always accessible in many cases due to large data sizes, privacy, or confidentiality. Many efforts have been made on addressing this problem for convolutional neural networks (CNNs) whose inputs lie in a grid domain within a continuous space such as images and videos, but largely overlook graph neural networks (GNNs) that handle non-grid data with different topology structures within a discrete space. The inherent differences between their inputs make these CNN-based approaches not applicable to GNNs. In this paper, we propose to our best knowledge the first dedicated approach to distilling knowledge from a GNN without graph data. The proposed graph-free KD (GFKD) learns graph topology structures for knowledge transfer by modeling them with multinomial distribution. We then introduce a gradient estimator to optimize this framework. Essentially, the gradients w.r.t. graph structures are obtained by only using GNN forward-propagation without back-propagation, which means that GFKD is compatible with modern GNN libraries such as DGL and Geometric. Moreover, we provide the strategies for handling different types of prior knowledge in the graph data or the GNNs. Extensive experiments demonstrate that GFKD achieves the state-of-the-art performance for distilling knowledge from GNNs without training data.

scholarly journals Scalable Graph Neural Network Training

2021 ◽  
Vol 55 (1) ◽  
pp. 68-76
Author(s):  
Marco Serafini
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Network Architectures ◽  
Neural Network Training ◽  
Large Graphs ◽  
Graph Data ◽  
Network Training ◽  
Neural Network Architectures ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) are a new and increasingly popular family of deep neural network architectures to perform learning on graphs. Training them efficiently is challenging due to the irregular nature of graph data. The problem becomes even more challenging when scaling to large graphs that exceed the capacity of single devices. Standard approaches to distributed DNN training, like data and model parallelism, do not directly apply to GNNs. Instead, two different approaches have emerged in the literature: whole-graph and sample-based training. In this paper, we review and compare the two approaches. Scalability is challenging with both approaches, but we make a case that research should focus on sample-based training since it is a more promising approach. Finally, we review recent systems supporting sample-based training.

scholarly journals Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

2022 ◽  
Vol 40 (4) ◽  
pp. 1-46
Author(s):  
Hao Peng ◽  
Ruitong Zhang ◽  
Yingtong Dou ◽  
Renyu Yang ◽  
Jingyi Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Message Passing ◽  
Representation Learning ◽  
Graph Representations ◽  
Graph Data ◽  
Neighborhood Information ◽  
Neighborhood Selection ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) have been widely used for the representation learning of various structured graph data, typically through message passing among nodes by aggregating their neighborhood information via different operations. While promising, most existing GNNs oversimplify the complexity and diversity of the edges in the graph and thus are inefficient to cope with ubiquitous heterogeneous graphs, which are typically in the form of multi-relational graph representations. In this article, we propose RioGNN , a novel Reinforced, recursive, and flexible neighborhood selection guided multi-relational Graph Neural Network architecture, to navigate complexity of neural network structures whilst maintaining relation-dependent representations. We first construct a multi-relational graph, according to the practical task, to reflect the heterogeneity of nodes, edges, attributes, and labels. To avoid the embedding over-assimilation among different types of nodes, we employ a label-aware neural similarity measure to ascertain the most similar neighbors based on node attributes. A reinforced relation-aware neighbor selection mechanism is developed to choose the most similar neighbors of a targeting node within a relation before aggregating all neighborhood information from different relations to obtain the eventual node embedding. Particularly, to improve the efficiency of neighbor selecting, we propose a new recursive and scalable reinforcement learning framework with estimable depth and width for different scales of multi-relational graphs. RioGNN can learn more discriminative node embedding with enhanced explainability due to the recognition of individual importance of each relation via the filtering threshold mechanism. Comprehensive experiments on real-world graph data and practical tasks demonstrate the advancements of effectiveness, efficiency, and the model explainability, as opposed to other comparative GNN models.

scholarly journals Imitation Learning with Graph Neural Networks for Improving Swarm Robustness under Restricted Communications

2021 ◽  
Vol 11 (19) ◽  
pp. 9055
Author(s):  
Ce Guo ◽  
Pengming Zhu ◽  
Zhiqian Zhou ◽  
Lin Lang ◽  
Zhiwen Zeng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Imitation Learning ◽  
Communication Constraints ◽  
Training Process ◽  
Simulation Experiments ◽  
Graph Data ◽  
Primary Motivation ◽  
Convolutional Networks ◽  
Speed Up ◽  
Graph Neural Networks

This paper focuses on generating distributed flocking strategies via imitation learning. The primary motivation is to improve the swarm robustness and achieve better consistency while respecting the communication constraints. This paper first proposes a quantitative metric of swarm robustness based on entropy evaluation. Then, the graph importance consistency is also proposed, which is one of the critical goals of the flocking task. Moreover, the importance-correlated directed graph convolutional networks (IDGCNs) are constructed for multidimensional feature extraction and structure-related aggregation of graph data. Next, by employing IDGCNs-based imitation learning, a distributed and scalable flocking strategy is obtained, and its performance is very close to the centralized strategy template while considering communication constraints. To speed up and simplify the training process, we train the flocking strategy with a small number of agents and set restrictions on communication. Finally, various simulation experiments are executed to verify the advantages of the obtained strategy in terms of realizing the swarm consistency and improving the swarm robustness. The results also show that the performance is well maintained while the scale of agents expands (tested with 20, 30, 40 robots).

Sign in / Sign up

Export Citation Format

Share Document