scholarly journals Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

2022 ◽  
Vol 40 (4) ◽  
pp. 1-46
Author(s):  
Hao Peng ◽  
Ruitong Zhang ◽  
Yingtong Dou ◽  
Renyu Yang ◽  
Jingyi Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Message Passing ◽  
Representation Learning ◽  
Graph Representations ◽  
Graph Data ◽  
Neighborhood Information ◽  
Neighborhood Selection ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) have been widely used for the representation learning of various structured graph data, typically through message passing among nodes by aggregating their neighborhood information via different operations. While promising, most existing GNNs oversimplify the complexity and diversity of the edges in the graph and thus are inefficient to cope with ubiquitous heterogeneous graphs, which are typically in the form of multi-relational graph representations. In this article, we propose RioGNN , a novel Reinforced, recursive, and flexible neighborhood selection guided multi-relational Graph Neural Network architecture, to navigate complexity of neural network structures whilst maintaining relation-dependent representations. We first construct a multi-relational graph, according to the practical task, to reflect the heterogeneity of nodes, edges, attributes, and labels. To avoid the embedding over-assimilation among different types of nodes, we employ a label-aware neural similarity measure to ascertain the most similar neighbors based on node attributes. A reinforced relation-aware neighbor selection mechanism is developed to choose the most similar neighbors of a targeting node within a relation before aggregating all neighborhood information from different relations to obtain the eventual node embedding. Particularly, to improve the efficiency of neighbor selecting, we propose a new recursive and scalable reinforcement learning framework with estimable depth and width for different scales of multi-relational graphs. RioGNN can learn more discriminative node embedding with enhanced explainability due to the recognition of individual importance of each relation via the filtering threshold mechanism. Comprehensive experiments on real-world graph data and practical tasks demonstrate the advancements of effectiveness, efficiency, and the model explainability, as opposed to other comparative GNN models.

scholarly journals Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

2020 ◽  
Author(s):  
Douglas Meneghetti ◽  
Reinaldo Bianchi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Communication Channels ◽  
Neural Network Architecture ◽  
Graph Representations ◽  
Labeled Graph ◽  
Multiple Agent ◽  
Multi Agent ◽  
Graph Neural Networks

This work proposes a neural network architecture that learns policies for multiple agent classes in a heterogeneous multi-agent reinforcement setting. The proposed network uses directed labeled graph representations for states, encodes feature vectors of different sizes for different entity classes, uses relational graph convolution layers to model different communication channels between entity types and learns distinct policies for different agent classes, sharing parameters wherever possible. Results have shown that specializing the communication channels between entity classes is a promising step to achieve higher performance in environments composed of heterogeneous entities.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks

2021 ◽  
Author(s):  
Jing Huang ◽  
Jie Yang
Keyword(s):  
Neural Networks ◽  
Message Passing ◽  
State Of The Art ◽  
Representation Learning ◽  
Graph Representation ◽  
Challenging Problem ◽  
Unified Framework ◽  
Real World Datasets ◽  
Graph Neural Networks ◽  
Research Domains

Hypergraph, an expressive structure with flexibility to model the higher-order correlations among entities, has recently attracted increasing attention from various research domains. Despite the success of Graph Neural Networks (GNNs) for graph representation learning, how to adapt the powerful GNN-variants directly into hypergraphs remains a challenging problem. In this paper, we propose UniGNN, a unified framework for interpreting the message passing process in graph and hypergraph neural networks, which can generalize general GNN models into hypergraphs. In this framework, meticulously-designed architectures aiming to deepen GNNs can also be incorporated into hypergraphs with the least effort. Extensive experiments have been conducted to demonstrate the effectiveness of UniGNN on multiple real-world datasets, which outperform the state-of-the-art approaches with a large margin. Especially for the DBLP dataset, we increase the accuracy from 77.4% to 88.8% in the semi-supervised hypernode classification task. We further prove that the proposed message-passing based UniGNN models are at most as powerful as the 1-dimensional Generalized Weisfeiler-Leman (1-GWL) algorithm in terms of distinguishing non-isomorphic hypergraphs. Our code is available at https://github.com/OneForward/UniGNN.

scholarly journals GSSNN: Graph Smoothing Splines Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 7007-7014
Author(s):  
Shichao Zhu ◽  
Lewei Zhou ◽  
Shirui Pan ◽  
Chuan Zhou ◽  
Guiying Yan ◽  
...  
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Representation Learning ◽  
Smoothing Splines ◽  
Graph Representation ◽  
Graph Classification ◽  
Topological Information ◽  
Graph Representations ◽  
The Arts ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in many graph data analysis tasks. However, they still suffer from two limitations for graph representation learning. First, they exploit non-smoothing node features which may result in suboptimal embedding and degenerated performance for graph classification. Second, they only exploit neighbor information but ignore global topological knowledge. Aiming to overcome these limitations simultaneously, in this paper, we propose a novel, flexible, and end-to-end framework, Graph Smoothing Splines Neural Networks (GSSNN), for graph classification. By exploiting the smoothing splines, which are widely used to learn smoothing fitting function in regression, we develop an effective feature smoothing and enhancement module Scaled Smoothing Splines (S3) to learn graph embedding. To integrate global topological information, we design a novel scoring module, which exploits closeness, degree, as well as self-attention values, to select important node features as knots for smoothing splines. These knots can be potentially used for interpreting classification results. In extensive experiments on biological and social datasets, we demonstrate that our model achieves state-of-the-arts and GSSNN is superior in learning more robust graph representations. Furthermore, we show that S3 module is easily plugged into existing GNNs to improve their performance.

scholarly journals Rumor Detection Based on SAGNN: Simplified Aggregation Graph Neural Networks

2021 ◽  
Vol 3 (1) ◽  
pp. 84-94
Author(s):  
Liang Zhang ◽  
Jingqun Li ◽  
Bin Zhou ◽  
Yan Jia
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Network Architectures ◽  
Neural Network Architecture ◽  
Convolutional Networks ◽  
Effective Manner ◽  
Deep Learning Network ◽  
Graph Neural Networks ◽  
Rumor Detection

Identifying fake news on media has been an important issue. This is especially true considering the wide spread of rumors on popular social networks such as Twitter. Various kinds of techniques have been proposed for automatic rumor detection. In this work, we study the application of graph neural networks for rumor classification at a lower level, instead of applying existing neural network architectures to detect rumors. The responses to true rumors and false rumors display distinct characteristics. This suggests that it is essential to capture such interactions in an effective manner for a deep learning network to achieve better rumor detection performance. To this end we present a simplified aggregation graph neural network architecture. Experiments on publicly available Twitter datasets demonstrate that the proposed network has performance on a par with or even better than that of state-of-the-art graph convolutional networks, while significantly reducing the computational complexity.

scholarly journals TSI-GNN: Extending Graph Neural Networks to Handle Missing Data in Temporal Settings

2021 ◽  
Vol 4 ◽  
Author(s):  
David Gordon ◽  
Panayiotis Petousis ◽  
Henry Zheng ◽  
Davina Zamanzadeh ◽  
Alex A.T. Bui
Keyword(s):  
Neural Networks ◽  
Missing Data ◽  
Bipartite Graph ◽  
Message Passing ◽  
Representation Learning ◽  
Temporal Information ◽  
Graph Representation ◽  
Sequence Information ◽  
Novel Approach ◽  
Graph Neural Networks

We present a novel approach for imputing missing data that incorporates temporal information into bipartite graphs through an extension of graph representation learning. Missing data is abundant in several domains, particularly when observations are made over time. Most imputation methods make strong assumptions about the distribution of the data. While novel methods may relax some assumptions, they may not consider temporality. Moreover, when such methods are extended to handle time, they may not generalize without retraining. We propose using a joint bipartite graph approach to incorporate temporal sequence information. Specifically, the observation nodes and edges with temporal information are used in message passing to learn node and edge embeddings and to inform the imputation task. Our proposed method, temporal setting imputation using graph neural networks (TSI-GNN), captures sequence information that can then be used within an aggregation function of a graph neural network. To the best of our knowledge, this is the first effort to use a joint bipartite graph approach that captures sequence information to handle missing data. We use several benchmark datasets to test the performance of our method against a variety of conditions, comparing to both classic and contemporary methods. We further provide insight to manage the size of the generated TSI-GNN model. Through our analysis we show that incorporating temporal information into a bipartite graph improves the representation at the 30% and 60% missing rate, specifically when using a nonlinear model for downstream prediction tasks in regularly sampled datasets and is competitive with existing temporal methods under different scenarios.

scholarly journals Scalable Graph Neural Network Training

2021 ◽  
Vol 55 (1) ◽  
pp. 68-76
Author(s):  
Marco Serafini
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Network Architectures ◽  
Neural Network Training ◽  
Large Graphs ◽  
Graph Data ◽  
Network Training ◽  
Neural Network Architectures ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) are a new and increasingly popular family of deep neural network architectures to perform learning on graphs. Training them efficiently is challenging due to the irregular nature of graph data. The problem becomes even more challenging when scaling to large graphs that exceed the capacity of single devices. Standard approaches to distributed DNN training, like data and model parallelism, do not directly apply to GNNs. Instead, two different approaches have emerged in the literature: whole-graph and sample-based training. In this paper, we review and compare the two approaches. Scalability is challenging with both approaches, but we make a case that research should focus on sample-based training since it is a more promising approach. Finally, we review recent systems supporting sample-based training.

scholarly journals Communicative Representation Learning on Attributed Molecular Graphs

2020 ◽  
Author(s):  
Ying Song ◽  
Shuangjia Zheng ◽  
Zhangming Niu ◽  
Zhang-hua Fu ◽  
Yutong Lu ◽  
...  
Keyword(s):  
Neural Network ◽  
Message Passing ◽  
Molecular Graph ◽  
Representation Learning ◽  
Generation Process ◽  
Molecular Graphs ◽  
Graph Modeling ◽  
Proposed Model ◽  
The One ◽  
Graph Neural Networks

Constructing proper representations of molecules lies at the core of numerous tasks such as molecular property prediction and drug design. Graph neural networks, especially message passing neural network (MPNN) and its variants, have recently made remarkable achievements in molecular graph modeling. Albeit powerful, the one-sided focuses on atom (node) or bond (edge) information of existing MPNN methods lead to the insufficient representations of the attributed molecular graphs. Herein, we propose a Communicative Message Passing Neural Network (CMPNN) to improve the molecular embedding by strengthening the message interactions between nodes and edges through a communicative kernel. In addition, the message generation process is enriched by introducing a new message booster module. Extensive experiments demonstrated that the proposed model obtained superior performances against state-of-the-art baselines on six chemical property datasets. Further visualization also showed better representation capacity of our model.

scholarly journals Rumor Detection Based on SAGNN: Simplified Aggregation Graph Neural Networks

2020 ◽  
Author(s):  
Liang Zhang ◽  
Jingqun Li ◽  
Bin Zhou ◽  
Yan Jia
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Network Architectures ◽  
Convolutional Networks ◽  
Effective Manner ◽  
The Media ◽  
Deep Learning Network ◽  
Graph Neural Networks ◽  
Rumor Detection

Identifying fake news on the media has been an important issue. This is especially true considering the wide spread of rumors on the popular social networks such as Twitter. Various kinds of techniques have been proposed for automatic rumor detection. In this work, we study the application of graph neural networks for rumor classification at a lower level, instead of applying existing neural network architectures to detect rumors. The responses to true rumors and false rumors display distinct characteristics. This suggests that it is essential to capture such interactions in an effective manner for a deep learning network to achieve better rumor detection performance. To this end we present a simplified aggregation graph neural network architecture. Experiments on publicly available Twitter datasets demonstrate that the proposed network has performance on a par with or even better than that of state-of-the-art graph convolutional networks, while significantly reducing the computational complexity.

scholarly journals Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation

2020 ◽  
Author(s):  
Shuo Zhang ◽  
Lei Xie
Keyword(s):  
Neural Networks ◽  
Theoretical Analysis ◽  
Message Passing ◽  
Representation Learning ◽  
Attention Mechanism ◽  
Structured Data ◽  
Clear Understanding ◽  
Graph Classification ◽  
Competitive Performance ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) are powerful for the representation learning of graph-structured data. Most of the GNNs use a message-passing scheme, where the embedding of a node is iteratively updated by aggregating the information from its neighbors. To achieve a better expressive capability of node influences, attention mechanism has grown to be popular to assign trainable weights to the nodes in aggregation. Though the attention-based GNNs have achieved remarkable results in various tasks, a clear understanding of their discriminative capacities is missing. In this work, we present a theoretical analysis of the representational properties of the GNN that adopts the attention mechanism as an aggregator. Our analysis determines all cases when those attention-based GNNs can always fail to distinguish certain distinct structures. Those cases appear due to the ignorance of cardinality information in attention-based aggregation. To improve the performance of attention-based GNNs, we propose cardinality preserved attention (CPA) models that can be applied to any kind of attention mechanisms. Our experiments on node and graph classification confirm our theoretical analysis and show the competitive performance of our CPA models. The code is available online: https://github.com/zetayue/CPA.

Sign in / Sign up

Export Citation Format

Share Document