scholarly journals Node-wise Localization of Graph Neural Networks

2021 ◽  
Author(s):  
Zemin Liu ◽  
Yuan Fang ◽  
Chenghao Liu ◽  
Steven C.H. Hoi
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Representation Learning ◽  
Global Model ◽  
Local Context ◽  
Local Contexts ◽  
Unique Model ◽  
Graph Neural Networks ◽  
Global And Local ◽  
Different Parts

Graph neural networks (GNNs) emerge as a powerful family of representation learning models on graphs. To derive node representations, they utilize a global model that recursively aggregates information from the neighboring nodes. However, different nodes reside at different parts of the graph in different local contexts, making their distributions vary across the graph. Ideally, how a node receives its neighborhood information should be a function of its local context, to diverge from the global GNN model shared by all nodes. To utilize node locality without overfitting, we propose a node-wise localization of GNNs by accounting for both global and local aspects of the graph. Globally, all nodes on the graph depend on an underlying global GNN to encode the general patterns across the graph; locally, each node is localized into a unique model as a function of the global model and its local context. Finally, we conduct extensive experiments on four benchmark graphs, and consistently obtain promising performance surpassing the state-of-the-art GNNs.

scholarly journals UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks

2021 ◽  
Author(s):  
Jing Huang ◽  
Jie Yang
Keyword(s):  
Neural Networks ◽  
Message Passing ◽  
State Of The Art ◽  
Representation Learning ◽  
Graph Representation ◽  
Challenging Problem ◽  
Unified Framework ◽  
Real World Datasets ◽  
Graph Neural Networks ◽  
Research Domains

Hypergraph, an expressive structure with flexibility to model the higher-order correlations among entities, has recently attracted increasing attention from various research domains. Despite the success of Graph Neural Networks (GNNs) for graph representation learning, how to adapt the powerful GNN-variants directly into hypergraphs remains a challenging problem. In this paper, we propose UniGNN, a unified framework for interpreting the message passing process in graph and hypergraph neural networks, which can generalize general GNN models into hypergraphs. In this framework, meticulously-designed architectures aiming to deepen GNNs can also be incorporated into hypergraphs with the least effort. Extensive experiments have been conducted to demonstrate the effectiveness of UniGNN on multiple real-world datasets, which outperform the state-of-the-art approaches with a large margin. Especially for the DBLP dataset, we increase the accuracy from 77.4% to 88.8% in the semi-supervised hypernode classification task. We further prove that the proposed message-passing based UniGNN models are at most as powerful as the 1-dimensional Generalized Weisfeiler-Leman (1-GWL) algorithm in terms of distinguishing non-isomorphic hypergraphs. Our code is available at https://github.com/OneForward/UniGNN.

scholarly journals GSSNN: Graph Smoothing Splines Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 7007-7014
Author(s):  
Shichao Zhu ◽  
Lewei Zhou ◽  
Shirui Pan ◽  
Chuan Zhou ◽  
Guiying Yan ◽  
...  
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Representation Learning ◽  
Smoothing Splines ◽  
Graph Representation ◽  
Graph Classification ◽  
Topological Information ◽  
Graph Representations ◽  
The Arts ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in many graph data analysis tasks. However, they still suffer from two limitations for graph representation learning. First, they exploit non-smoothing node features which may result in suboptimal embedding and degenerated performance for graph classification. Second, they only exploit neighbor information but ignore global topological knowledge. Aiming to overcome these limitations simultaneously, in this paper, we propose a novel, flexible, and end-to-end framework, Graph Smoothing Splines Neural Networks (GSSNN), for graph classification. By exploiting the smoothing splines, which are widely used to learn smoothing fitting function in regression, we develop an effective feature smoothing and enhancement module Scaled Smoothing Splines (S3) to learn graph embedding. To integrate global topological information, we design a novel scoring module, which exploits closeness, degree, as well as self-attention values, to select important node features as knots for smoothing splines. These knots can be potentially used for interpreting classification results. In extensive experiments on biological and social datasets, we demonstrate that our model achieves state-of-the-arts and GSSNN is superior in learning more robust graph representations. Furthermore, we show that S3 module is easily plugged into existing GNNs to improve their performance.

scholarly journals Graph Transformer for Graph-to-Sequence Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 7464-7471
Author(s):  
Deng Cai ◽  
Wai Lam
Keyword(s):  
Neural Networks ◽  
Information Exchange ◽  
State Of The Art ◽  
Structural Information ◽  
Representation Learning ◽  
Graph Representation ◽  
Text Generation ◽  
Proposed Model ◽  
Graph Neural Networks ◽  
Meaning Representation

The dominant graph-to-sequence transduction models employ graph neural networks for graph representation learning, where the structural information is reflected by the receptive field of neurons. Unlike graph neural networks that restrict the information exchange between immediate neighborhood, we propose a new model, known as Graph Transformer, that uses explicit relation encoding and allows direct communication between two distant nodes. It provides a more efficient way for global graph structure modeling. Experiments on the applications of text generation from Abstract Meaning Representation (AMR) and syntax-based neural machine translation show the superiority of our proposed model. Specifically, our model achieves 27.4 BLEU on LDC2015E86 and 29.7 BLEU on LDC2017T10 for AMR-to-text generation, outperforming the state-of-the-art results by up to 2.2 points. On the syntax-based translation tasks, our model establishes new single-model state-of-the-art BLEU scores, 21.3 for English-to-German and 14.1 for English-to-Czech, improving over the existing best results, including ensembles, by over 1 BLEU.

scholarly journals Multi-hop Attention Graph Neural Networks

2021 ◽  
Author(s):  
Guangtao Wang ◽  
Rex Ying ◽  
Jing Huang ◽  
Jure Leskovec
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Performance Metrics ◽  
State Of The Art ◽  
Structural Information ◽  
Representation Learning ◽  
Attention Mechanism ◽  
Graph Representation ◽  
Knowledge Graph ◽  
Graph Neural Networks

Self-attention mechanism in graph neural networks (GNNs) led to state-of-the-art performance on many graph representation learning tasks. Currently, at every layer, attention is computed between connected pairs of nodes and depends solely on the representation of the two nodes. However, such attention mechanism does not account for nodes that are not directly connected but provide important network context. Here we propose Multi-hop Attention Graph Neural Network (MAGNA), a principled way to incorporate multi-hop context information into every layer of attention computation. MAGNA diffuses the attention scores across the network, which increases the receptive field for every layer of the GNN. Unlike previous approaches, MAGNA uses a diffusion prior on attention values, to efficiently account for all paths between the pair of disconnected nodes. We demonstrate in theory and experiments that MAGNA captures large-scale structural information in every layer, and has a low-pass effect that eliminates noisy high-frequency information from graph data. Experimental results on node classification as well as the knowledge graph completion benchmarks show that MAGNA achieves state-of-the-art results: MAGNA achieves up to 5.7% relative error reduction over the previous state-of-the-art on Cora, Citeseer, and Pubmed. MAGNA also obtains the best performance on a large-scale Open Graph Benchmark dataset. On knowledge graph completion MAGNA advances state-of-the-art on WN18RR and FB15k-237 across four different performance metrics.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals Efficient Graph Collaborative Filtering via Contrastive Learning

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4666
Author(s):  
Zhiqiang Pan ◽  
Honghui Chen
Keyword(s):  
Collaborative Filtering ◽  
State Of The Art ◽  
Representation Learning ◽  
First Order ◽  
Satisfactory Performance ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Bayesian Personalized Ranking ◽  
Graph Neural Networks ◽  
Training Efficiency

Collaborative filtering (CF) aims to make recommendations for users by detecting user’s preference from the historical user–item interactions. Existing graph neural networks (GNN) based methods achieve satisfactory performance by exploiting the high-order connectivity between users and items, however they suffer from the poor training efficiency problem and easily introduce bias for information propagation. Moreover, the widely applied Bayesian personalized ranking (BPR) loss is insufficient to provide supervision signals for training due to the extremely sparse observed interactions. To deal with the above issues, we propose the Efficient Graph Collaborative Filtering (EGCF) method. Specifically, EGCF adopts merely one-layer graph convolution to model the collaborative signal for users and items from the first-order neighbors in the user–item interactions. Moreover, we introduce contrastive learning to enhance the representation learning of users and items by deriving the self-supervisions, which is jointly trained with the supervised learning. Extensive experiments are conducted on two benchmark datasets, i.e., Yelp2018 and Amazon-book, and the experimental results demonstrate that EGCF can achieve the state-of-the-art performance in terms of Recall and normalized discounted cumulative gain (NDCG), especially on ranking the target items at right positions. In addition, EGCF shows obvious advantages in the training efficiency compared with the competitive baselines, making it practicable for potential applications.

scholarly journals Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective

2020 ◽  
Author(s):  
Luís C. Lamb ◽  
Artur d’Avila Garcez ◽  
Marco Gori ◽  
Marcelo O.R. Prates ◽  
Pedro H.C. Avelar ◽  
...  
Keyword(s):  
Neural Networks ◽  
Constraint Satisfaction ◽  
State Of The Art ◽  
Relational Reasoning ◽  
Widespread Application ◽  
Symbolic Computing ◽  
Industry Research ◽  
Optimization Constraint ◽  
The Subject ◽  
Graph Neural Networks

Neural-symbolic computing has now become the subject of interest of both academic and industry research laboratories. Graph Neural Networks (GNNs) have been widely used in relational and symbolic domains, with widespread application of GNNs in combinatorial optimization, constraint satisfaction, relational reasoning and other scientific domains. The need for improved explainability, interpretability and trust of AI systems in general demands principled methodologies, as suggested by neural-symbolic computing. In this paper, we review the state-of-the-art on the use of GNNs as a model of neural-symbolic computing. This includes the application of GNNs in several domains as well as their relationship to current developments in neural-symbolic computing.

scholarly journals Joint Representation Learning of Legislator and Legislation for Roll Call Prediction

2020 ◽  
Author(s):  
Yuqiao Yang ◽  
Xiaoqiang Lin ◽  
Geng Lin ◽  
Zengfeng Huang ◽  
Changjian Jiang ◽  
...  
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Ideal Point ◽  
Representation Learning ◽  
Context Information ◽  
Roll Call ◽  
Triplet Loss ◽  
Joint Representation ◽  
Narrative Description ◽  
The Ideal

In this paper, we explore to learn representations of legislation and legislator for the prediction of roll call results. The most popular approach for this topic is named the ideal point model that relies on historical voting information for representation learning of legislators. It largely ignores the context information of the legislative data. We, therefore, propose to incorporate context information to learn dense representations for both legislators and legislation. For legislators, we incorporate relations among them via graph convolutional neural networks (GCN) for their representation learning. For legislation, we utilize its narrative description via recurrent neural networks (RNN) for representation learning. In order to align two kinds of representations in the same vector space, we introduce a triplet loss for the joint training. Experimental results on a self-constructed dataset show the effectiveness of our model for roll call results prediction compared to some state-of-the-art baselines.

scholarly journals Multi-View Attribute Graph Convolution Networks for Clustering

2020 ◽  
Author(s):  
Jiafeng Cheng ◽  
Qianqian Wang ◽  
Zhiqiang Tao ◽  
Deyan Xie ◽  
Quanxue Gao
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Graph Embedding ◽  
Structured Data ◽  
Attention Networks ◽  
Graph Data ◽  
Graph Reconstruction ◽  
Node Attributes ◽  
Graph Neural Networks ◽  
Geometric Relationship

Graph neural networks (GNNs) have made considerable achievements in processing graph-structured data. However, existing methods can not allocate learnable weights to different nodes in the neighborhood and lack of robustness on account of neglecting both node attributes and graph reconstruction. Moreover, most of multi-view GNNs mainly focus on the case of multiple graphs, while designing GNNs for solving graph-structured data of multi-view attributes is still under-explored. In this paper, we propose a novel Multi-View Attribute Graph Convolution Networks (MAGCN) model for the clustering task. MAGCN is designed with two-pathway encoders that map graph embedding features and learn the view-consistency information. Specifically, the first pathway develops multi-view attribute graph attention networks to reduce the noise/redundancy and learn the graph embedding features for each multi-view graph data. The second pathway develops consistent embedding encoders to capture the geometric relationship and probability distribution consistency among different views, which adaptively finds a consistent clustering embedding space for multi-view attributes. Experiments on three benchmark graph datasets show the superiority of our method compared with several state-of-the-art algorithms.

Sign in / Sign up

Export Citation Format

Share Document