scholarly journals Learning Attributed Graph Representation with Communicative Message Passing Transformer

2021 ◽  
Author(s):  
Jianwen Chen ◽  
Shuangjia Zheng ◽  
Ying Song ◽  
Jiahua Rao ◽  
Yuedong Yang
Keyword(s):  
Message Passing ◽  
Diffusion Mechanism ◽  
Molecular Graph ◽  
Representation Learning ◽  
Graph Connectivity ◽  
Graph Representation ◽  
Graph Properties ◽  
Attributed Graph ◽  
Attributed Graphs ◽  
Local Aggregation

Constructing appropriate representations of molecules lies at the core of numerous tasks such as material science, chemistry, and drug designs. Recent researches abstract molecules as attributed graphs and employ graph neural networks (GNN) for molecular representation learning, which have made remarkable achievements in molecular graph modeling. Albeit powerful, current models either are based on local aggregation operations and thus miss higher-order graph properties or focus on only node information without fully using the edge information. For this sake, we propose a Communicative Message Passing Transformer (CoMPT) neural network to improve the molecular graph representation by reinforcing message interactions between nodes and edges based on the Transformer architecture. Unlike the previous transformer-style GNNs that treat molecule as a fully connected graph, we introduce a message diffusion mechanism to leverage the graph connectivity inductive bias and reduce the message enrichment explosion. Extensive experiments demonstrated that the proposed model obtained superior performances (around 4% on average) against state-of-the-art baselines on seven chemical property datasets (graph-level tasks) and two chemical shift datasets (node-level tasks). Further visualization studies also indicated a better representation capacity achieved by our model.

scholarly journals Graph representation learning: a survey

2020 ◽  
Vol 9 ◽  
Author(s):  
Fenxiao Chen ◽  
Yun-Cheng Wang ◽  
Bin Wang ◽  
C.-C. Jay Kuo
Keyword(s):  
Graph Embedding ◽  
Large Data ◽  
Representation Learning ◽  
Graph Representation ◽  
Data Sets ◽  
Graph Data ◽  
Graph Properties ◽  
Wide Range ◽  
Regular Lattices ◽  
Low Dimensional

Abstract Research on graph representation learning has received great attention in recent years since most data in real-world applications come in the form of graphs. High-dimensional graph data are often in irregular forms. They are more difficult to analyze than image/video/audio data defined on regular lattices. Various graph embedding techniques have been developed to convert the raw graph data into a low-dimensional vector representation while preserving the intrinsic graph properties. In this review, we first explain the graph embedding task and its challenges. Next, we review a wide range of graph embedding techniques with insights. Then, we evaluate several stat-of-the-art methods against small and large data sets and compare their performance. Finally, potential applications and future directions are presented.

scholarly journals Hierarchical and Unsupervised Graph Representation Learning with Loukas’s Coarsening

Algorithms ◽  
2020 ◽  
Vol 13 (9) ◽  
pp. 206
Author(s):  
Louis Béthune ◽  
Yacouba Kaloga ◽  
Pierre Borgnat ◽  
Aurélien Garivier ◽  
Amaury Habrard
Keyword(s):  
State Of The Art ◽  
Back Propagation ◽  
Representation Learning ◽  
Graph Representation ◽  
High Quality ◽  
Attributed Graphs ◽  
Information Maximization ◽  
Classification Tasks ◽  
Micro Structures ◽  
Mutual Information Maximization

We propose a novel algorithm for unsupervised graph representation learning with attributed graphs. It combines three advantages addressing some current limitations of the literature: (i) The model is inductive: it can embed new graphs without re-training in the presence of new data; (ii) The method takes into account both micro-structures and macro-structures by looking at the attributed graphs at different scales; (iii) The model is end-to-end differentiable: it is a building block that can be plugged into deep learning pipelines and allows for back-propagation. We show that combining a coarsening method having strong theoretical guarantees with mutual information maximization suffices to produce high quality embeddings. We evaluate them on classification tasks with common benchmarks of the literature. We show that our algorithm is competitive with state of the art among unsupervised graph representation learning methods.

scholarly journals Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations

2021 ◽  
Author(s):  
Yingheng Wang ◽  
Yaosen Min ◽  
Erzhuo Shao ◽  
Ji Wu
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Structural Information ◽  
Molecular Graph ◽  
Representation Learning ◽  
Graph Representation ◽  
Input Graph ◽  
Recent Success ◽  
Real World Datasets ◽  
Comparative Results

ABSTRACTLearning generalizable, transferable, and robust representations for molecule data has always been a challenge. The recent success of contrastive learning (CL) for self-supervised graph representation learning provides a novel perspective to learn molecule representations. The most prevailing graph CL framework is to maximize the agreement of representations in different augmented graph views. However, existing graph CL frameworks usually adopt stochastic augmentations or schemes according to pre-defined rules on the input graph to obtain different graph views in various scales (e.g. node, edge, and subgraph), which may destroy topological semantemes and domain prior in molecule data, leading to suboptimal performance. Therefore, designing parameterized, learnable, and explainable augmentation is quite necessary for molecular graph contrastive learning. A well-designed parameterized augmentation scheme can preserve chemically meaningful structural information and intrinsically essential attributes for molecule graphs, which helps to learn representations that are insensitive to perturbation on unimportant atoms and bonds. In this paper, we propose a novel Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations, MolCLE for brevity, that self-adaptively incorporates chemically significative information from both topological and semantic aspects of molecular graphs. Specifically, we apply deep neural networks to parameterize the augmentation process for both the molecular graph topology and atom attributes, to highlight contributive molecular substructures and recognize underlying chemical semantemes. Comprehensive experiments on a variety of real-world datasets demonstrate that our proposed method consistently outperforms compared baselines, which verifies the effectiveness of the proposed framework. Detailedly, our self-supervised MolCLE model surpasses many supervised counterparts, and meanwhile only uses hundreds of thousands of parameters to achieve comparative results against the state-of-the-art baseline, which has tens of millions of parameters. We also provide detailed case studies to validate the explainability of augmented graph views.CCS CONCEPTS• Mathematics of computing → Graph algorithms; • Applied computing → Bioinformatics; • Computing methodologies → Neural networks; Unsupervised learning.

scholarly journals UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks

2021 ◽  
Author(s):  
Jing Huang ◽  
Jie Yang
Keyword(s):  
Neural Networks ◽  
Message Passing ◽  
State Of The Art ◽  
Representation Learning ◽  
Graph Representation ◽  
Challenging Problem ◽  
Unified Framework ◽  
Real World Datasets ◽  
Graph Neural Networks ◽  
Research Domains

Hypergraph, an expressive structure with flexibility to model the higher-order correlations among entities, has recently attracted increasing attention from various research domains. Despite the success of Graph Neural Networks (GNNs) for graph representation learning, how to adapt the powerful GNN-variants directly into hypergraphs remains a challenging problem. In this paper, we propose UniGNN, a unified framework for interpreting the message passing process in graph and hypergraph neural networks, which can generalize general GNN models into hypergraphs. In this framework, meticulously-designed architectures aiming to deepen GNNs can also be incorporated into hypergraphs with the least effort. Extensive experiments have been conducted to demonstrate the effectiveness of UniGNN on multiple real-world datasets, which outperform the state-of-the-art approaches with a large margin. Especially for the DBLP dataset, we increase the accuracy from 77.4% to 88.8% in the semi-supervised hypernode classification task. We further prove that the proposed message-passing based UniGNN models are at most as powerful as the 1-dimensional Generalized Weisfeiler-Leman (1-GWL) algorithm in terms of distinguishing non-isomorphic hypergraphs. Our code is available at https://github.com/OneForward/UniGNN.

scholarly journals TSI-GNN: Extending Graph Neural Networks to Handle Missing Data in Temporal Settings

2021 ◽  
Vol 4 ◽  
Author(s):  
David Gordon ◽  
Panayiotis Petousis ◽  
Henry Zheng ◽  
Davina Zamanzadeh ◽  
Alex A.T. Bui
Keyword(s):  
Neural Networks ◽  
Missing Data ◽  
Bipartite Graph ◽  
Message Passing ◽  
Representation Learning ◽  
Temporal Information ◽  
Graph Representation ◽  
Sequence Information ◽  
Novel Approach ◽  
Graph Neural Networks

We present a novel approach for imputing missing data that incorporates temporal information into bipartite graphs through an extension of graph representation learning. Missing data is abundant in several domains, particularly when observations are made over time. Most imputation methods make strong assumptions about the distribution of the data. While novel methods may relax some assumptions, they may not consider temporality. Moreover, when such methods are extended to handle time, they may not generalize without retraining. We propose using a joint bipartite graph approach to incorporate temporal sequence information. Specifically, the observation nodes and edges with temporal information are used in message passing to learn node and edge embeddings and to inform the imputation task. Our proposed method, temporal setting imputation using graph neural networks (TSI-GNN), captures sequence information that can then be used within an aggregation function of a graph neural network. To the best of our knowledge, this is the first effort to use a joint bipartite graph approach that captures sequence information to handle missing data. We use several benchmark datasets to test the performance of our method against a variety of conditions, comparing to both classic and contemporary methods. We further provide insight to manage the size of the generated TSI-GNN model. Through our analysis we show that incorporating temporal information into a bipartite graph improves the representation at the 30% and 60% missing rate, specifically when using a nonlinear model for downstream prediction tasks in regularly sampled datasets and is competitive with existing temporal methods under different scenarios.

scholarly journals Communicative Representation Learning on Attributed Molecular Graphs

2020 ◽  
Author(s):  
Ying Song ◽  
Shuangjia Zheng ◽  
Zhangming Niu ◽  
Zhang-hua Fu ◽  
Yutong Lu ◽  
...  
Keyword(s):  
Neural Network ◽  
Message Passing ◽  
Molecular Graph ◽  
Representation Learning ◽  
Generation Process ◽  
Molecular Graphs ◽  
Graph Modeling ◽  
Proposed Model ◽  
The One ◽  
Graph Neural Networks

Constructing proper representations of molecules lies at the core of numerous tasks such as molecular property prediction and drug design. Graph neural networks, especially message passing neural network (MPNN) and its variants, have recently made remarkable achievements in molecular graph modeling. Albeit powerful, the one-sided focuses on atom (node) or bond (edge) information of existing MPNN methods lead to the insufficient representations of the attributed molecular graphs. Herein, we propose a Communicative Message Passing Neural Network (CMPNN) to improve the molecular embedding by strengthening the message interactions between nodes and edges through a communicative kernel. In addition, the message generation process is enriched by introducing a new message booster module. Extensive experiments demonstrated that the proposed model obtained superior performances against state-of-the-art baselines on six chemical property datasets. Further visualization also showed better representation capacity of our model.

Graph Neural Networks for Prediction of Fuel Ignition Quality

2020 ◽  
Author(s):  
Artur Schweidtmann ◽  
Jan Rittig ◽  
Andrea König ◽  
Martin Grohe ◽  
Alexander Mitsos ◽  
...  
Keyword(s):  
Neural Networks ◽  
Octane Number ◽  
Molecular Graph ◽  
Chemical Properties ◽  
Graph Representation ◽  
Structure Property ◽  
Oxygenated Hydrocarbons ◽  
Physico Chemical ◽  
Ignition Quality ◽  
Graph Neural Networks

<div>Prediction of combustion-related properties of (oxygenated) hydrocarbons is an important and challenging task for which quantitative structure-property relationship (QSPR) models are frequently employed. Recently, a machine learning method, graph neural networks (GNNs), has shown promising results for the prediction of structure-property relationships. GNNs utilize a graph representation of molecules, where atoms correspond to nodes and bonds to edges containing information about the molecular structure. More specifically, GNNs learn physico-chemical properties as a function of the molecular graph in a supervised learning setup using a backpropagation algorithm. This end-to-end learning approach eliminates the need for selection of molecular descriptors or structural groups, as it learns optimal fingerprints through graph convolutions and maps the fingerprints to the physico-chemical properties by deep learning. We develop GNN models for predicting three fuel ignition quality indicators, i.e., the derived cetane number (DCN), the research octane number (RON), and the motor octane number (MON), of oxygenated and non-oxygenated hydrocarbons. In light of limited experimental data in the order of hundreds, we propose a combination of multi-task learning, transfer learning, and ensemble learning. The results show competitive performance of the proposed GNN approach compared to state-of-the-art QSPR models making it a promising field for future research. The prediction tool is available via a web front-end at www.avt.rwth-aachen.de/gnn.</div>

scholarly journals Hyperspectral Image Classification with Localized Graph Convolutional Filtering

2021 ◽  
Vol 13 (3) ◽  
pp. 526
Author(s):  
Shengliang Pu ◽  
Yuanfeng Wu ◽  
Xu Sun ◽  
Xiaotong Sun
Keyword(s):  
Hyperspectral Image ◽  
Principal Component ◽  
Representation Learning ◽  
Classification Performance ◽  
Hyperspectral Data ◽  
Spectral Graph Theory ◽  
Feature Reduction ◽  
Graph Representation ◽  
Novel Method ◽  
Local Graph

The nascent graph representation learning has shown superiority for resolving graph data. Compared to conventional convolutional neural networks, graph-based deep learning has the advantages of illustrating class boundaries and modeling feature relationships. Faced with hyperspectral image (HSI) classification, the priority problem might be how to convert hyperspectral data into irregular domains from regular grids. In this regard, we present a novel method that performs the localized graph convolutional filtering on HSIs based on spectral graph theory. First, we conducted principal component analysis (PCA) preprocessing to create localized hyperspectral data cubes with unsupervised feature reduction. These feature cubes combined with localized adjacent matrices were fed into the popular graph convolution network in a standard supervised learning paradigm. Finally, we succeeded in analyzing diversified land covers by considering local graph structure with graph convolutional filtering. Experiments on real hyperspectral datasets demonstrated that the presented method offers promising classification performance compared with other popular competitors.

Sign in / Sign up

Export Citation Format

Share Document