scholarly journals Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View

2020 ◽  
Vol 34 (04) ◽  
pp. 3438-3445 ◽  
Author(s):  
Deli Chen ◽  
Yankai Lin ◽  
Wei Li ◽  
Peng Li ◽  
Jie Zhou ◽  
...  
Keyword(s):  
Neural Networks ◽  
Quantitative Study ◽  
Critical Factor ◽  
Graph Topology ◽  
Severe Limitation ◽  
Quantitative Metrics ◽  
Wide Range ◽  
Model Predictions ◽  
Graph Neural Networks ◽  
Training Objective

Graph Neural Networks (GNNs) have achieved promising performance on a wide range of graph-based tasks. Despite their success, one severe limitation of GNNs is the over-smoothing issue (indistinguishable representations of nodes in different classes). In this work, we present a systematic and quantitative study on the over-smoothing issue of GNNs. First, we introduce two quantitative metrics, MAD and MADGap, to measure the smoothness and over-smoothness of the graph nodes representations, respectively. Then, we verify that smoothing is the nature of GNNs and the critical factor leading to over-smoothness is the low information-to-noise ratio of the message received by the nodes, which is partially determined by the graph topology. Finally, we propose two methods to alleviate the over-smoothing issue from the topological view: (1) MADReg which adds a MADGap-based regularizer to the training objective; (2) AdaEdge which optimizes the graph topology based on the model predictions. Extensive experiments on 7 widely-used graph datasets with 10 typical GNN models show that the two proposed methods are effective for relieving the over-smoothing issue, thus improving the performance of various GNN models.

scholarly journals When Do GNNs Work: Understanding and Improving Neighborhood Aggregation

2020 ◽  
Author(s):  
Yiqing Xie ◽  
Sha Li ◽  
Carl Yang ◽  
Raymond Chi-Wing Wong ◽  
Jiawei Han
Keyword(s):  
Neural Networks ◽  
Wide Range ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) have been shown to be powerful in a wide range of graph-related tasks. While there exists various GNN models, a critical common ingredient is neighborhood aggregation, where the embedding of each node is updated by referring to the embedding of its neighbors. This paper aims to provide a better understanding of this mechanisms by asking the following question: Is neighborhood aggregation always necessary and beneficial? In short, the answer is no. We carve out two conditions under which neighborhood aggregation is not helpful: (1) when a node's neighbors are highly dissimilar and (2) when a node's embedding is already similar with that of its neighbors. We propose novel metrics that quantitatively measure these two circumstances and integrate them into an Adaptive-layer module. Our experiments show that allowing for node-specific aggregation degrees have significant advantage over current GNNs.

scholarly journals Stroke recovery phenotyping through network trajectory approaches and graph neural networks

2021 ◽  
Author(s):  
Sanjukta Krishnagopal ◽  
Keith Lohse ◽  
Robynne Braun
Keyword(s):  
Neural Networks ◽  
Multivariate Regression ◽  
Network Science ◽  
Stroke Recovery ◽  
Neurological Injury ◽  
Time Points ◽  
Clinical Recovery ◽  
Wide Range ◽  
Graph Neural Networks ◽  
Symptom Progression

AbstractStroke is a leading cause of neurological injury characterized by impairments in multiple neurological domains including cognition, language, sensory and motor functions. Clinical recovery in these domains is tracked using a wide range of measures that may be continuous, ordinal, interval or categorical in nature, which presents challenges for standard multivariate regression approaches. This has hindered stroke researchers’ ability to achieve an integrated picture of the complex time-evolving interactions amongst symptoms. Here we use tools from network science and machine learning that are particularly well-suited to extracting underlying patterns in such data, and may assist in prediction of recovery patterns. To demonstrate the utility of this approach, we analyzed data from the NINDS tPA trial using the Trajectory Profile Clustering (TPC) method to identify distinct stroke recovery patterns for 11 different neurological domains at 5 discrete time points. Our analysis identified 3 distinct stroke trajectory profiles that align with clinically relevant stroke syndromes, characterized both by distinct clusters of symptoms, as well as differing degrees of symptom severity. We then validated our approach using graph neural networks to determine how well our model performed predictively for stratifying patients into these trajectory profiles at early vs. later time points post-stroke. We demonstrate that trajectory profile clustering is an effective method for identifying clinically relevant recovery subtypes in multidimensional longitudinal datasets, and for early prediction of symptom progression subtypes in individual patients. This paper is the first work introducing network trajectory approaches for stroke recovery phenotyping, and is aimed at enhancing the translation of such novel computational approaches for practical clinical application.

Graph Neural Networks for Prediction of Fuel Ignition Quality

2020 ◽  
Author(s):  
Artur Schweidtmann ◽  
Jan Rittig ◽  
Andrea König ◽  
Martin Grohe ◽  
Alexander Mitsos ◽  
...  
Keyword(s):  
Neural Networks ◽  
Octane Number ◽  
Molecular Graph ◽  
Chemical Properties ◽  
Graph Representation ◽  
Structure Property ◽  
Oxygenated Hydrocarbons ◽  
Physico Chemical ◽  
Ignition Quality ◽  
Graph Neural Networks

<div>Prediction of combustion-related properties of (oxygenated) hydrocarbons is an important and challenging task for which quantitative structure-property relationship (QSPR) models are frequently employed. Recently, a machine learning method, graph neural networks (GNNs), has shown promising results for the prediction of structure-property relationships. GNNs utilize a graph representation of molecules, where atoms correspond to nodes and bonds to edges containing information about the molecular structure. More specifically, GNNs learn physico-chemical properties as a function of the molecular graph in a supervised learning setup using a backpropagation algorithm. This end-to-end learning approach eliminates the need for selection of molecular descriptors or structural groups, as it learns optimal fingerprints through graph convolutions and maps the fingerprints to the physico-chemical properties by deep learning. We develop GNN models for predicting three fuel ignition quality indicators, i.e., the derived cetane number (DCN), the research octane number (RON), and the motor octane number (MON), of oxygenated and non-oxygenated hydrocarbons. In light of limited experimental data in the order of hundreds, we propose a combination of multi-task learning, transfer learning, and ensemble learning. The results show competitive performance of the proposed GNN approach compared to state-of-the-art QSPR models making it a promising field for future research. The prediction tool is available via a web front-end at www.avt.rwth-aachen.de/gnn.</div>

Conversational Emotion Recognition Using Self-Attention Mechanisms and Graph Neural Networks

2020 ◽  
Author(s):  
Zheng Lian ◽  
Jianhua Tao ◽  
Bin Liu ◽  
Jian Huang ◽  
Zhanlei Yang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Emotion Recognition ◽  
Graph Neural Networks

scholarly journals Adherence to Ketogenic and Mediterranean Study Diets in a Crossover Trial: The Keto–Med Randomized Trial

Nutrients ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 967
Author(s):  
Matthew J. Landry ◽  
Anthony Crimarco ◽  
Dalia Perelman ◽  
Lindsay R. Durand ◽  
Christina Petlura ◽  
...  
Keyword(s):  
Randomized Trial ◽  
Randomized Clinical Trials ◽  
Secondary Analysis ◽  
Critical Factor ◽  
Detailed Examination ◽  
Diet Patterns ◽  
Time Points ◽  
Study Results ◽  
Wide Range ◽  
Diet Pattern

Adherence is a critical factor to consider when interpreting study results from randomized clinical trials (RCTs) comparing one diet to another, but it is frequently not reported by researchers. The purpose of this secondary analysis of the Keto–Med randomized trial was to provide a detailed examination and comparison of the adherence to the two study diets (Well Formulated Ketogenic Diet (WFKD) and Mediterranean Plus (Med-Plus)) under the two conditions: all food being provided (delivered) and all food being obtained by individual participants (self-provided). Diet was assessed at six time points including baseline (x1), week 4 of each phase when participants were receiving food deliveries (x2), week 12 of each phase when participants were preparing and providing food on their own (x2), and 12 weeks after participants completed both diet phases and were free to choose their own diet pattern (x1). The adherence scores for WFKD and Med-Plus were developed specifically for this study. Average adherence to the two diet patterns was very similar during both on-study time points of the intervention. Throughout the study, a wide range of adherence was observed among participants—for both diet types and during both the delivery phase and self-provided phase. Insight from this assessment of adherence may aid other researchers when answering the important question of how to improve behavioral adherence during dietary trials. This study is registered at clinicaltrials.gov NCT03810378.

Sign in / Sign up

Export Citation Format

Share Document