scholarly journals The interplay between communities and homophily in semi-supervised classification using graph neural networks

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Hussain Hussain ◽  
Tomislav Duricic ◽  
Elisabeth Lex ◽  
Denis Helic ◽  
Roman Kern
Keyword(s):  
Neural Networks ◽  
Community Structure ◽  
Model Selection ◽  
Graph Structure ◽  
Information Theoretic ◽  
Node Classification ◽  
Selection For ◽  
Graph Neural Networks ◽  
Node Labels ◽  
The Impact

AbstractGraph Neural Networks (GNNs) are effective in many applications. Still, there is a limited understanding of the effect of common graph structures on the learning process of GNNs. To fill this gap, we study the impact of community structure and homophily on the performance of GNNs in semi-supervised node classification on graphs. Our methodology consists of systematically manipulating the structure of eight datasets, and measuring the performance of GNNs on the original graphs and the change in performance in the presence and the absence of community structure and/or homophily. Our results show the major impact of both homophily and communities on the classification accuracy of GNNs, and provide insights on their interplay. In particular, by analyzing community structure and its correlation with node labels, we are able to make informed predictions on the suitability of GNNs for classification on a given graph. Using an information-theoretic metric for community-label correlation, we devise a guideline for model selection based on graph structure. With our work, we provide insights on the abilities of GNNs and the impact of common network phenomena on their performance. Our work improves model selection for node classification in semi-supervised settings.

scholarly journals The Impact of Global Structural Information in Graph Neural Networks Applications

Data ◽  
2022 ◽  
Vol 7 (1) ◽  
pp. 10
Author(s):  
Davide Buffelli ◽  
Fabio Vandin
Keyword(s):  
Neural Networks ◽  
Structural Information ◽  
Global Information ◽  
Graph Structure ◽  
Practical Applications ◽  
Average Accuracy ◽  
Graph Neural Networks ◽  
The Impact ◽  
Regularization Strategy ◽  
Node Embeddings

Graph Neural Networks (GNNs) rely on the graph structure to define an aggregation strategy where each node updates its representation by combining information from its neighbours. A known limitation of GNNs is that, as the number of layers increases, information gets smoothed and squashed and node embeddings become indistinguishable, negatively affecting performance. Therefore, practical GNN models employ few layers and only leverage the graph structure in terms of limited, small neighbourhoods around each node. Inevitably, practical GNNs do not capture information depending on the global structure of the graph. While there have been several works studying the limitations and expressivity of GNNs, the question of whether practical applications on graph structured data require global structural knowledge or not remains unanswered. In this work, we empirically address this question by giving access to global information to several GNN models, and observing the impact it has on downstream performance. Our results show that global information can in fact provide significant benefits for common graph-related tasks. We further identify a novel regularization strategy that leads to an average accuracy improvement of more than 5% on all considered tasks.

scholarly journals How do loss functions impact the performance of graph neural networks?

2021 ◽  
Author(s):  
Gabriel Jonas Duarte ◽  
Tamara Arruda Pereira ◽  
Erik Jhones Nascimento ◽  
Diego Mesquita ◽  
Amauri Holanda Souza Junior
Keyword(s):  
Neural Networks ◽  
Mean Absolute Error ◽  
Absolute Error ◽  
Cross Entropy ◽  
Loss Functions ◽  
Hinge Loss ◽  
Significant Difference ◽  
Node Classification ◽  
Graph Neural Networks ◽  
The Impact

Graph neural networks (GNNs) have become the de facto approach for supervised learning on graph data.To train these networks, most practitioners employ the categorical cross-entropy (CE) loss. We can attribute this largely to the probabilistic interpretability of models trained using CE, since it corresponds to the negative log of the categorical/softmax likelihood.We can attribute this largely to the probabilistic interpretation of CE, since it corresponds to the negative log of the categorical/softmax likelihood.Nonetheless, recent works have shown that deep learning models can benefit from adopting other loss functions. For instance, neural networks trained with symmetric losses (e.g., mean absolute error) are robust to label noise. Nonetheless, loss functions are a modeling choice and other training criteria can be employed — e.g., hinge loss and mean absolute error (MAE). Perhaps surprisingly, the effect of using different losses on GNNs has not been explored. In this preliminary work, we gauge the impact of different loss functions to the performance of GNNs for node classification under i) noisy labels and ii) different sample sizes. In contrast to findings on Euclidean domains, our results for GNNs show that there is no significant difference between models trained with CE and other classical loss functions on both aforementioned scenarios.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

Graph-based knowledge tracing: Modeling student proficiency using graph neural networks

2021 ◽  
pp. 1-16
Author(s):  
Hiromi Nakagawa ◽  
Yusuke Iwasawa ◽  
Yutaka Matsuo
Keyword(s):  
Neural Networks ◽  
Student Performance ◽  
Classification Problem ◽  
Computer Assisted ◽  
Graph Structure ◽  
Inductive Bias ◽  
Additional Information ◽  
Knowledge Tracing ◽  
Open Datasets ◽  
Graph Neural Networks

Recent advancements in computer-assisted learning systems have caused an increase in the research in knowledge tracing, wherein student performance is predicted over time. Student coursework can potentially be structured as a graph. Incorporating this graph-structured nature into a knowledge tracing model as a relational inductive bias can improve its performance; however, previous methods, such as deep knowledge tracing, did not consider such a latent graph structure. Inspired by the recent successes of graph neural networks (GNNs), we herein propose a GNN-based knowledge tracing method, i.e., graph-based knowledge tracing. Casting the knowledge structure as a graph enabled us to reformulate the knowledge tracing task as a time-series node-level classification problem in the GNN. As the knowledge graph structure is not explicitly provided in most cases, we propose various implementations of the graph structure. Empirical validations on two open datasets indicated that our method could potentially improve the prediction of student performance and demonstrated more interpretable predictions compared to those of the previous methods, without the requirement of any additional information.

scholarly journals Online Planner Selection with Graph Neural Networks and Adaptive Scheduling

2020 ◽  
Vol 34 (04) ◽  
pp. 5077-5084
Author(s):  
Tengfei Ma ◽  
Patrick Ferber ◽  
Siyu Huo ◽  
Jie Chen ◽  
Michael Katz
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Automated Planning ◽  
Adaptive Scheduling ◽  
Optimal Planning ◽  
Graph Representations ◽  
Structural Graph ◽  
Scheduling Method ◽  
Graph Neural Networks ◽  
Node Labels

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference.Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based.The code is available at https://github.com/matenure/GNN_planner.

scholarly journals Graph Self Supervised Learning: the BT, the HSIC, and the VICReg

2021 ◽  
Author(s):  
Sayan Nag
Keyword(s):  
Neural Networks ◽  
Supervised Learning ◽  
Loss Function ◽  
Data Augmentation ◽  
Learning Strategy ◽  
Loss Functions ◽  
Augmentation Strategies ◽  
Batch Sizes ◽  
Graph Neural Networks ◽  
The Impact

Self-supervised learning and pre-training strategies have developed over the last few years especially for Convolutional Neural Networks (CNNs). Recently application of such methods can also be noticed for Graph Neural Networks (GNNs). In this paper, we have used a graph based self-supervised learning strategy with different loss functions (Barlow Twins[? ], HSIC[? ], VICReg[? ]) which have shown promising results when applied with CNNs previously. We have also proposed a hybrid loss function combining the advantages of VICReg and HSIC and called it as VICRegHSIC. The performance of these aforementioned methods have been compared when applied to two different datasets namely MUTAG and PROTEINS. Moreover, the impact of different batch sizes, projector dimensions and data augmentation strategies have also been explored. The results are preliminary and we will be continuing to explore with other datasets.

scholarly journals Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

2021 ◽  
Vol 4 ◽  
Author(s):  
Paul Y. Wang ◽  
Sandalika Sapra ◽  
Vivek Kurien George ◽  
Gabriel A. Silva
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Neural Systems ◽  
Graph Structure ◽  
Imaging Data ◽  
C Elegans ◽  
State Classification ◽  
Graph Neural Networks ◽  
Emergent Behaviors

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

Sign in / Sign up

Export Citation Format

Share Document