Graph-based knowledge tracing: Modeling student proficiency using graph neural networks

2021 ◽  
pp. 1-16
Author(s):  
Hiromi Nakagawa ◽  
Yusuke Iwasawa ◽  
Yutaka Matsuo
Keyword(s):  
Neural Networks ◽  
Student Performance ◽  
Classification Problem ◽  
Computer Assisted ◽  
Graph Structure ◽  
Inductive Bias ◽  
Additional Information ◽  
Knowledge Tracing ◽  
Open Datasets ◽  
Graph Neural Networks

Recent advancements in computer-assisted learning systems have caused an increase in the research in knowledge tracing, wherein student performance is predicted over time. Student coursework can potentially be structured as a graph. Incorporating this graph-structured nature into a knowledge tracing model as a relational inductive bias can improve its performance; however, previous methods, such as deep knowledge tracing, did not consider such a latent graph structure. Inspired by the recent successes of graph neural networks (GNNs), we herein propose a GNN-based knowledge tracing method, i.e., graph-based knowledge tracing. Casting the knowledge structure as a graph enabled us to reformulate the knowledge tracing task as a time-series node-level classification problem in the GNN. As the knowledge graph structure is not explicitly provided in most cases, we propose various implementations of the graph structure. Empirical validations on two open datasets indicated that our method could potentially improve the prediction of student performance and demonstrated more interpretable predictions compared to those of the previous methods, without the requirement of any additional information.

scholarly journals The Impact of Global Structural Information in Graph Neural Networks Applications

Data ◽  
2022 ◽  
Vol 7 (1) ◽  
pp. 10
Author(s):  
Davide Buffelli ◽  
Fabio Vandin
Keyword(s):  
Neural Networks ◽  
Structural Information ◽  
Global Information ◽  
Graph Structure ◽  
Practical Applications ◽  
Average Accuracy ◽  
Graph Neural Networks ◽  
The Impact ◽  
Regularization Strategy ◽  
Node Embeddings

Graph Neural Networks (GNNs) rely on the graph structure to define an aggregation strategy where each node updates its representation by combining information from its neighbours. A known limitation of GNNs is that, as the number of layers increases, information gets smoothed and squashed and node embeddings become indistinguishable, negatively affecting performance. Therefore, practical GNN models employ few layers and only leverage the graph structure in terms of limited, small neighbourhoods around each node. Inevitably, practical GNNs do not capture information depending on the global structure of the graph. While there have been several works studying the limitations and expressivity of GNNs, the question of whether practical applications on graph structured data require global structural knowledge or not remains unanswered. In this work, we empirically address this question by giving access to global information to several GNN models, and observing the impact it has on downstream performance. Our results show that global information can in fact provide significant benefits for common graph-related tasks. We further identify a novel regularization strategy that leads to an average accuracy improvement of more than 5% on all considered tasks.

scholarly journals The interplay between communities and homophily in semi-supervised classification using graph neural networks

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Hussain Hussain ◽  
Tomislav Duricic ◽  
Elisabeth Lex ◽  
Denis Helic ◽  
Roman Kern
Keyword(s):  
Neural Networks ◽  
Community Structure ◽  
Model Selection ◽  
Graph Structure ◽  
Information Theoretic ◽  
Node Classification ◽  
Selection For ◽  
Graph Neural Networks ◽  
Node Labels ◽  
The Impact

AbstractGraph Neural Networks (GNNs) are effective in many applications. Still, there is a limited understanding of the effect of common graph structures on the learning process of GNNs. To fill this gap, we study the impact of community structure and homophily on the performance of GNNs in semi-supervised node classification on graphs. Our methodology consists of systematically manipulating the structure of eight datasets, and measuring the performance of GNNs on the original graphs and the change in performance in the presence and the absence of community structure and/or homophily. Our results show the major impact of both homophily and communities on the classification accuracy of GNNs, and provide insights on their interplay. In particular, by analyzing community structure and its correlation with node labels, we are able to make informed predictions on the suitability of GNNs for classification on a given graph. Using an information-theoretic metric for community-label correlation, we devise a guideline for model selection based on graph structure. With our work, we provide insights on the abilities of GNNs and the impact of common network phenomena on their performance. Our work improves model selection for node classification in semi-supervised settings.

scholarly journals Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

2021 ◽  
Vol 4 ◽  
Author(s):  
Paul Y. Wang ◽  
Sandalika Sapra ◽  
Vivek Kurien George ◽  
Gabriel A. Silva
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Neural Systems ◽  
Graph Structure ◽  
Imaging Data ◽  
C Elegans ◽  
State Classification ◽  
Graph Neural Networks ◽  
Emergent Behaviors

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

scholarly journals CosG: A Graph-Based Contrastive Learning Method for Fact Verification

Sensors ◽  
2021 ◽  
Vol 21 (10) ◽  
pp. 3471
Author(s):  
Chonghao Chen ◽  
Jianming Zheng ◽  
Honghui Chen
Keyword(s):  
Neural Networks ◽  
Semantic Representation ◽  
Experimental Results ◽  
Graph Structure ◽  
Learning Method ◽  
Low Resource ◽  
Label Claim ◽  
Graph Neural Networks ◽  
Graph Propagation ◽  
Model Robustness

Fact verification aims to verify the authenticity of a given claim based on the retrieved evidence from Wikipedia articles. Existing works mainly focus on enhancing the semantic representation of evidence, e.g., introducing the graph structure to model the evidence relation. However, previous methods can’t well distinguish semantic-similar claims and evidences with distinct authenticity labels. In addition, the performances of graph-based models are limited by the over-smoothing problem of graph neural networks. To this end, we propose a graph-based contrastive learning method for fact verification abbreviated as CosG, which introduces a contrastive label-supervised task to help the encoder learn the discriminative representations for different-label claim-evidence pairs, as well as an unsupervised graph-contrast task, to alleviate the unique node features loss in the graph propagation. We conduct experiments on FEVER, a large benchmark dataset for fact verification. Experimental results show the superiority of our proposal against comparable baselines, especially for the claims that need multiple-evidences to verify. In addition, CosG presents better model robustness on the low-resource scenario.

scholarly journals Three-Dimensional Structural Geological Modeling Using Graph Neural Networks

2021 ◽  
Author(s):  
Michael Hillier ◽  
Florian Wellmann ◽  
Boyan Brodaric ◽  
Eric de Kemp ◽  
Ernst Schetselaar
Keyword(s):  
Neural Networks ◽  
Three Dimensional ◽  
Classification Problem ◽  
Future Research ◽  
Data Sampling ◽  
Neural Network Approach ◽  
Geological Unit ◽  
Point Data ◽  
Complex Settings ◽  
Graph Neural Networks

AbstractThree-dimensional structural geomodels are increasingly being used for a wide variety of scientific and societal purposes. Most advanced methods for generating these models are implicit approaches, but they suffer limitations in the types of interpolation constraints permitted, which can lead to poor modeling in structurally complex settings. A geometric deep learning approach, using graph neural networks, is presented in this paper as an alternative to classical implicit interpolation that is driven by a learning through training paradigm. The graph neural network approach consists of a developed architecture utilizing unstructured meshes as graphs on which coupled implicit and discrete geological unit modeling is performed, with the latter treated as a classification problem. The architecture generates three-dimensional structural models constrained by scattered point data, sampling geological units and interfaces as well as planar and linear orientations. The modeling capacity of the architecture for representing geological structures is demonstrated from its application on two diverse case studies. The benefits of the approach are (1) its ability to provide an expressive framework for incorporating interpolation constraints using loss functions and (2) its capacity to deal with both continuous and discrete properties simultaneously. Furthermore, a framework is established for future research for which additional geological constraints can be integrated into the modeling process.

scholarly journals Multilabel Classification Based on Graph Neural Networks

2021 ◽  
Author(s):  
Wei-Cheng Ye ◽  
Jia-Ching Wang
Keyword(s):  
Neural Networks ◽  
Gaussian Noise ◽  
Input Data ◽  
Multilabel Classification ◽  
Laplacian Matrices ◽  
Convolutional Networks ◽  
Multilabel Learning ◽  
Laplacian Graph ◽  
Open Datasets ◽  
Graph Neural Networks

Typical Laplacian embedding focuses on building Laplacian matrices prior to minimizing weights of connected graph components. However, for multilabel problems, it is difficult to determine such Laplacian graphs owing to multiple relations between vertices. Unlike typical approaches that require precomputed Laplacian matrices, this chapter presents a new method for automatically constructing Laplacian graphs during Laplacian embedding. By using trace minimization techniques, the topology of the Laplacian graph can be learned from input data, subsequently creating robust Laplacian embedding and influencing graph convolutional networks. Experiments on different open datasets with clean data and Gaussian noise were carried out. The noise level ranged from 6% to 12% of the maximum value of each dataset. Eleven different multilabel classification algorithms were used as the baselines for comparison. To verify the performance, three evaluation metrics specific to multilabel learning are proposed because multilabel learning is much more complicated than traditional single-label settings; each sample can be associated with multiple labels. The experimental results show that the proposed method performed better than the baselines, even when the data were contaminated by noise. The findings indicate that the proposed method is reliably robust against noise.

scholarly journals Graph Neural Networks and Their Current Applications in Bioinformatics

2021 ◽  
Vol 12 ◽  
Author(s):  
Xiao-Meng Zhang ◽  
Li Liang ◽  
Lin Liu ◽  
Ming-Jing Tang
Keyword(s):  
Neural Networks ◽  
Link Prediction ◽  
Structural Information ◽  
Quality Data ◽  
Graph Structure ◽  
Multiple Perspectives ◽  
Basic Principles ◽  
Rapid Accumulation ◽  
Excellent Method ◽  
Graph Neural Networks

Graph neural networks (GNNs), as a branch of deep learning in non-Euclidean space, perform particularly well in various tasks that process graph structure data. With the rapid accumulation of biological network data, GNNs have also become an important tool in bioinformatics. In this research, a systematic survey of GNNs and their advances in bioinformatics is presented from multiple perspectives. We first introduce some commonly used GNN models and their basic principles. Then, three representative tasks are proposed based on the three levels of structural information that can be learned by GNNs: node classification, link prediction, and graph generation. Meanwhile, according to the specific applications for various omics data, we categorize and discuss the related studies in three aspects: disease prediction, drug discovery, and biomedical imaging. Based on the analysis, we provide an outlook on the shortcomings of current studies and point out their developing prospect. Although GNNs have achieved excellent results in many biological tasks at present, they still face challenges in terms of low-quality data processing, methodology, and interpretability and have a long road ahead. We believe that GNNs are potentially an excellent method that solves various biological problems in bioinformatics research.

scholarly journals Graph Convolutional Networks by Architecture Search for PolSAR Image Classification

2021 ◽  
Vol 13 (7) ◽  
pp. 1404
Author(s):  
Hongying Liu ◽  
Derong Xu ◽  
Tianwen Zhu ◽  
Fanhua Shang ◽  
Yuanyuan Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Spatial Relations ◽  
Classification Problem ◽  
Search Method ◽  
Sample Distribution ◽  
Convolutional Networks ◽  
Training Samples ◽  
Graph Neural Networks

Classification of polarimetric synthetic aperture radar (PolSAR) images has achieved good results due to the excellent fitting ability of neural networks with a large number of training samples. However, the performance of most convolutional neural networks (CNNs) degrades dramatically when only a few labeled training samples are available. As one well-known class of semi-supervised learning methods, graph convolutional networks (GCNs) have gained much attention recently to address the classification problem with only a few labeled samples. As the number of layers grows in the network, the parameters dramatically increase. It is challenging to determine an optimal architecture manually. In this paper, we propose a neural architecture search method based GCN (ASGCN) for the classification of PolSAR images. We construct a novel graph whose nodes combines both the physical features and spatial relations between pixels or samples to represent the image. Then we build a new searching space whose components are empirically selected from some graph neural networks for architecture search and develop the differentiable architecture search method to construction our ASGCN. Moreover, to address the training of large-scale images, we present a new weighted mini-batch algorithm to reduce the computing memory consumption and ensure the balance of sample distribution, and also analyze and compare with other similar training strategies. Experiments on several real-world PolSAR datasets show that our method has improved the overall accuracy as much as 3.76% than state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document