Revisiting Graph Based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach

Graph Convolutional Networks~(GCNs) are state-of-the-art graph based representation learning models by iteratively stacking multiple layers of convolution aggregation operations and non-linear activation operations. Recently, in Collaborative Filtering~(CF) based Recommender Systems~(RS), by treating the user-item interaction behavior as a bipartite graph, some researchers model higher-layer collaborative signals with GCNs. These GCN based recommender models show superior performance compared to traditional works. However, these models suffer from training difficulty with non-linear activations for large user-item graphs. Besides, most GCN based models could not model deeper layers due to the over smoothing effect with the graph convolution operation. In this paper, we revisit GCN based CF models from two aspects. First, we empirically show that removing non-linearities would enhance recommendation performance, which is consistent with the theories in simple graph convolutional networks. Second, we propose a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user-item interaction data. The proposed model is a linear model and it is easy to train, scale to large datasets, and yield better efficiency and effectiveness on two real datasets. We publish the source code at https://github.com/newlei/LR-GCCF.

Download Full-text

Hybrid Low-Order and Higher-Order Graph Convolutional Networks

Computational Intelligence and Neuroscience ◽

10.1155/2020/3283890 ◽

2020 ◽

Vol 2020 ◽

pp. 1-9

Author(s):

Fangyuan Lei ◽

Xun Liu ◽

Qingyun Dai ◽

Bingo Wing-Kuen Ling ◽

Huimin Zhao ◽

...

Keyword(s):

Computational Complexity ◽

Large Scale ◽

Representation Learning ◽

Higher Order ◽

Semisupervised Learning ◽

Graph Representation ◽

Convolutional Network ◽

Convolutional Networks ◽

Proposed Model ◽

Low Order

With the higher-order neighborhood information of a graph network, the accuracy of graph representation learning classification can be significantly improved. However, the current higher-order graph convolutional networks have a large number of parameters and high computational complexity. Therefore, we propose a hybrid lower-order and higher-order graph convolutional network (HLHG) learning model, which uses a weight sharing mechanism to reduce the number of network parameters. To reduce the computational complexity, we propose a novel information fusion pooling layer to combine the high-order and low-order neighborhood matrix information. We theoretically compare the computational complexity and the number of parameters of the proposed model with those of the other state-of-the-art models. Experimentally, we verify the proposed model on large-scale text network datasets using supervised learning and on citation network datasets using semisupervised learning. The experimental results show that the proposed model achieves higher classification accuracy with a small set of trainable weight parameters.

Download Full-text

Graph convolutional networks: a comprehensive review

Computational Social Networks ◽

10.1186/s40649-019-0069-y ◽

2019 ◽

Vol 6 (1) ◽

Cited By ~ 27

Author(s):

Si Zhang ◽

Hanghang Tong ◽

Jiejun Xu ◽

Ross Maciejewski

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Network Models ◽

Representation Learning ◽

Superior Performance ◽

Learning Models ◽

Convolutional Network ◽

Comprehensive Review ◽

Convolutional Networks ◽

Graph Neural Networks

Abstract Graphs naturally appear in numerous application domains, ranging from social analysis, bioinformatics to computer vision. The unique capability of graphs enables capturing the structural relations among data, and thus allows to harvest more insights compared to analyzing data in isolation. However, it is often very challenging to solve the learning problems on graphs, because (1) many types of data are not originally structured as graphs, such as images and text data, and (2) for graph-structured data, the underlying connectivity patterns are often complex and diverse. On the other hand, the representation learning has achieved great successes in many areas. Thereby, a potential solution is to learn the representation of graphs in a low-dimensional Euclidean space, such that the graph properties can be preserved. Although tremendous efforts have been made to address the graph representation learning problem, many of them still suffer from their shallow learning mechanisms. Deep learning models on graphs (e.g., graph neural networks) have recently emerged in machine learning and other related areas, and demonstrated the superior performance in various problems. In this survey, despite numerous types of graph neural networks, we conduct a comprehensive review specifically on the emerging field of graph convolutional networks, which is one of the most prominent graph deep learning models. First, we group the existing graph convolutional network models into two categories based on the types of convolutions and highlight some graph convolutional network models in details. Then, we categorize different graph convolutional networks according to the areas of their applications. Finally, we present several open challenges in this area and discuss potential directions for future research.

Download Full-text

Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks for Fake News Detection

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3451215 ◽

2021 ◽

Vol 17 (3) ◽

pp. 1-23

Author(s):

Shengsheng Qian ◽

Jun Hu ◽

Quan Fang ◽

Changsheng Xu

Keyword(s):

Social Media ◽

Visual Information ◽

Representation Learning ◽

Fake News ◽

Unified Framework ◽

Model Learning ◽

Convolutional Network ◽

Textual Information ◽

Convolutional Networks ◽

Real World Datasets

In this article, we focus on fake news detection task and aim to automatically identify the fake news from vast amount of social media posts. To date, many approaches have been proposed to detect fake news, which includes traditional learning methods and deep learning-based models. However, there are three existing challenges: (i) How to represent social media posts effectively, since the post content is various and highly complicated; (ii) how to propose a data-driven method to increase the flexibility of the model to deal with the samples in different contexts and news backgrounds; and (iii) how to fully utilize the additional auxiliary information (the background knowledge and multi-modal information) of posts for better representation learning. To tackle the above challenges, we propose a novel Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks (KMAGCN) to capture the semantic representations by jointly modeling the textual information, knowledge concepts, and visual information into a unified framework for fake news detection. We model posts as graphs and use a knowledge-aware multi-modal adaptive graph learning principal for the effective feature learning. Compared with existing methods, the proposed KMAGCN addresses challenges from three aspects: (1) It models posts as graphs to capture the non-consecutive and long-range semantic relations; (2) it proposes a novel adaptive graph convolutional network to handle the variability of graph data; and (3) it leverages textual information, knowledge concepts and visual information jointly for model learning. We have conducted extensive experiments on three public real-world datasets and superior results demonstrate the effectiveness of KMAGCN compared with other state-of-the-art algorithms.

Download Full-text

Sentence Compression Using BERT and Graph Convolutional Networks

Applied Sciences ◽

10.3390/app11219910 ◽

2021 ◽

Vol 11 (21) ◽

pp. 9910

Author(s):

Yo-Han Park ◽

Gyong-Ho Lee ◽

Yong-Seok Choi ◽

Kong-Joo Lee

Keyword(s):

Language Processing ◽

Large Scale ◽

Convolutional Network ◽

Convolutional Networks ◽

Compression Model ◽

Sentence Compression ◽

Dependency Tree ◽

Proposed Model ◽

Syntactic Information ◽

Input Sentence

Sentence compression is a natural language-processing task that produces a short paraphrase of an input sentence by deleting words from the input sentence while ensuring grammatical correctness and preserving meaningful core information. This study introduces a graph convolutional network (GCN) into a sentence compression task to encode syntactic information, such as dependency trees. As we upgrade the GCN to activate a directed edge, the compression model with the GCN layers can distinguish between parent and child nodes in a dependency tree when aggregating adjacent nodes. Furthermore, by increasing the number of GCN layers, the model can gradually collect high-order information of a dependency tree when propagating node information through the layers. We implement a sentence compression model for Korean and English, respectively. This model consists of three components: pre-trained BERT model, GCN layers, and a scoring layer. The scoring layer can determine whether a word should remain in a compressed sentence by relying on the word vector containing contextual and syntactic information encoded by BERT and GCN layers. To train and evaluate the proposed model, we used the Google sentence compression dataset for English and a Korean sentence compression corpus containing about 140,000 sentence pairs for Korean. The experimental results demonstrate that the proposed model achieves state-of-the-art performance for English. To the best of our knowledge, this sentence compression model based on the deep learning model trained with a large-scale corpus is the first attempt for Korean.

Download Full-text

Multi-task temporal convolutional networks for joint recognition of surgical phases and steps in gastric bypass procedures

International Journal of Computer Assisted Radiology and Surgery ◽

10.1007/s11548-021-02388-z ◽

2021 ◽

Author(s):

Sanat Ramesh ◽

Diego Dall’Alba ◽

Cristians Gonzalez ◽

Tong Yu ◽

Pietro Mascagni ◽

...

Keyword(s):

Gastric Bypass ◽

Automatic Segmentation ◽

Joint Modeling ◽

Superior Performance ◽

Computer Assisted ◽

Convolutional Network ◽

Single Task ◽

Convolutional Networks ◽

Multi Stage ◽

Gastric Bypass Procedure

Abstract Purpose Automatic segmentation and classification of surgical activity is crucial for providing advanced support in computer-assisted interventions and autonomous functionalities in robot-assisted surgeries. Prior works have focused on recognizing either coarse activities, such as phases, or fine-grained activities, such as gestures. This work aims at jointly recognizing two complementary levels of granularity directly from videos, namely phases and steps. Methods We introduce two correlated surgical activities, phases and steps, for the laparoscopic gastric bypass procedure. We propose a multi-task multi-stage temporal convolutional network (MTMS-TCN) along with a multi-task convolutional neural network (CNN) training setup to jointly predict the phases and steps and benefit from their complementarity to better evaluate the execution of the procedure. We evaluate the proposed method on a large video dataset consisting of 40 surgical procedures (Bypass40). Results We present experimental results from several baseline models for both phase and step recognition on the Bypass40. The proposed MTMS-TCN method outperforms single-task methods in both phase and step recognition by 1-2% in accuracy, precision and recall. Furthermore, for step recognition, MTMS-TCN achieves a superior performance of 3-6% compared to LSTM-based models on all metrics. Conclusion In this work, we present a multi-task multi-stage temporal convolutional network for surgical activity recognition, which shows improved results compared to single-task models on a gastric bypass dataset with multi-level annotations. The proposed method shows that the joint modeling of phases and steps is beneficial to improve the overall recognition of each type of activity.

Download Full-text

Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6048 ◽

2020 ◽

Vol 34 (04) ◽

pp. 5892-5899

Author(s):

Ke Sun ◽

Zhouchen Lin ◽

Zhanxing Zhu

Keyword(s):

Supervised Learning ◽

State Of The Art ◽

Difficult Problem ◽

Superior Performance ◽

Learning Approach ◽

Training Algorithm ◽

Convolutional Network ◽

Convolutional Networks ◽

Learning Tasks ◽

Multi Stage

Graph Convolutional Networks (GCNs) play a crucial role in graph learning tasks, however, learning graph embedding with few supervised signals is still a difficult problem. In this paper, we propose a novel training algorithm for Graph Convolutional Network, called Multi-Stage Self-Supervised (M3S) Training Algorithm, combined with self-supervised learning approach, focusing on improving the generalization performance of GCNs on graphs with few labeled nodes. Firstly, a Multi-Stage Training Framework is provided as the basis of M3S training method. Then we leverage DeepCluster technique, a popular form of self-supervised learning, and design corresponding aligning mechanism on the embedding space to refine the Multi-Stage Training Framework, resulting in M3S Training Algorithm. Finally, extensive experimental results verify the superior performance of our algorithm on graphs with few labeled nodes under different label rates compared with other state-of-the-art approaches.

Download Full-text

Multi-Class Imbalanced Graph Convolutional Network Learning

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/398 ◽

2020 ◽

Author(s):

Min Shi ◽

Yufei Tang ◽

Xingquan Zhu ◽

David Wilson ◽

Jianxun Liu

Keyword(s):

State Of The Art ◽

Representation Learning ◽

Convolutional Network ◽

Network Learning ◽

Convolutional Networks ◽

Space Experiments ◽

Latent Distribution ◽

Adversarial Training ◽

Quality Of Fit

Networked data often demonstrate the Pareto principle (i.e., 80/20 rule) with skewed class distributions, where most vertices belong to a few majority classes and minority classes only contain a handful of instances. When presented with imbalanced class distributions, existing graph embedding learning tends to bias to nodes from majority classes, leaving nodes from minority classes under-trained. In this paper, we propose Dual-Regularized Graph Convolutional Networks (DR-GCN) to handle multi-class imbalanced graphs, where two types of regularization are imposed to tackle class imbalanced representation learning. To ensure that all classes are equally represented, we propose a class-conditioned adversarial training process to facilitate the separation of labeled nodes. Meanwhile, to maintain training equilibrium (i.e., retaining quality of fit across all classes), we force unlabeled nodes to follow a similar latent distribution to the labeled nodes by minimizing their difference in the embedding space. Experiments on real-world imbalanced graphs demonstrate that DR-GCN outperforms the state-of-the-art methods in node classification, graph clustering, and visualization.

Download Full-text

A gating context-aware text classification model with BERT and graph convolutional networks

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-201051 ◽

2021 ◽

pp. 1-13

Author(s):

Weiqi Gao ◽

Hao Huang

Keyword(s):

Text Classification ◽

Structural Data ◽

Classification Model ◽

Context Aware ◽

Convolutional Network ◽

Convolutional Networks ◽

Gating Mechanism ◽

Document Frequency ◽

Proposed Model ◽

Benchmark Datasets

Graph convolutional networks (GCNs), which are capable of effectively processing graph-structural data, have been successfully applied in text classification task. Existing studies on GCN based text classification model largely concerns with the utilization of word co-occurrence and Term Frequency-Inverse Document Frequency (TF–IDF) information for graph construction, which to some extent ignore the context information of the texts. To solve this problem, we propose a gating context-aware text classification model with Bidirectional Encoder Representations from Transformers (BERT) and graph convolutional network, named as Gating Context GCN (GC-GCN). More specifically, we integrates the graph embedding with BERT embedding by using a GCN with gating mechanism enables the acquisition of context coding. We carry out text classification experiments to show the effectiveness of the proposed model. Experimental results shown our model has respectively obtained 0.19%, 0.57%, 1.05% and 1.17% improvements over the Text-GCN baseline on the 20NG, R8, R52, and Ohsumed benchmark datasets. Furthermore, to overcome the problem that word co-occurrence and TF–IDF are not suitable for graph construction for short texts, Euclidean distance is used to combine with word co-occurrence and TF–IDF information. We obtain an improvement by 1.38% on the MR dataset compared to Text-GCN baseline.

Download Full-text

EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.5984 ◽

2020 ◽

Vol 34 (04) ◽

pp. 5363-5370 ◽

Cited By ~ 2

Author(s):

Aldo Pareja ◽

Giacomo Domeniconi ◽

Jie Chen ◽

Tengfei Ma ◽

Toyotaro Suzumura ◽

...

Keyword(s):

Neural Networks ◽

Temporal Dynamics ◽

Representation Learning ◽

Graph Representation ◽

Full Time ◽

Convolutional Network ◽

Frequent Change ◽

Temporal Dimension ◽

Convolutional Networks ◽

Node Embeddings

Graph representation learning resurges as a trending research subject owing to the widespread use of deep learning for Euclidean data, which inspire various creative designs of neural networks in the non-Euclidean domain, particularly graphs. With the success of these graph neural networks (GNN) in the static setting, we approach further practical scenarios where the graph dynamically evolves. Existing approaches typically resort to node embeddings and use a recurrent neural network (RNN, broadly speaking) to regulate the embeddings and learn the temporal dynamics. These methods require the knowledge of a node in the full time span (including both training and testing) and are less applicable to the frequent change of the node set. In some extreme scenarios, the node sets at different time steps may completely differ. To resolve this challenge, we propose EvolveGCN, which adapts the graph convolutional network (GCN) model along the temporal dimension without resorting to node embeddings. The proposed approach captures the dynamism of the graph sequence through using an RNN to evolve the GCN parameters. Two architectures are considered for the parameter evolution. We evaluate the proposed approach on tasks including link prediction, edge classification, and node classification. The experimental results indicate a generally higher performance of EvolveGCN compared with related approaches. The code is available at https://github.com/IBM/EvolveGCN.

Download Full-text

Dual-dropout graph convolutional network for predicting synthetic lethality in human cancers

Bioinformatics ◽

10.1093/bioinformatics/btaa211 ◽

2020 ◽

Vol 36 (16) ◽

pp. 4458-4465 ◽

Cited By ~ 4

Author(s):

Ruichu Cai ◽

Xuexin Chen ◽

Yuan Fang ◽

Min Wu ◽

Yuexing Hao

Keyword(s):

Link Prediction ◽

Synthetic Lethality ◽

Gene Interaction ◽

Superior Performance ◽

Coarse Grained ◽

Supplementary Information ◽

Convolutional Network ◽

Sparse Graphs ◽

Fine Grained ◽

Convolutional Networks

Abstract Motivation Synthetic lethality (SL) is a promising form of gene interaction for cancer therapy, as it is able to identify specific genes to target at cancer cells without disrupting normal cells. As high-throughput wet-lab settings are often costly and face various challenges, computational approaches have become a practical complement. In particular, predicting SLs can be formulated as a link prediction task on a graph of interacting genes. Although matrix factorization techniques have been widely adopted in link prediction, they focus on mapping genes to latent representations in isolation, without aggregating information from neighboring genes. Graph convolutional networks (GCN) can capture such neighborhood dependency in a graph. However, it is still challenging to apply GCN for SL prediction as SL interactions are extremely sparse, which is more likely to cause overfitting. Results In this article, we propose a novel dual-dropout GCN (DDGCN) for learning more robust gene representations for SL prediction. We employ both coarse-grained node dropout and fine-grained edge dropout to address the issue that standard dropout in vanilla GCN is often inadequate in reducing overfitting on sparse graphs. In particular, coarse-grained node dropout can efficiently and systematically enforce dropout at the node (gene) level, while fine-grained edge dropout can further fine-tune the dropout at the interaction (edge) level. We further present a theoretical framework to justify our model architecture. Finally, we conduct extensive experiments on human SL datasets and the results demonstrate the superior performance of our model in comparison with state-of-the-art methods. Availability and implementation DDGCN is implemented in Python 3.7, open-source and freely available at https://github.com/CXX1113/Dual-DropoutGCN. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text