Multi-Channel Graph Neural Networks

The classification of graph-structured data has be-come increasingly crucial in many disciplines. It has been observed that the implicit or explicit hierarchical community structures preserved in real-world graphs could be useful for downstream classification applications. A straightforward way to leverage the hierarchical structure is to make use the pooling algorithms to cluster nodes into fixed groups, and shrink the input graph layer by layer to learn the pooled graphs.However, the pool shrinking discards the graph details to make it hard to distinguish two non-isomorphic graphs, and the fixed clustering ignores the inherent multiple characteristics of nodes. To compensate the shrinking loss and learn the various nodes’ characteristics, we propose the multi-channel graph neural networks (MuchGNN). Motivated by the underlying mechanisms developed in convolutional neural networks, we define the tailored graph convolutions to learn a series of graph channels at each layer, and shrink the graphs hierarchically to en-code the pooled structures. Experimental results on real-world datasets demonstrate the superiority of MuchGNN over the state-of-the-art methods.

Download Full-text

Efficient streaming subgraph isomorphism with graph neural networks

Proceedings of the VLDB Endowment ◽

10.14778/3446095.3446097 ◽

2021 ◽

Vol 14 (5) ◽

pp. 730-742

Author(s):

Chi Thang Duong ◽

Trung Dung Hoang ◽

Hongzhi Yin ◽

Matthias Weidlich ◽

Quoc Viet Hung Nguyen ◽

...

Keyword(s):

Neural Networks ◽

Data Management ◽

Real World ◽

Subgraph Isomorphism ◽

Graph Embeddings ◽

Dynamic Setting ◽

Continuous Stream ◽

Management Policies ◽

Real World Datasets ◽

Graph Neural Networks

Queries to detect isomorphic subgraphs are important in graph-based data management. While the problem of subgraph isomorphism search has received considerable attention for the static setting of a single query, or a batch thereof, existing approaches do not scale to a dynamic setting of a continuous stream of queries. In this paper, we address the scalability challenges induced by a stream of subgraph isomorphism queries by caching and re-use of previous results. We first present a novel subgraph index based on graph embeddings that serves as the foundation for efficient stream processing. It enables not only effective caching and re-use of results, but also speeds-up traditional algorithms for subgraph isomorphism in case of cache misses. Moreover, we propose cache management policies that incorporate notions of reusability of query results. Experiments using real-world datasets demonstrate the effectiveness of our approach in handling isomorphic subgraph search for streams of queries.

Download Full-text

Fuzzy Overclustering: Semi-Supervised Classification of Fuzzy Labels with Overclustering and Inverse Cross-Entropy

Sensors ◽

10.3390/s21196661 ◽

2021 ◽

Vol 21 (19) ◽

pp. 6661

Author(s):

Lars Schmarje ◽

Johannes Brünger ◽

Monty Santarossa ◽

Simon-Martin Schröder ◽

Rainer Kiko ◽

...

Keyword(s):

Deep Learning ◽

Real World ◽

Supervised Classification ◽

Limited Information ◽

Classification Problems ◽

Previous State ◽

Supervised Methods ◽

Real World Datasets ◽

Fuzzy Labels

Deep learning has been successfully applied to many classification problems including underwater challenges. However, a long-standing issue with deep learning is the need for large and consistently labeled datasets. Although current approaches in semi-supervised learning can decrease the required amount of annotated data by a factor of 10 or even more, this line of research still uses distinct classes. For underwater classification, and uncurated real-world datasets in general, clean class boundaries can often not be given due to a limited information content in the images and transitional stages of the depicted objects. This leads to different experts having different opinions and thus producing fuzzy labels which could also be considered ambiguous or divergent. We propose a novel framework for handling semi-supervised classifications of such fuzzy labels. It is based on the idea of overclustering to detect substructures in these fuzzy labels. We propose a novel loss to improve the overclustering capability of our framework and show the benefit of overclustering for fuzzy labels. We show that our framework is superior to previous state-of-the-art semi-supervised methods when applied to real-world plankton data with fuzzy labels. Moreover, we acquire 5 to 10% more consistent predictions of substructures.

Download Full-text

Graph Neural Networks: Taxonomy, Advances, and Trends

ACM Transactions on Intelligent Systems and Technology ◽

10.1145/3495161 ◽

2022 ◽

Vol 13 (1) ◽

pp. 1-54

Author(s):

Yu Zhou ◽

Haixia Zheng ◽

Xin Huang ◽

Shufeng Hao ◽

Dengao Li ◽

...

Keyword(s):

Neural Networks ◽

Real World ◽

Research Community ◽

Future Research ◽

Research Directions ◽

Comprehensive Review ◽

Future Research Directions ◽

Graph Neural Networks ◽

Low Dimensional

Graph neural networks provide a powerful toolkit for embedding real-world graphs into low-dimensional spaces according to specific tasks. Up to now, there have been several surveys on this topic. However, they usually lay emphasis on different angles so that the readers cannot see a panorama of the graph neural networks. This survey aims to overcome this limitation and provide a systematic and comprehensive review on the graph neural networks. First of all, we provide a novel taxonomy for the graph neural networks, and then refer to up to 327 relevant literatures to show the panorama of the graph neural networks. All of them are classified into the corresponding categories. In order to drive the graph neural networks into a new stage, we summarize four future research directions so as to overcome the challenges faced. It is expected that more and more scholars can understand and exploit the graph neural networks and use them in their research community.

Download Full-text

Neural Networks in Credit Risk Classification of Companies in the Construction Sector

Econometric Research in Finance ◽

10.33119/erfin.2017.2.2.1 ◽

2018 ◽

Vol 2 (2) ◽

pp. 63-77 ◽

Cited By ~ 1

Author(s):

Aleksandra Wójcicka

Keyword(s):

Neural Networks ◽

Decision Making ◽

Credit Risk ◽

Financial Institutions ◽

Real World ◽

Decision Making Process ◽

Construction Sector ◽

Real World Data ◽

Input Variables

The financial sector (banks, financial institutions, etc.) is the sector most exposed to financial and credit risk, as one of the basic objectives of banks' activity (as a specific enterprise) is granting credit and loans. Because credit risk is one of the problems constantly faced by banks, identification of potential good and bad customers is an extremely important task. This paper investigates the use of different structures of neural networks to support the preliminary credit risk decision-making process. The results are compared among the models and juxtaposed with real-world data. Moreover, different sets and subsets of entry data are analyzed to find the best input variables (financial ratios).

Download Full-text

UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/353 ◽

2021 ◽

Author(s):

Jing Huang ◽

Jie Yang

Keyword(s):

Neural Networks ◽

Message Passing ◽

State Of The Art ◽

Representation Learning ◽

Graph Representation ◽

Challenging Problem ◽

Unified Framework ◽

Real World Datasets ◽

Graph Neural Networks ◽

Research Domains

Hypergraph, an expressive structure with flexibility to model the higher-order correlations among entities, has recently attracted increasing attention from various research domains. Despite the success of Graph Neural Networks (GNNs) for graph representation learning, how to adapt the powerful GNN-variants directly into hypergraphs remains a challenging problem. In this paper, we propose UniGNN, a unified framework for interpreting the message passing process in graph and hypergraph neural networks, which can generalize general GNN models into hypergraphs. In this framework, meticulously-designed architectures aiming to deepen GNNs can also be incorporated into hypergraphs with the least effort. Extensive experiments have been conducted to demonstrate the effectiveness of UniGNN on multiple real-world datasets, which outperform the state-of-the-art approaches with a large margin. Especially for the DBLP dataset, we increase the accuracy from 77.4% to 88.8% in the semi-supervised hypernode classification task. We further prove that the proposed message-passing based UniGNN models are at most as powerful as the 1-dimensional Generalized Weisfeiler-Leman (1-GWL) algorithm in terms of distinguishing non-isomorphic hypergraphs. Our code is available at https://github.com/OneForward/UniGNN.

Download Full-text

Semi-Supervised Deep Metric Learning Networks for Classification of Polarimetric SAR Data

Remote Sensing ◽

10.3390/rs12101593 ◽

2020 ◽

Vol 12 (10) ◽

pp. 1593

Author(s):

Hongying Liu ◽

Ruyi Luo ◽

Fanhua Shang ◽

Xuechun Meng ◽

Shuiping Gou ◽

...

Keyword(s):

Real World ◽

Metric Learning ◽

Feature Learning ◽

Linear Mapping ◽

Layer By Layer ◽

Learning Networks ◽

Distance Metric ◽

Linear Projection ◽

Deep Metric Learning

Recently, classification methods based on deep learning have attained sound results for the classification of Polarimetric synthetic aperture radar (PolSAR) data. However, they generally require a great deal of labeled data to train their models, which limits their potential real-world applications. This paper proposes a novel semi-supervised deep metric learning network (SSDMLN) for feature learning and classification of PolSAR data. Inspired by distance metric learning, we construct a network, which transforms the linear mapping of metric learning into the non-linear projection in the layer-by-layer learning. With the prior knowledge of the sample categories, the network also learns a distance metric under which all pairs of similarly labeled samples are closer and dissimilar samples have larger relative distances. Moreover, we introduce a new manifold regularization to reduce the distance between neighboring samples since they are more likely to be homogeneous. The categorizing is achieved by using a simple classifier. Several experiments on both synthetic and real-world PolSAR data from different sensors are conducted and they demonstrate the effectiveness of SSDMLN with limited labeled samples, and SSDMLN is superior to state-of-the-art methods.

Download Full-text

Knowledge-Enhanced Graph Neural Networks for Sequential Recommendation

Information ◽

10.3390/info11080388 ◽

2020 ◽

Vol 11 (8) ◽

pp. 388

Author(s):

Baocheng Wang ◽

Wentao Cai

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Internet Technology ◽

Sequential Data ◽

Current Interest ◽

Important Method ◽

Real World Datasets ◽

Set Up ◽

Graph Neural Networks ◽

Better Than

With the rapid increase in the popularity of big data and internet technology, sequential recommendation has become an important method to help people find items they are potentially interested in. Traditional recommendation methods use only recurrent neural networks (RNNs) to process sequential data. Although effective, the results may be unable to capture both the semantic-based preference and the complex transitions between items adequately. In this paper, we model separated session sequences into session graphs and capture complex transitions using graph neural networks (GNNs). We further link items in interaction sequences with existing external knowledge base (KB) entities and integrate the GNN-based recommender with key-value memory networks (KV-MNs) to incorporate KB knowledge. Specifically, we set a key matrix to many relation embeddings that learned from KB, corresponding to many entity attributes, and set up a set of value matrices storing the semantic-based preferences of different users for the corresponding attribute. By using a hybrid of a GNN and KV-MN, each session is represented as the combination of the current interest (i.e., sequential preference) and the global preference (i.e., semantic-based preference) of that session. Extensive experiments on three public real-world datasets show that our method performs better than baseline algorithms consistently.

Download Full-text

Learning Graph Neural Networks with Positive and Unlabeled Nodes

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3450316 ◽

2021 ◽

Vol 15 (6) ◽

pp. 1-25

Author(s):

Man Wu ◽

Shirui Pan ◽

Lan Du ◽

Xingquan Zhu

Keyword(s):

Neural Networks ◽

Short Distance ◽

Attention Mechanism ◽

Long Distance ◽

Model Learning ◽

Transductive Learning ◽

Single Class ◽

Aggregate Information ◽

Real World Datasets ◽

Graph Neural Networks

Graph neural networks (GNNs) are important tools for transductive learning tasks, such as node classification in graphs, due to their expressive power in capturing complex interdependency between nodes. To enable GNN learning, existing works typically assume that labeled nodes, from two or multiple classes, are provided, so that a discriminative classifier can be learned from the labeled data. In reality, this assumption might be too restrictive for applications, as users may only provide labels of interest in a single class for a small number of nodes. In addition, most GNN models only aggregate information from short distances ( e.g. , 1-hop neighbors) in each round, and fail to capture long-distance relationship in graphs. In this article, we propose a novel GNN framework, long-short distance aggregation networks, to overcome these limitations. By generating multiple graphs at different distance levels, based on the adjacency matrix, we develop a long-short distance attention model to model these graphs. The direct neighbors are captured via a short-distance attention mechanism, and neighbors with long distance are captured by a long-distance attention mechanism. Two novel risk estimators are further employed to aggregate long-short-distance networks, for PU learning and the loss is back-propagated for model learning. Experimental results on real-world datasets demonstrate the effectiveness of our algorithm.

Download Full-text

Structured Probabilistic End-to-End Learning from Crowds

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/210 ◽

2020 ◽

Author(s):

Zhijun Chen ◽

Huimin Wang ◽

Hailong Sun ◽

Pengpeng Chen ◽

Tao Han ◽

...

Keyword(s):

Neural Networks ◽

Real World ◽

Probabilistic Model ◽

Deep Neural Networks ◽

State Of The Art ◽

Probabilistic Approach ◽

Probabilistic Interpretation ◽

End To End ◽

Real World Datasets ◽

The Relationship

End-to-end learning from crowds has recently been introduced as an EM-free approach to training deep neural networks directly from noisy crowdsourced annotations. It models the relationship between true labels and annotations with a specific type of neural layer, termed as the crowd layer, which can be trained using pure backpropagation. Parameters of the crowd layer, however, can hardly be interpreted as annotator reliability, as compared with the more principled probabilistic approach. The lack of probabilistic interpretation further prevents extensions of the approach to account for important factors of annotation processes, e.g., instance difficulty. This paper presents SpeeLFC, a structured probabilistic model that incorporates the constraints of probability axioms for parameters of the crowd layer, which allows to explicitly model annotator reliability while benefiting from the end-to-end training of neural networks. Moreover, we propose SpeeLFC-D, which further takes into account instance difficulty. Extensive validation on real-world datasets shows that our methods improve the state-of-the-art.

Download Full-text

Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3460865 ◽

2022 ◽

Vol 16 (2) ◽

pp. 1-18

Author(s):

Hanlu Wu ◽

Tengfei Ma ◽

Lingfei Wu ◽

Fangli Xu ◽

Shouling Ji

Keyword(s):

Neural Network ◽

Neural Networks ◽

State Of The Art ◽

Superior Performance ◽

True Label ◽

Label Aggregation ◽

Correlation Information ◽

Real World Datasets ◽

Graph Neural Networks ◽

High Level

Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts. However, due to the high level of noise from the non-experts, a label aggregation model that infers the true label from noisy crowdsourced labels is required. In this article, we propose a novel framework based on graph neural networks for aggregating crowd labels. We construct a heterogeneous graph between workers and tasks and derive a new graph neural network to learn the representations of nodes and the true labels. Besides, we exploit the unknown latent interaction between the same type of nodes (workers or tasks) by adding a homogeneous attention layer in the graph neural networks. Experimental results on 13 real-world datasets show superior performance over state-of-the-art models.

Download Full-text