scholarly journals Independence Promoted Graph Disentangled Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 4916-4923
Author(s):  
Yanbei Liu ◽  
Xiao Wang ◽  
Shu Wu ◽  
Zhitao Xiao
Keyword(s):  
Real World ◽  
Experimental Studies ◽  
Representation Learning ◽  
Graph Visualization ◽  
Graph Classification ◽  
Latent Factors ◽  
Convolutional Networks ◽  
Network Applications ◽  
The Arts ◽  
Latent Representations

We address the problem of disentangled representation learning with independent latent factors in graph convolutional networks (GCNs). The current methods usually learn node representation by describing its neighborhood as a perceptual whole in a holistic manner while ignoring the entanglement of the latent factors. However, a real-world graph is formed by the complex interaction of many latent factors (e.g., the same hobby, education or work in social network). While little effort has been made toward exploring the disentangled representation in GCNs. In this paper, we propose a novel Independence Promoted Graph Disentangled Networks (IPGDN) to learn disentangled node representation while enhancing the independence among node representations. In particular, we firstly present disentangled representation learning by neighborhood routing mechanism, and then employ the Hilbert-Schmidt Independence Criterion (HSIC) to enforce independence between the latent representations, which is effectively integrated into a graph convolutional framework as a regularizer at the output layer. Experimental studies on real-world graphs validate our model and demonstrate that our algorithms outperform the state-of-the-arts by a wide margin in different network applications, including semi-supervised graph classification, graph clustering and graph visualization.

scholarly journals Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity

2019 ◽  
Author(s):  
Yunsheng Bai ◽  
Hao Ding ◽  
Yang Qiao ◽  
Agustin Marinovic ◽  
Ken Gu ◽  
...  
Keyword(s):  
Neural Network ◽  
Vector Space ◽  
General Framework ◽  
Representation Learning ◽  
Generation Mechanism ◽  
Graph Visualization ◽  
Graph Classification ◽  
Training Set ◽  
Multi Scale ◽  
Novel Approach

We introduce a novel approach to graph-level representation learning, which is to embed an entire graph into a vector space where the embeddings of two graphs preserve their graph-graph proximity. Our approach, UGraphEmb, is a general framework that provides a novel means to performing graph-level embedding in a completely unsupervised and inductive manner. The learned neural network can be considered as a function that receives any graph as input, either seen or unseen in the training set, and transforms it into an embedding. A novel graph-level embedding generation mechanism called Multi-Scale Node Attention (MSNA), is proposed. Experiments on five real graph datasets show that UGraphEmb achieves competitive accuracy in the tasks of graph classification, similarity ranking, and graph visualization.

scholarly journals GSSNN: Graph Smoothing Splines Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 7007-7014
Author(s):  
Shichao Zhu ◽  
Lewei Zhou ◽  
Shirui Pan ◽  
Chuan Zhou ◽  
Guiying Yan ◽  
...  
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Representation Learning ◽  
Smoothing Splines ◽  
Graph Representation ◽  
Graph Classification ◽  
Topological Information ◽  
Graph Representations ◽  
The Arts ◽  
Graph Neural Networks

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in many graph data analysis tasks. However, they still suffer from two limitations for graph representation learning. First, they exploit non-smoothing node features which may result in suboptimal embedding and degenerated performance for graph classification. Second, they only exploit neighbor information but ignore global topological knowledge. Aiming to overcome these limitations simultaneously, in this paper, we propose a novel, flexible, and end-to-end framework, Graph Smoothing Splines Neural Networks (GSSNN), for graph classification. By exploiting the smoothing splines, which are widely used to learn smoothing fitting function in regression, we develop an effective feature smoothing and enhancement module Scaled Smoothing Splines (S3) to learn graph embedding. To integrate global topological information, we design a novel scoring module, which exploits closeness, degree, as well as self-attention values, to select important node features as knots for smoothing splines. These knots can be potentially used for interpreting classification results. In extensive experiments on biological and social datasets, we demonstrate that our model achieves state-of-the-arts and GSSNN is superior in learning more robust graph representations. Furthermore, we show that S3 module is easily plugged into existing GNNs to improve their performance.

scholarly journals Graph convolutional and attention models for entity classification in multilayer networks

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Lorenzo Zangari ◽  
Roberto Interdonato ◽  
Antonio Calió ◽  
Andrea Tagarelli
Keyword(s):  
State Of The Art ◽  
Structural Characteristics ◽  
Representation Learning ◽  
Efficient Solutions ◽  
Single Type ◽  
Graph Classification ◽  
Multilayer Networks ◽  
Attention Networks ◽  
Convolutional Networks ◽  
Graph Neural Networks

AbstractGraph Neural Networks (GNNs) are powerful tools that are nowadays reaching state of the art performances in a plethora of different tasks such as node classification, link prediction and graph classification. A challenging aspect in this context is to redefine basic deep learning operations, such as convolution, on graph-like structures, where nodes generally have unordered neighborhoods of varying size. State-of-the-art GNN approaches such as Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs) work on monoplex networks only, i.e., on networks modeling a single type of relation among an homogeneous set of nodes. The aim of this work is to generalize such approaches by proposing a GNN framework for representation learning and semi-supervised classification in multilayer networks with attributed entities, and arbitrary number of layers and intra-layer and inter-layer connections between nodes. We instantiate our framework with two new formulations of GAT and GCN models, namely and , specifically devised for general, attributed multilayer networks. The proposed approaches are evaluated on an entity classification task on nine widely used real-world network datasets coming from different domains and with different structural characteristics. Results show that both our proposed and methods provide effective and efficient solutions to the problem of entity classification in multilayer attributed networks, being faster to learn and offering better accuracy than the competitors. Furthermore, results show how our methods are able to take advantage of the presence of real attributes for the entities, in addition to arbitrary inter-layer connections between the nodes in the various layers.

scholarly journals CuCo: Graph Representation with Curriculum Contrastive Learning

2021 ◽  
Author(s):  
Guanyi Chu ◽  
Xiao Wang ◽  
Chuan Shi ◽  
Xunqiang Jiang
Keyword(s):  
Real World ◽  
Scoring Function ◽  
Representation Learning ◽  
Parameter Analysis ◽  
Graph Representation ◽  
Training Procedure ◽  
Graph Classification ◽  
Graph Augmentation ◽  
Real World Datasets ◽  
The Impact

Graph-level representation learning is to learn low-dimensional representation for the entire graph, which has shown a large impact on real-world applications. Recently, limited by expensive labeled data, contrastive learning based graph-level representation learning attracts considerable attention. However, these methods mainly focus on graph augmentation for positive samples, while the effect of negative samples is less explored. In this paper, we study the impact of negative samples on learning graph-level representations, and a novel curriculum contrastive learning framework for self-supervised graph-level representation, called CuCo, is proposed. Specifically, we introduce four graph augmentation techniques to obtain the positive and negative samples, and utilize graph neural networks to learn their representations. Then a scoring function is proposed to sort negative samples from easy to hard and a pacing function is to automatically select the negative samples in each training procedure. Extensive experiments on fifteen graph classification real-world datasets, as well as the parameter analysis, well demonstrate that our proposed CuCo yields truly encouraging results in terms of performance on classification and convergence.

scholarly journals Adversarially Regularized Graph Autoencoder for Graph Embedding

2018 ◽  
Author(s):  
Shirui Pan ◽  
Ruiqi Hu ◽  
Guodong Long ◽  
Jing Jiang ◽  
Lina Yao ◽  
...  
Keyword(s):  
Real World ◽  
Topological Structure ◽  
Dimensional Space ◽  
Experimental Studies ◽  
Graph Embedding ◽  
Compact Representation ◽  
Graph Visualization ◽  
Graph Data ◽  
Training Scheme ◽  
Low Dimensional

Graph embedding is an effective method to represent graph data in a low dimensional space for graph analytics.  Most existing embedding algorithms typically focus on preserving the topological structure or minimizing the reconstruction errors of graph data,  but they have mostly ignored the data distribution of the latent codes from the graphs, which often results in inferior embedding in  real-world  graph data. In this paper, we propose a novel adversarial graph embedding framework for graph data. The framework encodes the topological structure and node content in a graph to a compact representation, on which a decoder is trained to reconstruct the graph structure. Furthermore, the latent representation is enforced to match a prior distribution via an adversarial training scheme. To learn a robust embedding,  two variants of adversarial approaches,  adversarially regularized graph autoencoder (ARGA) and adversarially regularized variational graph autoencoder (ARVGA), are developed. Experimental studies on real-world graphs validate our design and demonstrate that our algorithms outperform baselines by a wide margin in link prediction,  graph clustering, and graph visualization tasks.

Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks for Fake News Detection

2021 ◽  
Vol 17 (3) ◽  
pp. 1-23
Author(s):  
Shengsheng Qian ◽  
Jun Hu ◽  
Quan Fang ◽  
Changsheng Xu
Keyword(s):  
Social Media ◽  
Visual Information ◽  
Representation Learning ◽  
Fake News ◽  
Unified Framework ◽  
Model Learning ◽  
Convolutional Network ◽  
Textual Information ◽  
Convolutional Networks ◽  
Real World Datasets

In this article, we focus on fake news detection task and aim to automatically identify the fake news from vast amount of social media posts. To date, many approaches have been proposed to detect fake news, which includes traditional learning methods and deep learning-based models. However, there are three existing challenges: (i) How to represent social media posts effectively, since the post content is various and highly complicated; (ii) how to propose a data-driven method to increase the flexibility of the model to deal with the samples in different contexts and news backgrounds; and (iii) how to fully utilize the additional auxiliary information (the background knowledge and multi-modal information) of posts for better representation learning. To tackle the above challenges, we propose a novel Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks (KMAGCN) to capture the semantic representations by jointly modeling the textual information, knowledge concepts, and visual information into a unified framework for fake news detection. We model posts as graphs and use a knowledge-aware multi-modal adaptive graph learning principal for the effective feature learning. Compared with existing methods, the proposed KMAGCN addresses challenges from three aspects: (1) It models posts as graphs to capture the non-consecutive and long-range semantic relations; (2) it proposes a novel adaptive graph convolutional network to handle the variability of graph data; and (3) it leverages textual information, knowledge concepts and visual information jointly for model learning. We have conducted extensive experiments on three public real-world datasets and superior results demonstrate the effectiveness of KMAGCN compared with other state-of-the-art algorithms.

What Can Experimental Studies of Bias Tell Us About Real-World Group Disparities?

2021 ◽  
pp. 1-82
Author(s):  
Joseph Cesario
Keyword(s):  
Real World ◽  
Experimental Studies ◽  
White Men ◽  
Human Mind ◽  
Behavioral Differences ◽  
Social Categories ◽  
Systematic Testing ◽  
Black And White ◽  
Research Tradition ◽  
The Social

Abstract This article questions the widespread use of experimental social psychology to understand real-world group disparities. Standard experimental practice is to design studies in which participants make judgments of targets who vary only on the social categories to which they belong. This is typically done under simplified decision landscapes and with untrained decision makers. For example, to understand racial disparities in police shootings, researchers show pictures of armed and unarmed Black and White men to undergraduates and have them press "shoot" and "don't shoot" buttons. Having demonstrated categorical bias under these conditions, researchers then use such findings to claim that real-world disparities are also due to decision-maker bias. I describe three flaws inherent in this approach, flaws which undermine any direct contribution of experimental studies to explaining group disparities. First, the decision landscapes used in experimental studies lack crucial components present in actual decisions (Missing Information Flaw). Second, categorical effects in experimental studies are not interpreted in light of other effects on outcomes, including behavioral differences across groups (Missing Forces Flaw). Third, there is no systematic testing of whether the contingencies required to produce experimental effects are present in real-world decisions (Missing Contingencies Flaw). I apply this analysis to three research topics to illustrate the scope of the problem. I discuss how this research tradition has skewed our understanding of the human mind within and beyond the discipline and how results from experimental studies of bias are generally misunderstood. I conclude by arguing that the current research tradition should be abandoned.

Exploiting Graph Convolutional Networks for Representation Learning of Mobile App Usage

2019 ◽  
Author(s):  
Keiichi Ochiai ◽  
Naoki Yamamoto ◽  
Takashi Hamatani ◽  
Yusuke Fukazawa ◽  
Takayasu Yamaguchi
Keyword(s):  
Representation Learning ◽  
Mobile App ◽  
Convolutional Networks

scholarly journals A DIVERSIFIED DEEP BELIEF NETWORK FOR HYPERSPECTRAL IMAGE CLASSIFICATION

2016 ◽  
Vol XLI-B7 ◽  
pp. 443-449
Author(s):  
P. Zhong ◽  
Z. Q. Gong ◽  
C. Schönlieb
Keyword(s):  
Image Classification ◽  
Real World ◽  
Hyperspectral Image ◽  
Classification Performance ◽  
Fine Tuning ◽  
Hyperspectral Image Classification ◽  
Latent Factors ◽  
Deep Model ◽  
Training Samples ◽  
Limited Training Samples

In recent years, researches in remote sensing demonstrated that deep architectures with multiple layers can potentially extract abstract and invariant features for better hyperspectral image classification. Since the usual real-world hyperspectral image classification task cannot provide enough training samples for a supervised deep model, such as convolutional neural networks (CNNs), this work turns to investigate the deep belief networks (DBNs), which allow unsupervised training. The DBN trained over limited training samples usually has many “dead” (never responding) or “potential over-tolerant” (always responding) latent factors (neurons), which decrease the DBN’s description ability and thus finally decrease the hyperspectral image classification performance. This work proposes a new diversified DBN through introducing a diversity promoting prior over the latent factors during the DBN pre-training and fine-tuning procedures. The diversity promoting prior in the training procedures will encourage the latent factors to be uncorrelated, such that each latent factor focuses on modelling unique information, and all factors will be summed up to capture a large proportion of information and thus increase description ability and classification performance of the diversified DBNs. The proposed method was evaluated over the well-known real-world hyperspectral image dataset. The experiments demonstrate that the diversified DBNs can obtain much better results than original DBNs and comparable or even better performances compared with other recent hyperspectral image classification methods.

scholarly journals Clustering-Based Relational Unsupervised Representation Learning with an Explicit Distributed Representation

2017 ◽  
Author(s):  
Sebastijan Dumancic ◽  
Hendrik Blockeel
Keyword(s):  
Relational Learning ◽  
Representation Learning ◽  
Relational Data ◽  
Distributed Representation ◽  
Learning Tasks ◽  
Wide Range ◽  
Lower Complexity ◽  
Classification Tasks ◽  
Latent Representations

The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describes relationships among instances. In this work we introduce an approach for relational unsupervised representation learning. Viewing a relational dataset as a hypergraph, new features are obtained by clustering vertices and hyperedges. To find a representation suited for many relational learning tasks, a wide range of similarities between relational objects is considered, e.g. feature and structural similarities. We experimentally evaluate the proposed approach and show that models learned on such latent representations perform better, have lower complexity, and outperform the existing approaches on classification tasks.

Sign in / Sign up

Export Citation Format

Share Document