Learning intents behind interactions with high-order graph for session-based intelligent recommendation

Session-based recommendation is an overwhelming task owing to the inherent ambiguity in anonymous behaviors. Graph convolutional neural networks are receiving wide attention for session-based recommendation research for the sake of their ability to capture the complex transitions of interactions between sessions. Recent research on session-based recommendations mainly focuses on sequential patterns by utilizing graph neural networks. However, it is undeniable that proposed methods are still difficult to capture higher-order interactions between contextual interactions in the same session and has room for improvement. To solve it, we propose a new method based on graph attention mechanism and target oriented items to effectively propagate information, HOGAN for brevity. Higher-order graph attention networks are used to select the importance of different neighborhoods in the graph that consists of a sequence of user actions for recommendation applications. The complementarity between high-order networks is adopted to aggregate and propagate useful signals from the long distant neighbors to solve the long-range dependency capturing problem. Experimental results consistently display that HOGAN has a significantly improvement to 71.53% on precision for the Yoochoose1_64 dataset and enhances the property of the session-based recommendation task.

Download Full-text

Multi-View Attribute Graph Convolution Networks for Clustering

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/411 ◽

2020 ◽

Cited By ~ 1

Author(s):

Jiafeng Cheng ◽

Qianqian Wang ◽

Zhiqiang Tao ◽

Deyan Xie ◽

Quanxue Gao

Keyword(s):

Neural Networks ◽

State Of The Art ◽

Graph Embedding ◽

Structured Data ◽

Attention Networks ◽

Graph Data ◽

Graph Reconstruction ◽

Node Attributes ◽

Graph Neural Networks ◽

Geometric Relationship

Graph neural networks (GNNs) have made considerable achievements in processing graph-structured data. However, existing methods can not allocate learnable weights to different nodes in the neighborhood and lack of robustness on account of neglecting both node attributes and graph reconstruction. Moreover, most of multi-view GNNs mainly focus on the case of multiple graphs, while designing GNNs for solving graph-structured data of multi-view attributes is still under-explored. In this paper, we propose a novel Multi-View Attribute Graph Convolution Networks (MAGCN) model for the clustering task. MAGCN is designed with two-pathway encoders that map graph embedding features and learn the view-consistency information. Specifically, the first pathway develops multi-view attribute graph attention networks to reduce the noise/redundancy and learn the graph embedding features for each multi-view graph data. The second pathway develops consistent embedding encoders to capture the geometric relationship and probability distribution consistency among different views, which adaptively finds a consistent clustering embedding space for multi-view attributes. Experiments on three benchmark graph datasets show the superiority of our method compared with several state-of-the-art algorithms.

Download Full-text

Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33014602 ◽

2019 ◽

Vol 33 ◽

pp. 4602-4609 ◽

Cited By ~ 36

Author(s):

Christopher Morris ◽

Martin Ritzert ◽

Matthias Fey ◽

William L. Hamilton ◽

Jan Eric Lenssen ◽

...

Keyword(s):

Neural Networks ◽

Multiple Scales ◽

Graph Isomorphism ◽

Higher Order ◽

Point Of View ◽

Theoretical Point ◽

Graph Classification ◽

Graph Neural Networks ◽

Vector Representations

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically—showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the 1-dimensional Weisfeiler-Leman graph isomorphism heuristic (1-WL). We show that GNNs have the same expressiveness as the 1-WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called k-dimensional GNNs (k-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.

Download Full-text

Graph Deep Learning for Long Range Forecasting

10.5194/egusphere-egu21-9141 ◽

2021 ◽

Author(s):

Salva Rühling Cachay ◽

Emma Erickson ◽

Arthur Fender C. Bucker ◽

Ernest Pokropek ◽

Willa Potosnak ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Long Range ◽

Large Scale ◽

State Of The Art ◽

Southern Oscillation ◽

Predictive Skill ◽

Anomaly Pattern ◽

Previous State ◽

Graph Neural Networks

Deep learning-based models have been recently shown to be competitive with, or even outperform, state-of-the-art long range forecasting models, such as for projecting the El Ni&#241;o-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale dependencies, such as teleconnections, that are particularly important for long range projections. Hence, we propose to explicitly model large-scale dependencies with Graph Neural Networks (GNN) to enhance explainability and improve the predictive skill of long lead time forecasts.In preliminary experiments focusing on ENSO, our GNN model outperforms previous state-of-the-art machine learning based systems for forecasts up to 6 months ahead. The explicit modeling of information flow via edges makes our model more explainable, and it is indeed shown to learn a sensible graph structure from scratch that correlates with the ENSO anomaly pattern for a given number of lead months.&#160;

Download Full-text

The impact of high-order interactions on the rate of synchronous discharge and information transmission in somatosensory cortex

Philosophical Transactions of The Royal Society A Mathematical Physical and Engineering Sciences ◽

10.1098/rsta.2009.0082 ◽

2009 ◽

Vol 367 (1901) ◽

pp. 3297-3310 ◽

Cited By ~ 57

Author(s):

Fernando Montani ◽

Robin A. A. Ince ◽

Riccardo Senatore ◽

Ehsan Arabzadeh ◽

Mathew E. Diamond ◽

...

Keyword(s):

Neural Networks ◽

Somatosensory Cortex ◽

Local Population ◽

Interaction Model ◽

Higher Order ◽

High Order ◽

Neural Population ◽

Somatosensory Information ◽

Representational Capacity ◽

The Impact

Understanding the operations of neural networks in the brain requires an understanding of whether interactions among neurons can be described by a pairwise interaction model, or whether a higher order interaction model is needed. In this article we consider the rate of synchronous discharge of a local population of neurons, a macroscopic index of the activation of the neural network that can be measured experimentally. We analyse a model based on physics’ maximum entropy principle that evaluates whether the probability of synchronous discharge can be described by interactions up to any given order. When compared with real neural population activity obtained from the rat somatosensory cortex, the model shows that interactions of at least order three or four are necessary to explain the data. We use Shannon information to compute the impact of high-order correlations on the amount of somatosensory information transmitted by the rate of synchronous discharge, and we find that correlations of higher order progressively decrease the information available through the neural population. These results are compatible with the hypothesis that high-order interactions play a role in shaping the dynamics of neural networks, and that they should be taken into account when computing the representational capacity of neural populations.

Download Full-text

Message Passing Attention Networks for Document Understanding

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6376 ◽

2020 ◽

Vol 34 (05) ◽

pp. 8544-8551 ◽

Cited By ~ 2

Author(s):

Giannis Nikolentzos ◽

Antoine Tixier ◽

Michalis Vazirgiannis

Keyword(s):

Neural Networks ◽

Text Classification ◽

Message Passing ◽

State Of The Art ◽

Structured Data ◽

Attention Networks ◽

Document Understanding ◽

Standard Text ◽

Graph Neural Networks ◽

The Impact

Graph neural networks have recently emerged as a very effective framework for processing graph-structured data. These models have achieved state-of-the-art performance in many tasks. Most graph neural networks can be described in terms of message passing, vertex update, and readout functions. In this paper, we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). We also propose several hierarchical variants of MPAD. Experiments conducted on 10 standard text classification datasets show that our architectures are competitive with the state-of-the-art. Ablation studies reveal further insights about the impact of the different components on performance. Code is publicly available at: https://github.com/giannisnik/mpad.

Download Full-text

Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks

IEEE Transactions on Knowledge and Data Engineering ◽

10.1109/tkde.2021.3074654 ◽

2021 ◽

pp. 1-1

Author(s):

Jianxin Li ◽

Hao Peng ◽

Yuwei Cao ◽

Yingtong Dou ◽

Hekai Zhang ◽

...

Keyword(s):

Neural Networks ◽

Higher Order ◽

Graph Neural Networks

Download Full-text

CellVGAE: An unsupervised scRNA-seq analysis workflow with graph attention networks

10.1101/2020.12.20.423645 ◽

2020 ◽

Author(s):

David Buterez ◽

Ioana Bica ◽

Ifrah Tariq ◽

Helena Andrés-Terré ◽

Pietro Liò

Keyword(s):

Neural Networks ◽

High Resolution ◽

Dimensionality Reduction ◽

Single Cell ◽

Rna Sequencing ◽

Feedforward Neural Networks ◽

Attention Networks ◽

Single Cell Rna Sequencing ◽

Analysis Workflow ◽

Graph Neural Networks

AbstractCurrently, single-cell RNA sequencing (scRNA-seq) allows high-resolution views of individual cells, for libraries of up to (tens of) thousands of samples. In this study, we introduce the use of graph neural networks (GNN) in the unsupervised study of scRNA-seq data, namely for dimensionality reduction and clustering. Motivated by the success of non-neural graph-based techniques in bioinformatics, as well as the now common feedforward neural networks being applied to scRNA-seq measurements, we develop an architecture based on a variational graph autoencoder with graph attention layers that works directly on the connectivity of cells. With the help of three case studies, we show that our model, named CellVGAE, can be effectively used for exploratory analysis, even on challenging datasets, by extracting meaningful features from the data and providing the means to visualise and interpret different aspects of the model. Furthermore, we evaluate the dimensionality reduction and clustering performance on 9 well-annotated datasets, where we compare with leading neural and non-neural techniques. CellVGAE outperforms competing methods in all 9 scenarios. Finally, we show that CellVGAE is more interpretable than existing architectures by analysing the graph attention coefficients. The software and code to generate all the figures are available at https://github.com/davidbuterez/CellVGAE.

Download Full-text