Online Planner Selection with Graph Neural Networks and Adaptive Scheduling

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference.Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based.The code is available at https://github.com/matenure/GNN_planner.

Download Full-text

SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

ACM Transactions on Graphics ◽

10.1145/3450284 ◽

2021 ◽

Vol 40 (3) ◽

pp. 1-13

Author(s):

Lumin Yang ◽

Jiajie Zhuang ◽

Hongbo Fu ◽

Xiangzhi Wei ◽

Kun Zhou ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Large Scale ◽

State Of The Art ◽

Semantic Segmentation ◽

Structure Information ◽

Graph Neural Networks ◽

Node Labels ◽

Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

Download Full-text

Deep Learning Strategies for ProtoDUNE Raw Data Denoising

Computing and Software for Big Science ◽

10.1007/s41781-021-00077-9 ◽

2022 ◽

Vol 6 (1) ◽

Author(s):

Marco Rossi ◽

Sofia Vallecorsa

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Learning Strategies ◽

Simulation Data ◽

Raw Data ◽

Digital Detector ◽

Speed Up ◽

Neural Network Hardware ◽

Graph Neural Networks ◽

High Level

AbstractIn this work, we investigate different machine learning-based strategies for denoising raw simulation data from the ProtoDUNE experiment. The ProtoDUNE detector is hosted by CERN and it aims to test and calibrate the technologies for DUNE, a forthcoming experiment in neutrino physics. The reconstruction workchain consists of converting digital detector signals into physical high-level quantities. We address the first step in reconstruction, namely raw data denoising, leveraging deep learning algorithms. We design two architectures based on graph neural networks, aiming to enhance the receptive field of basic convolutional neural networks. We benchmark this approach against traditional algorithms implemented by the DUNE collaboration. We test the capabilities of graph neural network hardware accelerator setups to speed up training and inference processes.

Download Full-text

Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

10.5753/eniac.2020.12161 ◽

2020 ◽

Author(s):

Douglas Meneghetti ◽

Reinaldo Bianchi

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Communication Channels ◽

Neural Network Architecture ◽

Graph Representations ◽

Labeled Graph ◽

Multiple Agent ◽

Multi Agent ◽

Graph Neural Networks

This work proposes a neural network architecture that learns policies for multiple agent classes in a heterogeneous multi-agent reinforcement setting. The proposed network uses directed labeled graph representations for states, encodes feature vectors of different sizes for different entity classes, uses relational graph convolution layers to model different communication channels between entity types and learns distinct policies for different agent classes, sharing parameters wherever possible. Results have shown that specializing the communication channels between entity classes is a promising step to achieve higher performance in environments composed of heterogeneous entities.

Download Full-text

Biological network analysis with deep learning

Briefings in Bioinformatics ◽

10.1093/bib/bbaa257 ◽

2020 ◽

Author(s):

Giulia Muzio ◽

Leslie O’Bray ◽

Karsten Borgwardt

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Biological Networks ◽

Protein Function ◽

Regulatory Networks ◽

Biological Network ◽

Gene Interaction ◽

Disease Diagnosis ◽

Interaction Prediction ◽

Graph Neural Networks

Abstract Recent advancements in experimental high-throughput technologies have expanded the availability and quantity of molecular data in biology. Given the importance of interactions in biological processes, such as the interactions between proteins or the bonds within a chemical compound, this data is often represented in the form of a biological network. The rise of this data has created a need for new computational tools to analyze networks. One major trend in the field is to use deep learning for this goal and, more specifically, to use methods that work with networks, the so-called graph neural networks (GNNs). In this article, we describe biological networks and review the principles and underlying algorithms of GNNs. We then discuss domains in bioinformatics in which graph neural networks are frequently being applied at the moment, such as protein function prediction, protein–protein interaction prediction and in silico drug discovery and development. Finally, we highlight application areas such as gene regulatory networks and disease diagnosis where deep learning is emerging as a new tool to answer classic questions like gene interaction prediction and automatic disease prediction from data.

Download Full-text

Deep learning in bioinformatics: introduction, application, and perspective in big data era

10.1101/563601 ◽

2019 ◽

Cited By ~ 9

Author(s):

Yu Li ◽

Chao Huang ◽

Lizhong Ding ◽

Zhongxiao Li ◽

Yijie Pan ◽

...

Keyword(s):

Neural Networks ◽

Big Data ◽

Deep Learning ◽

Generative Adversarial Networks ◽

Great Success ◽

Research Directions ◽

Adversarial Networks ◽

Variational Autoencoder ◽

The Common ◽

Graph Neural Networks

AbstractDeep learning, which is especially formidable in handling big data, has achieved great success in various fields, including bioinformatics. With the advances of the big data era in biology, it is foreseeable that deep learning will become increasingly important in the field and will be incorporated in vast majorities of analysis pipelines. In this review, we provide both the exoteric introduction of deep learning, and concrete examples and implementations of its representative applications in bioinformatics. We start from the recent achievements of deep learning in the bioinformatics field, pointing out the problems which are suitable to use deep learning. After that, we introduce deep learning in an easy-to-understand fashion, from shallow neural networks to legendary convolutional neural networks, legendary recurrent neural networks, graph neural networks, generative adversarial networks, variational autoencoder, and the most recent state-of-the-art architectures. After that, we provide eight examples, covering five bioinformatics research directions and all the four kinds of data type, with the implementation written in Tensorflow and Keras. Finally, we discuss the common issues, such as overfitting and interpretability, that users will encounter when adopting deep learning methods and provide corresponding suggestions. The implementations are freely available at https://github.com/lykaust15/Deep_learning_examples.

Download Full-text

The interplay between communities and homophily in semi-supervised classification using graph neural networks

Applied Network Science ◽

10.1007/s41109-021-00423-1 ◽

2021 ◽

Vol 6 (1) ◽

Author(s):

Hussain Hussain ◽

Tomislav Duricic ◽

Elisabeth Lex ◽

Denis Helic ◽

Roman Kern

Keyword(s):

Neural Networks ◽

Community Structure ◽

Model Selection ◽

Graph Structure ◽

Information Theoretic ◽

Node Classification ◽

Selection For ◽

Graph Neural Networks ◽

Node Labels ◽

The Impact

AbstractGraph Neural Networks (GNNs) are effective in many applications. Still, there is a limited understanding of the effect of common graph structures on the learning process of GNNs. To fill this gap, we study the impact of community structure and homophily on the performance of GNNs in semi-supervised node classification on graphs. Our methodology consists of systematically manipulating the structure of eight datasets, and measuring the performance of GNNs on the original graphs and the change in performance in the presence and the absence of community structure and/or homophily. Our results show the major impact of both homophily and communities on the classification accuracy of GNNs, and provide insights on their interplay. In particular, by analyzing community structure and its correlation with node labels, we are able to make informed predictions on the suitability of GNNs for classification on a given graph. Using an information-theoretic metric for community-label correlation, we devise a guideline for model selection based on graph structure. With our work, we provide insights on the abilities of GNNs and the impact of common network phenomena on their performance. Our work improves model selection for node classification in semi-supervised settings.

Download Full-text

Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

Frontiers in Artificial Intelligence ◽

10.3389/frai.2021.618372 ◽

2021 ◽

Vol 4 ◽

Author(s):

Paul Y. Wang ◽

Sandalika Sapra ◽

Vivek Kurien George ◽

Gabriel A. Silva

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Deep Learning ◽

Neural Systems ◽

Graph Structure ◽

Imaging Data ◽

C Elegans ◽

State Classification ◽

Graph Neural Networks ◽

Emergent Behaviors

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

Download Full-text

Multi-Channel Pooling Graph Neural Networks

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/199 ◽

2021 ◽

Author(s):

Jinlong Du ◽

Senzhang Wang ◽

Hao Miao ◽

Jiaqiang Zhang

Keyword(s):

Neural Networks ◽

Local Structure ◽

Superior Performance ◽

Topology Structure ◽

Graph Representations ◽

Convolution Operation ◽

Global Topology ◽

Benchmark Datasets ◽

Graph Neural Networks ◽

Local Topology

Graph pooling is a critical operation to downsample a graph in graph neural networks. Existing coarsening pooling methods (e.g. DiffPool) mostly focus on capturing the global topology structure by assigning the nodes into several coarse clusters, while dropping pooling methods (e.g. SAGPool) try to preserve the local topology structure by selecting the top-k representative nodes. However, there lacks an effective method to integrate the two types of methods so that both the local and the global topology structure of a graph can be well captured. To address this issue, we propose a Multi-channel Graph Pooling method named MuchPool, which captures the local structure, the global structure, and node feature simultaneously in graph pooling. Specifically, we use two channels to conduct dropping pooling based on the local topology and node features respectively, and one channel to conduct coarsening pooling. Then a cross-channel convolution operation is designed to refine the graph representations of different channels. Finally, the pooling results are aggregated as the final pooled graph. Extensive experiments on six benchmark datasets present the superior performance of MuchPool. The code of this work is publicly available at Github.

Download Full-text

GSSNN: Graph Smoothing Splines Neural Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6185 ◽

2020 ◽

Vol 34 (04) ◽

pp. 7007-7014

Author(s):

Shichao Zhu ◽

Lewei Zhou ◽

Shirui Pan ◽

Chuan Zhou ◽

Guiying Yan ◽

...

Keyword(s):

Neural Networks ◽

State Of The Art ◽

Representation Learning ◽

Smoothing Splines ◽

Graph Representation ◽

Graph Classification ◽

Topological Information ◽

Graph Representations ◽

The Arts ◽

Graph Neural Networks

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in many graph data analysis tasks. However, they still suffer from two limitations for graph representation learning. First, they exploit non-smoothing node features which may result in suboptimal embedding and degenerated performance for graph classification. Second, they only exploit neighbor information but ignore global topological knowledge. Aiming to overcome these limitations simultaneously, in this paper, we propose a novel, flexible, and end-to-end framework, Graph Smoothing Splines Neural Networks (GSSNN), for graph classification. By exploiting the smoothing splines, which are widely used to learn smoothing fitting function in regression, we develop an effective feature smoothing and enhancement module Scaled Smoothing Splines (S3) to learn graph embedding. To integrate global topological information, we design a novel scoring module, which exploits closeness, degree, as well as self-attention values, to select important node features as knots for smoothing splines. These knots can be potentially used for interpreting classification results. In extensive experiments on biological and social datasets, we demonstrate that our model achieves state-of-the-arts and GSSNN is superior in learning more robust graph representations. Furthermore, we show that S3 module is easily plugged into existing GNNs to improve their performance.

Download Full-text

Model agnostic generation of counterfactual explanations for molecules

10.33774/chemrxiv-2021-4qkg8-v2 ◽

2021 ◽

Author(s):

Geemi P Wellawatte ◽

Aditi Seshadri ◽

Andrew D White

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Model Prediction ◽

Chemical Structure ◽

Specific Model ◽

Black Box ◽

Major Barrier ◽

Forest Models ◽

Random Forest Models ◽

Graph Neural Networks

An outstanding challenge in deep learning in chemistry is its lack of interpretability. The inability of explaining why a neural network makes a prediction is a major barrier to deployment of AI models. This not only dissuades chemists from using deep learning predictions, but also has led to neural networks learning spurious correlations that are difficult to notice. Counterfactuals are a category of explanations that provide a rationale behind a model prediction with satisfying properties like providing chemical structure insights. Yet, counterfactuals are have been previously limited to specific model architectures or required reinforcement learning as a separate process. In this work, we show a universal model-agnostic approach that can explain any black-box model prediction. We demonstrate this method on random forest models, sequence models, and graph neural networks in both classification and regression.

Download Full-text