scholarly journals Model agnostic generation of counterfactual explanations for molecules

2021 ◽  
Author(s):  
Geemi P Wellawatte ◽  
Aditi Seshadri ◽  
Andrew D White
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Model Prediction ◽  
Chemical Structure ◽  
Specific Model ◽  
Black Box ◽  
Major Barrier ◽  
Forest Models ◽  
Random Forest Models ◽  
Graph Neural Networks

An outstanding challenge in deep learning in chemistry is its lack of interpretability. The inability of explaining why a neural network makes a prediction is a major barrier to deployment of AI models. This not only dissuades chemists from using deep learning predictions, but also has led to neural networks learning spurious correlations that are difficult to notice. Counterfactuals are a category of explanations that provide a rationale behind a model prediction with satisfying properties like providing chemical structure insights. Yet, counterfactuals are have been previously limited to specific model architectures or required reinforcement learning as a separate process. In this work, we show a universal model-agnostic approach that can explain any black-box model prediction. We demonstrate this method on random forest models, sequence models, and graph neural networks in both classification and regression.

scholarly journals Deep Learning Strategies for ProtoDUNE Raw Data Denoising

2022 ◽  
Vol 6 (1) ◽  
Author(s):  
Marco Rossi ◽  
Sofia Vallecorsa
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Learning Strategies ◽  
Simulation Data ◽  
Raw Data ◽  
Digital Detector ◽  
Speed Up ◽  
Neural Network Hardware ◽  
Graph Neural Networks ◽  
High Level

AbstractIn this work, we investigate different machine learning-based strategies for denoising raw simulation data from the ProtoDUNE experiment. The ProtoDUNE detector is hosted by CERN and it aims to test and calibrate the technologies for DUNE, a forthcoming experiment in neutrino physics. The reconstruction workchain consists of converting digital detector signals into physical high-level quantities. We address the first step in reconstruction, namely raw data denoising, leveraging deep learning algorithms. We design two architectures based on graph neural networks, aiming to enhance the receptive field of basic convolutional neural networks. We benchmark this approach against traditional algorithms implemented by the DUNE collaboration. We test the capabilities of graph neural network hardware accelerator setups to speed up training and inference processes.

scholarly journals Online Planner Selection with Graph Neural Networks and Adaptive Scheduling

2020 ◽  
Vol 34 (04) ◽  
pp. 5077-5084
Author(s):  
Tengfei Ma ◽  
Patrick Ferber ◽  
Siyu Huo ◽  
Jie Chen ◽  
Michael Katz
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Automated Planning ◽  
Adaptive Scheduling ◽  
Optimal Planning ◽  
Graph Representations ◽  
Structural Graph ◽  
Scheduling Method ◽  
Graph Neural Networks ◽  
Node Labels

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference.Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based.The code is available at https://github.com/matenure/GNN_planner.

scholarly journals Biological network analysis with deep learning

2020 ◽  
Author(s):  
Giulia Muzio ◽  
Leslie O’Bray ◽  
Karsten Borgwardt
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Biological Networks ◽  
Protein Function ◽  
Regulatory Networks ◽  
Biological Network ◽  
Gene Interaction ◽  
Disease Diagnosis ◽  
Interaction Prediction ◽  
Graph Neural Networks

Abstract Recent advancements in experimental high-throughput technologies have expanded the availability and quantity of molecular data in biology. Given the importance of interactions in biological processes, such as the interactions between proteins or the bonds within a chemical compound, this data is often represented in the form of a biological network. The rise of this data has created a need for new computational tools to analyze networks. One major trend in the field is to use deep learning for this goal and, more specifically, to use methods that work with networks, the so-called graph neural networks (GNNs). In this article, we describe biological networks and review the principles and underlying algorithms of GNNs. We then discuss domains in bioinformatics in which graph neural networks are frequently being applied at the moment, such as protein function prediction, protein–protein interaction prediction and in silico drug discovery and development. Finally, we highlight application areas such as gene regulatory networks and disease diagnosis where deep learning is emerging as a new tool to answer classic questions like gene interaction prediction and automatic disease prediction from data.

scholarly journals Deep learning in bioinformatics: introduction, application, and perspective in big data era

2019 ◽  
Author(s):  
Yu Li ◽  
Chao Huang ◽  
Lizhong Ding ◽  
Zhongxiao Li ◽  
Yijie Pan ◽  
...  
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Deep Learning ◽  
Generative Adversarial Networks ◽  
Great Success ◽  
Research Directions ◽  
Adversarial Networks ◽  
Variational Autoencoder ◽  
The Common ◽  
Graph Neural Networks

AbstractDeep learning, which is especially formidable in handling big data, has achieved great success in various fields, including bioinformatics. With the advances of the big data era in biology, it is foreseeable that deep learning will become increasingly important in the field and will be incorporated in vast majorities of analysis pipelines. In this review, we provide both the exoteric introduction of deep learning, and concrete examples and implementations of its representative applications in bioinformatics. We start from the recent achievements of deep learning in the bioinformatics field, pointing out the problems which are suitable to use deep learning. After that, we introduce deep learning in an easy-to-understand fashion, from shallow neural networks to legendary convolutional neural networks, legendary recurrent neural networks, graph neural networks, generative adversarial networks, variational autoencoder, and the most recent state-of-the-art architectures. After that, we provide eight examples, covering five bioinformatics research directions and all the four kinds of data type, with the implementation written in Tensorflow and Keras. Finally, we discuss the common issues, such as overfitting and interpretability, that users will encounter when adopting deep learning methods and provide corresponding suggestions. The implementations are freely available at https://github.com/lykaust15/Deep_learning_examples.

scholarly journals Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

2021 ◽  
Vol 4 ◽  
Author(s):  
Paul Y. Wang ◽  
Sandalika Sapra ◽  
Vivek Kurien George ◽  
Gabriel A. Silva
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Neural Systems ◽  
Graph Structure ◽  
Imaging Data ◽  
C Elegans ◽  
State Classification ◽  
Graph Neural Networks ◽  
Emergent Behaviors

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

Sanitizing hidden activations for improving adversarial robustness of convolutional neural networks

2021 ◽  
pp. 1-11
Author(s):  
Tianshi Mu ◽  
Kequan Lin ◽  
Huabing Zhang ◽  
Jian Wang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Black Box ◽  
Experimental Results ◽  
Amplification Effect ◽  
Wide Range ◽  
Adversarial Examples

Deep learning is gaining significant traction in a wide range of areas. Whereas, recent studies have demonstrated that deep learning exhibits the fatal weakness on adversarial examples. Due to the black-box nature and un-transparency problem of deep learning, it is difficult to explain the reason for the existence of adversarial examples and also hard to defend against them. This study focuses on improving the adversarial robustness of convolutional neural networks. We first explore how adversarial examples behave inside the network through visualization. We find that adversarial examples produce perturbations in hidden activations, which forms an amplification effect to fool the network. Motivated by this observation, we propose an approach, termed as sanitizing hidden activations, to help the network correctly recognize adversarial examples by eliminating or reducing the perturbations in hidden activations. To demonstrate the effectiveness of our approach, we conduct experiments on three widely used datasets: MNIST, CIFAR-10 and ImageNet, and also compare with state-of-the-art defense techniques. The experimental results show that our sanitizing approach is more generalized to defend against different kinds of attacks and can effectively improve the adversarial robustness of convolutional neural networks.

scholarly journals Estimating latent positions of actors using Neural Networks in R with GCN4R

2020 ◽  
Author(s):  
Joshua Levy ◽  
Carly Bobak ◽  
Brock Christensen ◽  
Louis Vaickus ◽  
James O’Malley
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Learning Community ◽  
Biological Networks ◽  
Peer Effects ◽  
Prediction Models ◽  
Research Community ◽  
Social Network Data ◽  
Covariate Information ◽  
Graph Neural Networks

AbstractNetwork analysis methods are useful to better understand and contextualize relationships between entities. While statistical and machine learning prediction models generally assume independence between actors, network-based statistical methods for social network data allow for dyadic dependence between actors. While numerous methods have been developed for the R statistical software to analyze such data, deep learning methods have not been implemented in this language. Here, we introduce GCN4R, an R library for fitting graph neural networks on independent networks to aggregate actor covariate information to yield meaningful embeddings for a variety of network-based tasks (e.g. community detection, peer effects models, social influence). We provide an extensive overview of insights and methods utilized by the deep learning community on learning on social and biological networks, followed by a tutorial that demonstrates some of the capabilities of the GCN4R framework to make these methods more accessible to the R research community.

scholarly journals Winsorization for Robust Bayesian Neural Networks

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1546
Author(s):  
Somya Sharma ◽  
Snigdhansu Chatterjee
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Black Box ◽  
Machine Learning Techniques ◽  
Learning Techniques ◽  
Versatile Technique ◽  
Aberrant Observations ◽  
Probabilistic Machine Learning ◽  
Different Levels

With the advent of big data and the popularity of black-box deep learning methods, it is imperative to address the robustness of neural networks to noise and outliers. We propose the use of Winsorization to recover model performances when the data may have outliers and other aberrant observations. We provide a comparative analysis of several probabilistic artificial intelligence and machine learning techniques for supervised learning case studies. Broadly, Winsorization is a versatile technique for accounting for outliers in data. However, different probabilistic machine learning techniques have different levels of efficiency when used on outlier-prone data, with or without Winsorization. We notice that Gaussian processes are extremely vulnerable to outliers, while deep learning techniques in general are more robust.

scholarly journals How Convolutional Neural Networks Diagnose Plant Disease

2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Yosuke Toda ◽  
Fumio Okura
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Plant Disease ◽  
Disease Diagnosis ◽  
Black Box ◽  
Plant Diseases ◽  
Training Dataset ◽  
Great Success ◽  
Human Decision

Deep learning with convolutional neural networks (CNNs) has achieved great success in the classification of various plant diseases. However, a limited number of studies have elucidated the process of inference, leaving it as an untouchable black box. Revealing the CNN to extract the learned feature as an interpretable form not only ensures its reliability but also enables the validation of the model authenticity and the training dataset by human intervention. In this study, a variety of neuron-wise and layer-wise visualization methods were applied using a CNN, trained with a publicly available plant disease image dataset. We showed that neural networks can capture the colors and textures of lesions specific to respective diseases upon diagnosis, which resembles human decision-making. While several visualization methods were used as they are, others had to be optimized to target a specific layer that fully captures the features to generate consequential outputs. Moreover, by interpreting the generated attention maps, we identified several layers that were not contributing to inference and removed such layers inside the network, decreasing the number of parameters by 75% without affecting the classification accuracy. The results provide an impetus for the CNN black box users in the field of plant science to better understand the diagnosis process and lead to further efficient use of deep learning for plant disease diagnosis.

scholarly journals A Practical Tutorial on Graph Neural Networks

2022 ◽  
Author(s):  
Isaac Ronald Ward ◽  
Jack Joyner ◽  
Casey Lickfold ◽  
Yulan Guo ◽  
Mohammed Bennamoun
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Learning ◽  
Input Data ◽  
Unstructured Data ◽  
Data Types ◽  
Learning Techniques ◽  
Graph Neural Networks ◽  
Unique Ability

Graph neural networks (GNNs) have recently grown in popularity in the field of artificial intelligence (AI) due to their unique ability to ingest relatively unstructured data types as input data. Although some elements of the GNN architecture are conceptually similar in operation to traditional neural networks (and neural network variants), other elements represent a departure from traditional deep learning techniques. This tutorial exposes the power and novelty of GNNs to AI practitioners by collating and presenting details regarding the motivations, concepts, mathematics, and applications of the most common and performant variants of GNNs. Importantly, we present this tutorial concisely, alongside practical examples, thus providing a practical and accessible tutorial on the topic of GNNs.

Sign in / Sign up

Export Citation Format

Share Document