scholarly journals Exploring the Value of Nodes with Multicommunity Membership for Classification with Graph Convolutional Neural Networks

Information ◽  
2021 ◽  
Vol 12 (4) ◽  
pp. 170
Author(s):  
Michael Hopwood ◽  
Phuong Pho ◽  
Alexander V. Mantzaris
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Network Topology ◽  
Sampling Method ◽  
Sampling Methods ◽  
Weak Ties ◽  
Multiple Sampling ◽  
Strength Of Weak Ties ◽  
Network Topologies ◽  
Graph Neural Networks

Sampling is an important step in the machine learning process because it prioritizes samples that help the model best summarize the important concepts required for the task at hand. The process of determining the best sampling method has been rarely studied in the context of graph neural networks. In this paper, we evaluate multiple sampling methods (i.e., ascending and descending) that sample based off different definitions of centrality (i.e., Voterank, Pagerank, degree) to observe its relation with network topology. We find that no sampling method is superior across all network topologies. Additionally, we find situations where ascending sampling provides better classification scores, showing the strength of weak ties. Two strategies are then created to predict the best sampling method, one that observes the homogeneous connectivity of the nodes, and one that observes the network topology. In both methods, we are able to evaluate the best sampling direction consistently.

scholarly journals Benchmarking Graph Neural Networks for Materials Chemistry

2021 ◽  
Author(s):  
Victor Fung ◽  
Jiaxin Zhang ◽  
Eric Juarez ◽  
Bobby Sumpter
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Materials Chemistry ◽  
Learning Models ◽  
High Data ◽  
Hyperparameter Selection ◽  
Computational Materials ◽  
Conventional Models ◽  
Graph Neural Networks ◽  
Machine Learning Models

Graph neural networks (GNNs) have received intense interest as a rapidly expanding class of machine learning models remarkably well-suited for materials applications. To date, a number of successful GNNs have been proposed and demonstrated for systems ranging from crystal stability to electronic property prediction and to surface chemistry and heterogeneous catalysis. However, a consistent benchmark of these models remains lacking, hindering the development and consistent evaluation of new models in the materials field. Here, we present a workflow and testing platform, MatDeepLearn, for quickly and reproducibly assessing and comparing GNNs and other machine learning models. We use this platform to optimize and evaluate a selection of top performing GNNs on several representative datasets in computational materials chemistry. From our investigations we note the importance of hyperparameter selection and find roughly similar performances for the top models once optimized. We identify several strengths in GNNs over conventional models in cases with compositionally diverse datasets and in its overall flexibility with respect to inputs, due to learned rather than defined representations. Meanwhile several weaknesses of GNNs are also observed including high data requirements, and suggestions for further improvement for applications in materials chemistry are proposed.

scholarly journals Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

2021 ◽  
Vol 4 ◽  
Author(s):  
Paul Y. Wang ◽  
Sandalika Sapra ◽  
Vivek Kurien George ◽  
Gabriel A. Silva
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Neural Systems ◽  
Graph Structure ◽  
Imaging Data ◽  
C Elegans ◽  
State Classification ◽  
Graph Neural Networks ◽  
Emergent Behaviors

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

scholarly journals Heterogeneous Network Representation Learning

2020 ◽  
Author(s):  
Yuxiao Dong ◽  
Ziniu Hu ◽  
Kuansan Wang ◽  
Yizhou Sun ◽  
Jie Tang
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Heterogeneous Network ◽  
Representation Learning ◽  
Machine Learning Algorithms ◽  
Future Directions ◽  
Relational Properties ◽  
Open Research ◽  
Different Types ◽  
Graph Neural Networks

Representation learning has offered a revolutionary learning paradigm for various AI domains. In this survey, we examine and review the problem of representation learning with the focus on heterogeneous networks, which consists of different types of vertices and relations. The goal of this problem is to automatically project objects, most commonly, vertices, in an input heterogeneous network into a latent embedding space such that both the structural and relational properties of the network can be encoded and preserved. The embeddings (representations) can be then used as the features to machine learning algorithms for addressing corresponding network tasks. To learn expressive embeddings, current research developments can fall into two major categories: shallow embedding learning and graph neural networks. After a thorough review of the existing literature, we identify several critical challenges that remain unaddressed and discuss future directions. Finally, we build the Heterogeneous Graph Benchmark to facilitate open research for this rapidly-developing topic.

scholarly journals Combinatorial Optimization and Reasoning with Graph Neural Networks

2021 ◽  
Author(s):  
Quentin Cappart ◽  
Didier Chételat ◽  
Elias B. Khalil ◽  
Andrea Lodi ◽  
Christopher Morris ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Combinatorial Optimization ◽  
Computer Science ◽  
Operations Research ◽  
Building Block ◽  
Related Data ◽  
Problem Instances ◽  
Graph Neural Networks

Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have mostly focused on solving problem instances in isolation, ignoring the fact that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning, especially graph neural networks, as a key building block for combinatorial tasks, either directly as solvers or by enhancing the former. This paper presents a conceptual review of recent key advancements in this emerging field, aiming at researchers in both optimization and machine learning.

scholarly journals DeepEP: a deep learning framework for identifying essential proteins

2019 ◽  
Vol 20 (S16) ◽  
Author(s):  
Min Zeng ◽  
Min Li ◽  
Fang-Xiang Wu ◽  
Yaohang Li ◽  
Yi Pan
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Sampling Method ◽  
Ppi Network ◽  
Essential Proteins ◽  
Learning Framework ◽  
Multi Scale ◽  
Topological Features

Abstract Background Essential proteins are crucial for cellular life and thus, identification of essential proteins is an important topic and a challenging problem for researchers. Recently lots of computational approaches have been proposed to handle this problem. However, traditional centrality methods cannot fully represent the topological features of biological networks. In addition, identifying essential proteins is an imbalanced learning problem; but few current shallow machine learning-based methods are designed to handle the imbalanced characteristics. Results We develop DeepEP based on a deep learning framework that uses the node2vec technique, multi-scale convolutional neural networks and a sampling technique to identify essential proteins. In DeepEP, the node2vec technique is applied to automatically learn topological and semantic features for each protein in protein-protein interaction (PPI) network. Gene expression profiles are treated as images and multi-scale convolutional neural networks are applied to extract their patterns. In addition, DeepEP uses a sampling method to alleviate the imbalanced characteristics. The sampling method samples the same number of the majority and minority samples in a training epoch, which is not biased to any class in training process. The experimental results show that DeepEP outperforms traditional centrality methods. Moreover, DeepEP is better than shallow machine learning-based methods. Detailed analyses show that the dense vectors which are generated by node2vec technique contribute a lot to the improved performance. It is clear that the node2vec technique effectively captures the topological and semantic properties of PPI network. The sampling method also improves the performance of identifying essential proteins. Conclusion We demonstrate that DeepEP improves the prediction performance by integrating multiple deep learning techniques and a sampling method. DeepEP is more effective than existing methods.

scholarly journals Benchmarking graph neural networks for materials chemistry

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Victor Fung ◽  
Jiaxin Zhang ◽  
Eric Juarez ◽  
Bobby G. Sumpter
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Materials Chemistry ◽  
Learning Models ◽  
High Data ◽  
Hyperparameter Selection ◽  
Computational Materials ◽  
Conventional Models ◽  
Graph Neural Networks ◽  
Machine Learning Models

AbstractGraph neural networks (GNNs) have received intense interest as a rapidly expanding class of machine learning models remarkably well-suited for materials applications. To date, a number of successful GNNs have been proposed and demonstrated for systems ranging from crystal stability to electronic property prediction and to surface chemistry and heterogeneous catalysis. However, a consistent benchmark of these models remains lacking, hindering the development and consistent evaluation of new models in the materials field. Here, we present a workflow and testing platform, MatDeepLearn, for quickly and reproducibly assessing and comparing GNNs and other machine learning models. We use this platform to optimize and evaluate a selection of top performing GNNs on several representative datasets in computational materials chemistry. From our investigations we note the importance of hyperparameter selection and find roughly similar performances for the top models once optimized. We identify several strengths in GNNs over conventional models in cases with compositionally diverse datasets and in its overall flexibility with respect to inputs, due to learned rather than defined representations. Meanwhile several weaknesses of GNNs are also observed including high data requirements, and suggestions for further improvement for applications in materials chemistry are discussed.

scholarly journals Machine Learning Meets the Semantic Web

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Konstantinos Ilias Kotis ◽  
Konstantina Zachila ◽  
Evaggelos Paparidis
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Semantic Web ◽  
Graph Representation ◽  
Order Relations ◽  
Knowledge Graphs ◽  
Graph Neural Networks ◽  
Different Levels ◽  
Remarkable Progress ◽  
Open Issues

Remarkable progress in research has shown the efficiency of Knowledge Graphs (KGs) in extracting valuable external knowledge in various domains. A Knowledge Graph (KG) can illustrate high-order relations that connect two objects with one or multiple related attributes. The emerging Graph Neural Networks (GNN) can extract both object characteristics and relations from KGs. This paper presents how Machine Learning (ML) meets the Semantic Web and how KGs are related to Neural Networks and Deep Learning. The paper also highlights important aspects of this area of research, discussing open issues such as the bias hidden in KGs at different levels of graph representation.

scholarly journals Benchmarking Graph Neural Networks for Materials Chemistry

2021 ◽  
Author(s):  
Victor Fung ◽  
Jiaxin Zhang ◽  
Eric Juarez ◽  
Bobby Sumpter
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Materials Chemistry ◽  
Learning Models ◽  
High Data ◽  
Hyperparameter Selection ◽  
Computational Materials ◽  
Conventional Models ◽  
Graph Neural Networks ◽  
Machine Learning Models

Graph neural networks (GNNs) have received intense interest as a rapidly expanding class of machine learning models remarkably well-suited for materials applications. To date, a number of successful GNNs have been proposed and demonstrated for systems ranging from crystal stability to electronic property prediction and to surface chemistry and heterogeneous catalysis. However, a consistent benchmark of these models remains lacking, hindering the development and consistent evaluation of new models in the materials field. Here, we present a workflow and testing platform, MatDeepLearn, for quickly and reproducibly assessing and comparing GNNs and other machine learning models. We use this platform to optimize and evaluate a selection of top performing GNNs on several representative datasets in computational materials chemistry. From our investigations we note the importance of hyperparameter selection and find roughly similar performances for the top models once optimized. We identify several strengths in GNNs over conventional models in cases with compositionally diverse datasets and in its overall flexibility with respect to inputs, due to learned rather than defined representations. Meanwhile several weaknesses of GNNs are also observed including high data requirements, and suggestions for further improvement for applications in materials chemistry are proposed.

Sign in / Sign up

Export Citation Format

Share Document