scholarly journals Imitation Learning with Graph Neural Networks for Improving Swarm Robustness under Restricted Communications

2021 ◽  
Vol 11 (19) ◽  
pp. 9055
Author(s):  
Ce Guo ◽  
Pengming Zhu ◽  
Zhiqian Zhou ◽  
Lin Lang ◽  
Zhiwen Zeng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Imitation Learning ◽  
Communication Constraints ◽  
Training Process ◽  
Simulation Experiments ◽  
Graph Data ◽  
Primary Motivation ◽  
Convolutional Networks ◽  
Speed Up ◽  
Graph Neural Networks

This paper focuses on generating distributed flocking strategies via imitation learning. The primary motivation is to improve the swarm robustness and achieve better consistency while respecting the communication constraints. This paper first proposes a quantitative metric of swarm robustness based on entropy evaluation. Then, the graph importance consistency is also proposed, which is one of the critical goals of the flocking task. Moreover, the importance-correlated directed graph convolutional networks (IDGCNs) are constructed for multidimensional feature extraction and structure-related aggregation of graph data. Next, by employing IDGCNs-based imitation learning, a distributed and scalable flocking strategy is obtained, and its performance is very close to the centralized strategy template while considering communication constraints. To speed up and simplify the training process, we train the flocking strategy with a small number of agents and set restrictions on communication. Finally, various simulation experiments are executed to verify the advantages of the obtained strategy in terms of realizing the swarm consistency and improving the swarm robustness. The results also show that the performance is well maintained while the scale of agents expands (tested with 20, 30, 40 robots).

scholarly journals Deep Learning Strategies for ProtoDUNE Raw Data Denoising

2022 ◽  
Vol 6 (1) ◽  
Author(s):  
Marco Rossi ◽  
Sofia Vallecorsa
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Learning Strategies ◽  
Simulation Data ◽  
Raw Data ◽  
Digital Detector ◽  
Speed Up ◽  
Neural Network Hardware ◽  
Graph Neural Networks ◽  
High Level

AbstractIn this work, we investigate different machine learning-based strategies for denoising raw simulation data from the ProtoDUNE experiment. The ProtoDUNE detector is hosted by CERN and it aims to test and calibrate the technologies for DUNE, a forthcoming experiment in neutrino physics. The reconstruction workchain consists of converting digital detector signals into physical high-level quantities. We address the first step in reconstruction, namely raw data denoising, leveraging deep learning algorithms. We design two architectures based on graph neural networks, aiming to enhance the receptive field of basic convolutional neural networks. We benchmark this approach against traditional algorithms implemented by the DUNE collaboration. We test the capabilities of graph neural network hardware accelerator setups to speed up training and inference processes.

scholarly journals Multi-View Attribute Graph Convolution Networks for Clustering

2020 ◽  
Author(s):  
Jiafeng Cheng ◽  
Qianqian Wang ◽  
Zhiqiang Tao ◽  
Deyan Xie ◽  
Quanxue Gao
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Graph Embedding ◽  
Structured Data ◽  
Attention Networks ◽  
Graph Data ◽  
Graph Reconstruction ◽  
Node Attributes ◽  
Graph Neural Networks ◽  
Geometric Relationship

Graph neural networks (GNNs) have made considerable achievements in processing graph-structured data. However, existing methods can not allocate learnable weights to different nodes in the neighborhood and lack of robustness on account of neglecting both node attributes and graph reconstruction. Moreover, most of multi-view GNNs mainly focus on the case of multiple graphs, while designing GNNs for solving graph-structured data of multi-view attributes is still under-explored. In this paper, we propose a novel Multi-View Attribute Graph Convolution Networks (MAGCN) model for the clustering task. MAGCN is designed with two-pathway encoders that map graph embedding features and learn the view-consistency information. Specifically, the first pathway develops multi-view attribute graph attention networks to reduce the noise/redundancy and learn the graph embedding features for each multi-view graph data. The second pathway develops consistent embedding encoders to capture the geometric relationship and probability distribution consistency among different views, which adaptively finds a consistent clustering embedding space for multi-view attributes. Experiments on three benchmark graph datasets show the superiority of our method compared with several state-of-the-art algorithms.

Optimasi Akurasi Metode Convolutional Neural Network untuk Identifikasi Jenis Sampah

2021 ◽  
Vol 5 (2) ◽  
pp. 312-318
Author(s):  
Rima Dias Ramadhani ◽  
Afandi Nur Aziz Thohari ◽  
Condro Kartiko ◽  
Apri Junaidi ◽  
Tri Ginanjar Laksana ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Computer Science ◽  
No Value ◽  
Convolutional Neural Networks ◽  
Training Process ◽  
Indonesian Government ◽  
Speed Up ◽  
Model Training

Waste is goods / materials that have no value in the scope of production, where in some cases the waste is disposed of carelessly and can damage the environment. The Indonesian government in 2019 recorded waste reaching 66-67 million tons, which is higher than the previous year, which was 64 million tons. Waste is differentiated based on its type, namely organic and anorganic waste. In the field of computer science, the process of sensing the type waste can be done using a camera and the Convolutional Neural Networks (CNN) method, which is a type of neural network that works by receiving input in the form of images. The input will be trained using CNN architecture so that it will produce output that can recognize the object being inputted. This study optimizes the use of the CNN method to obtain accurate results in identifying types of waste. Optimization is done by adding several hyperparameters to the CNN architecture. By adding hyperparameters, the accuracy value is 91.2%. Meanwhile, if the hyperparameter is not used, the accuracy value is only 67.6%. There are three hyperparameters used to increase the accuracy value of the model. They are dropout, padding, and stride. 20% increase in dropout to increase training overfit. Whereas padding and stride are used to speed up the model training process.

Convolutional Graph Neural Networks

2020 ◽  
pp. 107-123
Author(s):  
J. Joshua Thomas ◽  
Tran Huu Ngoc Tran ◽  
Gilberto Pérez Lechuga ◽  
Bahari Belaton
Keyword(s):  
Deep Learning ◽  
Drug Discovery ◽  
Research Area ◽  
Molecular Fingerprints ◽  
Graph Data ◽  
Target Interaction ◽  
Convolutional Networks ◽  
Variational Autoencoder ◽  
Recent Trends ◽  
Graph Neural Networks

Applying deep learning to the pervasive graph data is significant because of the unique characteristics of graphs. Recently, substantial amounts of research efforts have been keen on this area, greatly advancing graph-analyzing techniques. In this study, the authors comprehensively review different kinds of deep learning methods applied to graphs. They discuss with existing literature into sub-components of two: graph convolutional networks, graph autoencoders, and recent trends including chemoinformatics research area including molecular fingerprints and drug discovery. They further experiment with variational autoencoder (VAE) analyze how these apply in drug target interaction (DTI) and applications with ephemeral outline on how they assist the drug discovery pipeline and discuss potential research directions.

scholarly journals Kriging Convolutional Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 3187-3194
Author(s):  
Gabriel Appleby ◽  
Linfeng Liu ◽  
Li-Ping Liu
Keyword(s):  
Neural Networks ◽  
Spatial Interpolation ◽  
Recent Progress ◽  
Kriging Method ◽  
Convolutional Networks ◽  
Spatial Locality ◽  
Estimation Problems ◽  
Specific Configuration ◽  
Graph Neural Networks ◽  
Direct Use

Spatial interpolation is a class of estimation problems where locations with known values are used to estimate values at other locations, with an emphasis on harnessing spatial locality and trends. Traditional kriging methods have strong Gaussian assumptions, and as a result, often fail to capture complexities within the data. Inspired by the recent progress of graph neural networks, we introduce Kriging Convolutional Networks (KCN), a method of combining advantages of Graph Neural Networks (GNN) and kriging. Compared to standard GNNs, KCNs make direct use of neighboring observations when generating predictions. KCNs also contain the kriging method as a specific configuration. Empirically, we show that this model outperforms GNNs and kriging in several applications.

scholarly journals Rumor Detection Based on SAGNN: Simplified Aggregation Graph Neural Networks

2020 ◽  
Author(s):  
Liang Zhang ◽  
Jingqun Li ◽  
Bin Zhou ◽  
Yan Jia
Keyword(s):  
Neural Networks ◽  
Numerical Experiments ◽  
State Of The Art ◽  
Fake News ◽  
Wide Spread ◽  
Convolutional Networks ◽  
The Media ◽  
Graph Neural Networks ◽  
Rumor Detection ◽  
Simple Network

Identifying fake news on the media has been an important issue. This is especially true considering the wide spread of rumors on the popular social networks such as Twitter. Various kinds of techniques have been proposed to detect rumors. In this work, we study the application of graph neural networks for the task of rumor detection, and present a simplified new architecture to classify rumors. Numerical experiments show that the proposed simple network has comparable to or even better performance than state-of-the art graph convolutional networks, while having significantly reduced the computational complexity.

scholarly journals Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective

2019 ◽  
Author(s):  
Kaidi Xu ◽  
Hongge Chen ◽  
Sijia Liu ◽  
Pin-Yu Chen ◽  
Tsui-Wei Weng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Classification Performance ◽  
Original Graph ◽  
Graph Data ◽  
Gradient Based ◽  
Significant Performance ◽  
Adversarial Training ◽  
Graph Neural Networks ◽  
Attack And Defense ◽  
Discrete Graph

Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification. However, only few work has addressed the adversarial robustness of GNNs. In this paper, we first present a novel gradient-based attack method that facilitates the difficulty of tackling discrete graph data. When comparing to current adversarial attacks on GNNs, the results show that by only perturbing a small number of edge perturbations, including addition and deletion, our optimization-based attack can lead to a noticeable decrease in classification performance. Moreover, leveraging our gradient-based attack, we propose the first optimization-based adversarial training for GNNs. Our method yields higher robustness against both different gradient based and greedy attack methods without sacrifice classification accuracy on original graph.

scholarly journals Graph-Free Knowledge Distillation for Graph Neural Networks

2021 ◽  
Author(s):  
Xiang Deng ◽  
Zhongfei Zhang
Keyword(s):  
Neural Networks ◽  
Back Propagation ◽  
Multinomial Distribution ◽  
Large Data ◽  
Training Data ◽  
Grid Data ◽  
Continuous Space ◽  
Graph Data ◽  
Knowledge Distillation ◽  
Graph Neural Networks

Knowledge distillation (KD) transfers knowledge from a teacher network to a student by enforcing the student to mimic the outputs of the pretrained teacher on training data. However, data samples are not always accessible in many cases due to large data sizes, privacy, or confidentiality. Many efforts have been made on addressing this problem for convolutional neural networks (CNNs) whose inputs lie in a grid domain within a continuous space such as images and videos, but largely overlook graph neural networks (GNNs) that handle non-grid data with different topology structures within a discrete space. The inherent differences between their inputs make these CNN-based approaches not applicable to GNNs. In this paper, we propose to our best knowledge the first dedicated approach to distilling knowledge from a GNN without graph data. The proposed graph-free KD (GFKD) learns graph topology structures for knowledge transfer by modeling them with multinomial distribution. We then introduce a gradient estimator to optimize this framework. Essentially, the gradients w.r.t. graph structures are obtained by only using GNN forward-propagation without back-propagation, which means that GFKD is compatible with modern GNN libraries such as DGL and Geometric. Moreover, we provide the strategies for handling different types of prior knowledge in the graph data or the GNNs. Extensive experiments demonstrate that GFKD achieves the state-of-the-art performance for distilling knowledge from GNNs without training data.

scholarly journals Rumor Detection Based on SAGNN: Simplified Aggregation Graph Neural Networks

2021 ◽  
Vol 3 (1) ◽  
pp. 84-94
Author(s):  
Liang Zhang ◽  
Jingqun Li ◽  
Bin Zhou ◽  
Yan Jia
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Network Architectures ◽  
Neural Network Architecture ◽  
Convolutional Networks ◽  
Effective Manner ◽  
Deep Learning Network ◽  
Graph Neural Networks ◽  
Rumor Detection

Identifying fake news on media has been an important issue. This is especially true considering the wide spread of rumors on popular social networks such as Twitter. Various kinds of techniques have been proposed for automatic rumor detection. In this work, we study the application of graph neural networks for rumor classification at a lower level, instead of applying existing neural network architectures to detect rumors. The responses to true rumors and false rumors display distinct characteristics. This suggests that it is essential to capture such interactions in an effective manner for a deep learning network to achieve better rumor detection performance. To this end we present a simplified aggregation graph neural network architecture. Experiments on publicly available Twitter datasets demonstrate that the proposed network has performance on a par with or even better than that of state-of-the-art graph convolutional networks, while significantly reducing the computational complexity.

scholarly journals Personal Interest Attention Graph Neural Networks for Session-Based Recommendation

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1500
Author(s):  
Xiangde Zhang ◽  
Yuan Zhou ◽  
Jianping Wang ◽  
Xiaojun Lu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Objective Function ◽  
Cross Entropy ◽  
Personal Interest ◽  
Entropy Loss ◽  
Convolutional Networks ◽  
The Cross ◽  
Graph Neural Networks

Session-based recommendations aim to predict a user’s next click based on the user’s current and historical sessions, which can be applied to shopping websites and APPs. Existing session-based recommendation methods cannot accurately capture the complex transitions between items. In addition, some approaches compress sessions into a fixed representation vector without taking into account the user’s interest preferences at the current moment, thus limiting the accuracy of recommendations. Considering the diversity of items and users’ interests, a personalized interest attention graph neural network (PIA-GNN) is proposed for session-based recommendation. This approach utilizes personalized graph convolutional networks (PGNN) to capture complex transitions between items, invoking an interest-aware mechanism to activate users’ interest in different items adaptively. In addition, a self-attention layer is used to capture long-term dependencies between items when capturing users’ long-term preferences. In this paper, the cross-entropy loss is used as the objective function to train our model. We conduct rich experiments on two real datasets, and the results show that PIA-GNN outperforms existing personalized session-aware recommendation methods.

Sign in / Sign up

Export Citation Format

Share Document