scholarly journals Derivative-free optimization adversarial attacks for graph convolutional networks

2021 ◽  
Vol 7 ◽  
pp. e693
Author(s):  
Runze Yang ◽  
Teng Long
Keyword(s):  
Search Space ◽  
Classification Performance ◽  
Convolutional Networks ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Adversarial Examples ◽  
Series Of Experiments ◽  
Adversarial Attack ◽  
Direct Attack

In recent years, graph convolutional networks (GCNs) have emerged rapidly due to their excellent performance in graph data processing. However, recent researches show that GCNs are vulnerable to adversarial attacks. An attacker can maliciously modify edges or nodes of the graph to mislead the model’s classification of the target nodes, or even cause a degradation of the model’s overall classification performance. In this paper, we first propose a black-box adversarial attack framework based on derivative-free optimization (DFO) to generate graph adversarial examples without using gradient and apply advanced DFO algorithms conveniently. Second, we implement a direct attack algorithm (DFDA) using the Nevergrad library based on the framework. Additionally, we overcome the problem of large search space by redesigning the perturbation vector using constraint size. Finally, we conducted a series of experiments on different datasets and parameters. The results show that DFDA outperforms Nettack in most cases, and it can achieve an average attack success rate of more than 95% on the Cora dataset when perturbing at most eight edges. This demonstrates that our framework can fully exploit the potential of DFO methods in node classification adversarial attacks.

An empirical study on the GEOtop hydrological model optimal estimation and uncertainty reduction using supercomputers

2021 ◽  
Author(s):  
Giacomo Bertoldi ◽  
Stefano Campanella ◽  
Emanuele Cordano ◽  
Alberto Sartori
Keyword(s):  
Learning Community ◽  
Hydrological Model ◽  
Computing Time ◽  
Optimal Estimation ◽  
Search Space ◽  
Water Retention Curve ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Discretionary Choices ◽  
Computational Aspects

<p>Proper characterization of uncertainty remains a major research and operational challenge in Earth and Environmental Systems Models (EESMs). In fact, model calibration is often more an art than a science: one must make several discretionary choices, guided more by his own experience and intuition than by the scientific method. In practice, this means that the result of calibration (CA) could be suboptimal. One of the challenges of CA is the large number of parameters involved in EESM, which hence are usually selected with the help of a preliminary sensitivity analysis (SA). Finally, the computational burden of EESMs models and the large volume of the search space make SA and CA very time-consuming processes.</p><p>This work applies a modern HPC approach to optimize a complex, over parameterized hydrological model, improving the computational efficiency of SA/CA. We apply the derivative-free optimization algorithms implemented in the Facebook Nevergrad Python library (Rapin and Teytaud, 2018) on a HPC cluster, thanks to the Dask framework (Dask Development Team, 2016).</p><p>The approach has been applied to the GEOtop hydrological model (Rigon et al., 2006; Endrizzi et al., 2014) to predict the time evolution of variables as soil water content and evapotranspiration for several mountain agricultural sites in South Tyrol with different elevation, land cover (pasture, meadow, orchard), soil types.</p><p>We performed simulations on one-dimensional domains, where the model solves the energy and water budget equations in a column of soil and neglects the lateral water fluxes.  Even neglecting the distribution of parameters across layers of soil, considering a homogeneous column, one has tens of parameters, controlling soil and vegetation properties, where only a few of them are experimentally available. </p><p>Because the interpretation of global SA could be difficult or misleading and the number of model evaluations needed by SA is comparable with CA, we employed the following strategy. We performed CA using a full set of continuous parameters and SA after CA, using the samples collected during CA, to interpret the results. However, given the above-mentioned computational challenges, this strategy is possible only using HPC resources. For this reason, we focused on the computational aspects of calibration from an HPC perspective and examined the scaling of these algorithms and their implementation up to 1024 cores on a cluster. Other issues that we had to address were the complex shape of the search space and robustness of CA and SA against model convergence failure.</p><p>HPC  techniques allow to calibrate models with a high number of parameters within a reasonable computing time and  exploring the parameters space properly. This is particularly important with noisy, multimodal objective functions. In our case, HPC was essential to determine the  parameters controlling the water retention curve, which is highly not linear.  The developed  framework, which is published and freely available on GitHub, shows also how libraries and tools used within the machine learning community could be useful and easily adapted to EESMs CA.</p>

scholarly journals Cycle-Consistent Adversarial GAN: The Integration of Adversarial Attack and Defense

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Lingyun Jiang ◽  
Kai Qiao ◽  
Ruoxi Qin ◽  
Linyuan Wang ◽  
Wanting Yu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Small Magnitude ◽  
Defense Strategies ◽  
Adversarial Examples ◽  
Adversarial Attack ◽  
Public Datasets ◽  
Attack And Defense

In image classification of deep learning, adversarial examples where input is intended to add small magnitude perturbations may mislead deep neural networks (DNNs) to incorrect results, which means DNNs are vulnerable to them. Different attack and defense strategies have been proposed to better research the mechanism of deep learning. However, those researches in these networks are only for one aspect, either an attack or a defense. There is in the improvement of offensive and defensive performance, and it is difficult to promote each other in the same framework. In this paper, we propose Cycle-Consistent Adversarial GAN (CycleAdvGAN) to generate adversarial examples, which can learn and approximate the distribution of the original instances and adversarial examples, especially promoting attackers and defenders to confront each other and improve their ability. For CycleAdvGAN, once the GeneratorA and D are trained, GA can generate adversarial perturbations efficiently for any instance, improving the performance of the existing attack methods, and GD can generate recovery adversarial examples to clean instances, defending against existing attack methods. We apply CycleAdvGAN under semiwhite-box and black-box settings on two public datasets MNIST and CIFAR10. Using the extensive experiments, we show that our method has achieved the state-of-the-art adversarial attack method and also has efficiently improved the defense ability, which made the integration of adversarial attack and defense come true. In addition, it has improved the attack effect only trained on the adversarial dataset generated by any kind of adversarial attack.

scholarly journals Adversarial Attack and Defense on Deep Neural Network-Based Voice Processing Systems: An Overview

2021 ◽  
Vol 11 (18) ◽  
pp. 8450
Author(s):  
Xiaojiao Chen ◽  
Sheng Li ◽  
Hao Huang
Keyword(s):  
Deep Neural Networks ◽  
Daily Lives ◽  
Systematic Classification ◽  
Online Purchases ◽  
Adversarial Examples ◽  
Adversarial Attack ◽  
Attack And Defense ◽  
Voice Processing ◽  
Significant Attention

Voice Processing Systems (VPSes), now widely deployed, have become deeply involved in people’s daily lives, helping drive the car, unlock the smartphone, make online purchases, etc. Unfortunately, recent research has shown that those systems based on deep neural networks are vulnerable to adversarial examples, which attract significant attention to VPS security. This review presents a detailed introduction to the background knowledge of adversarial attacks, including the generation of adversarial examples, psychoacoustic models, and evaluation indicators. Then we provide a concise introduction to defense methods against adversarial attacks. Finally, we propose a systematic classification of adversarial attacks and defense methods, with which we hope to provide a better understanding of the classification and structure for beginners in this field.

scholarly journals ALBERT over Match-LSTM Network for Intelligent Questions Classification in Chinese

Agronomy ◽  
2021 ◽  
Vol 11 (8) ◽  
pp. 1530
Author(s):  
Xiaomin Wang ◽  
Haoriqin Wang ◽  
Guocheng Zhao ◽  
Zhichao Liu ◽  
Huarui Wu
Keyword(s):  
Question Answering ◽  
Information Service ◽  
Classification Performance ◽  
Short Text ◽  
Svm Model ◽  
Series Of Experiments ◽  
Lstm Network ◽  
Sparse Features ◽  
Gated Recurrent Unit

This paper introduces a series of experiments with an ALBERT over match-LSTM network on the top of pre-trained word vectors, for accurate classification of intelligent question answering and thus the guarantee of precise information service. To improve the performance of data classification, a short text classification method based on an ALBERT and match-LSTM model was proposed to overcome the limitations of the classification process, such as few vocabularies, sparse features, large amount of data, lots of noise and poor normalization. In the model, Jieba word segmentation tools and agricultural dictionary were selected to text segmentation, GloVe algorithm was then adopted to expand the text characteristic and weighted word vector according to the text of key vector, bi-directional gated recurrent unit was applied to catch the context feature information and multi-convolutional neural networks were finally established to gain local multidimensional characteristics of text. Batch normalization, Dropout, Global Average Pooling and Global Max Pooling were utilized to solve overfitting problem. The results showed that the model could classify questions accurately, with a precision of 96.8%. Compared with other classification models, such as multi-SVM model and CNN model, ALBERT+match-LSTM had obvious advantages in classification performance in intelligent Agri-tech information service.

scholarly journals Beyond Digital Domain: Fooling Deep Learning Based Recognition System in Physical World

2020 ◽  
Vol 34 (01) ◽  
pp. 1088-1095
Author(s):  
Kaichen Yang ◽  
Tzungyu Tsai ◽  
Honggang Yu ◽  
Tsung-Yi Ho ◽  
Yier Jin
Keyword(s):  
Object Detection ◽  
Defense Mechanisms ◽  
Recognition System ◽  
Physical World ◽  
Physical Attack ◽  
Adversarial Examples ◽  
Series Of Experiments ◽  
Adversarial Attack ◽  
Outdoor Experiments ◽  
Indoor And Outdoor

Adversarial examples that can fool deep neural network (DNN) models in computer vision present a growing threat. The current methods of launching adversarial attacks concentrate on attacking image classifiers by adding noise to digital inputs. The problem of attacking object detection models and adversarial attacks in physical world are rarely touched. Some prior works are proposed to launch physical adversarial attack against object detection models, but limited by certain aspects. In this paper, we propose a novel physical adversarial attack targeting object detection models. Instead of simply printing images, we manufacture real metal objects that could achieve the adversarial effect. In both indoor and outdoor experiments we show our physical adversarial objects can fool widely applied object detection models including SSD, YOLO and Faster R-CNN in various environments. We also test our attack in a variety of commercial platforms for object detection and demonstrate that our attack is still valid on these platforms. Consider the potential defense mechanisms our adversarial objects may encounter, we conduct a series of experiments to evaluate the effect of existing defense methods on our physical attack.

scholarly journals Optimal Power Flow Using Genetic Algorithm

2021 ◽  
Author(s):  
Fernando Buzzulini Prioste
Keyword(s):  
Genetic Algorithm ◽  
Power Flow ◽  
Optimal Power Flow ◽  
Optimization Technique ◽  
Search Space ◽  
Test System ◽  
Optimal Power ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Parameter Search Space

This paper presents a genetic algorithm (GA) to solve Optimal Power Flow (OPF) problems, optimizing electricity generation fuel cost. The GA based OPF is a derivative free optimization technique that relies on the evaluation of several points in the parameter search space strictly on the objective function. A 3 bus system and the IEEE 30 bus test system are used to validate the developed GA based OPF by means of comparisons with an interior point based optimal power flow.

HyperNOMAD

2021 ◽  
Vol 47 (3) ◽  
pp. 1-27
Author(s):  
Dounia Lakhmiri ◽  
Sébastien Le Digabel ◽  
Christophe Tribes
Keyword(s):  
Neural Network ◽  
Learning Process ◽  
Deep Neural Network ◽  
Search Space ◽  
Categorical Variables ◽  
Current State ◽  
Derivative Free Optimization ◽  
Highly Sensitive ◽  
Derivative Free ◽  
Application Tuning

The performance of deep neural networks is highly sensitive to the choice of the hyperparameters that define the structure of the network and the learning process. When facing a new application, tuning a deep neural network is a tedious and time-consuming process that is often described as a “dark art.” This explains the necessity of automating the calibration of these hyperparameters. Derivative-free optimization is a field that develops methods designed to optimize time-consuming functions without relying on derivatives. This work introduces the HyperNOMAD package, an extension of the NOMAD software that applies the MADS algorithm [7] to simultaneously tune the hyperparameters responsible for both the architecture and the learning process of a deep neural network (DNN). This generic approach allows for an important flexibility in the exploration of the search space by taking advantage of categorical variables. HyperNOMAD is tested on the MNIST, Fashion-MNIST, and CIFAR-10 datasets and achieves results comparable to the current state of the art.

scholarly journals INCORPORATING INTERFEROMETRIC COHERENCE INTO LULC CLASSIFICATION OF AIRBORNE POLSAR-IMAGES USING FULLY CONVOLUTIONAL NETWORKS

2020 ◽  
Vol XLIII-B1-2020 ◽  
pp. 115-122
Author(s):  
S. Schmitz ◽  
M. Weinmann ◽  
A. Thiele
Keyword(s):  
High Resolution ◽  
Semantic Segmentation ◽  
Classification Performance ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
Classification Framework ◽  
Sar Data ◽  
Image Pairs ◽  
Airborne Sar

Abstract. Inspired by the application of state-of-the-art Fully Convolutional Networks (FCNs) for the semantic segmentation of high-resolution optical imagery, recent works transfer this methodology successfully to pixel-wise land use and land cover (LULC) classification of PolSAR data. So far, mainly single PolSAR images are included in the FCN-based classification processes. To further increase classification accuracy, this paper presents an approach for integrating interferometric coherence derived from co-registered image pairs into a FCN-based classification framework. A network based on an encoder-decoder structure with two separated encoder branches is presented for this task. It extracts features from polarimetric backscattering intensities on the one hand and interferometric coherence on the other hand. Based on a joint representation of the complementary features pixel-wise classification is performed. To overcome the scarcity of labelled SAR data for training and testing, annotations are generated automatically by fusing available LULC products. Experimental evaluation is performed on high-resolution airborne SAR data, captured over the German Wadden Sea. The results demonstrate that the proposed model produces smooth and accurate classification maps. A comparison with a single-branch FCN model indicates that the appropriate integration of interferometric coherence enables the improvement of classification performance.

scholarly journals A Fast Method for Fuzzy Rules Learning with Derivative-Free Optimization by Formulating Independent Evaluations of Each Fuzzy Rule

2021 ◽  
Vol 25 (2) ◽  
pp. 213-225
Author(s):  
Kiyohiko Uehara ◽  
Kaoru Hirota ◽  
◽  
Keyword(s):  
Fuzzy Rule ◽  
Optimization Methods ◽  
Search Space ◽  
Fuzzy Rules ◽  
Fast Method ◽  
Evaluation Functions ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Effective Use ◽  
Learning Data

A method is proposed for evaluating fuzzy rules independently of each other in fuzzy rules learning. The proposed method is named α-FUZZI-ES (α-weight-based fuzzy-rule independent evaluations) in this paper. In α-FUZZI-ES, the evaluation value of a fuzzy system is divided out among the fuzzy rules by using the compatibility degrees of the learning data. By the effective use of α-FUZZI-ES, a method for fast fuzzy rules learning is proposed. This is named α-FUZZI-ES learning (α-FUZZI-ES-based fuzzy rules learning) in this paper. α-FUZZI-ES learning is especially effective when evaluation functions are not differentiable and derivative-based optimization methods cannot be applied to fuzzy rules learning. α-FUZZI-ES learning makes it possible to optimize fuzzy rules independently of each other. This property reduces the dimensionality of the search space in finding the optimum fuzzy rules. Thereby, α-FUZZI-ES learning can attain fast convergence in fuzzy rules optimization. Moreover, α-FUZZI-ES learning can be efficiently performed with hardware in parallel to optimize fuzzy rules independently of each other. Numerical results show that α-FUZZI-ES learning is superior to the exemplary conventional scheme in terms of accuracy and convergence speed when the evaluation function is non-differentiable.

Sign in / Sign up

Export Citation Format

Share Document