scholarly journals Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 5883-5891
Author(s):  
Jianwen Sun ◽  
Tianwei Zhang ◽  
Xiaofei Xie ◽  
Lei Ma ◽  
Yan Zheng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
Critical Point ◽  
State Of The Art ◽  
Great Success ◽  
Severe Damage ◽  
Minimal Set ◽  
Adversarial Attack ◽  
Attack Strategy ◽  
Critical Moments

Adversarial attacks against conventional Deep Learning (DL) systems and algorithms have been widely studied, and various defenses were proposed. However, the possibility and feasibility of such attacks against Deep Reinforcement Learning (DRL) are less explored. As DRL has achieved great success in various complex tasks, designing effective adversarial attacks is an indispensable prerequisite towards building robust DRL algorithms. In this paper, we introduce two novel adversarial attack techniques to stealthily and efficiently attack the DRL agents. These two techniques enable an adversary to inject adversarial samples in a minimal set of critical moments while causing the most severe damage to the agent. The first technique is the critical point attack: the adversary builds a model to predict the future environmental states and agent's actions, assesses the damage of each possible attack strategy, and selects the optimal one. The second technique is the antagonist attack: the adversary automatically learns a domain-agnostic model to discover the critical moments of attacking the agent in an episode. Experimental results demonstrate the effectiveness of our techniques. Specifically, to successfully attack the DRL agent, our critical point technique only requires 1 (TORCS) or 2 (Atari Pong and Breakout) steps, and the antagonist technique needs fewer than 5 steps (4 Mujoco tasks), which are significant improvements over state-of-the-art methods.

scholarly journals Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing

2019 ◽  
Vol 33 ◽  
pp. 3598-3605 ◽  
Author(s):  
Ryosuke Furuta ◽  
Naoto Inoue ◽  
Toshihiko Yamasaki
Keyword(s):  
Image Processing ◽  
Reinforcement Learning ◽  
State Of The Art ◽  
Great Success ◽  
Local Color ◽  
Convolutional Network ◽  
Fully Convolutional Network ◽  
Effective Learning ◽  
Pixel Value ◽  
Problem Setting

This paper tackles a new problem setting: reinforcement learning with pixel-wise rewards (pixelRL) for image processing. After the introduction of the deep Q-network, deep RL has been achieving great success. However, the applications of deep RL for image processing are still limited. Therefore, we extend deep RL to pixelRL for various image processing applications. In pixelRL, each pixel has an agent, and the agent changes the pixel value by taking an action. We also propose an effective learning method for pixelRL that significantly improves the performance by considering not only the future states of the own pixel but also those of the neighbor pixels. The proposed method can be applied to some image processing tasks that require pixel-wise manipulations, where deep RL has never been applied.We apply the proposed method to three image processing tasks: image denoising, image restoration, and local color enhancement. Our experimental results demonstrate that the proposed method achieves comparable or better performance, compared with the state-of-the-art methods based on supervised learning.

scholarly journals Cycle-Consistent Adversarial GAN: The Integration of Adversarial Attack and Defense

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Lingyun Jiang ◽  
Kai Qiao ◽  
Ruoxi Qin ◽  
Linyuan Wang ◽  
Wanting Yu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Small Magnitude ◽  
Defense Strategies ◽  
Adversarial Examples ◽  
Adversarial Attack ◽  
Public Datasets ◽  
Attack And Defense

In image classification of deep learning, adversarial examples where input is intended to add small magnitude perturbations may mislead deep neural networks (DNNs) to incorrect results, which means DNNs are vulnerable to them. Different attack and defense strategies have been proposed to better research the mechanism of deep learning. However, those researches in these networks are only for one aspect, either an attack or a defense. There is in the improvement of offensive and defensive performance, and it is difficult to promote each other in the same framework. In this paper, we propose Cycle-Consistent Adversarial GAN (CycleAdvGAN) to generate adversarial examples, which can learn and approximate the distribution of the original instances and adversarial examples, especially promoting attackers and defenders to confront each other and improve their ability. For CycleAdvGAN, once the GeneratorA and D are trained, GA can generate adversarial perturbations efficiently for any instance, improving the performance of the existing attack methods, and GD can generate recovery adversarial examples to clean instances, defending against existing attack methods. We apply CycleAdvGAN under semiwhite-box and black-box settings on two public datasets MNIST and CIFAR10. Using the extensive experiments, we show that our method has achieved the state-of-the-art adversarial attack method and also has efficiently improved the defense ability, which made the integration of adversarial attack and defense come true. In addition, it has improved the attack effect only trained on the adversarial dataset generated by any kind of adversarial attack.

scholarly journals Medical Knowledge Graph in Chinese Using Deep Semantic Mobile Computation Based on IoT and WoT

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Wanheng Liu ◽  
Ling Yin ◽  
Cong Wang ◽  
Fulin Liu ◽  
Zhiyu Ni
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Medical Knowledge ◽  
Disease Diagnosis ◽  
Knowledge Graph ◽  
Great Success ◽  
Smart Healthcare ◽  
Made In

In this paper, a novel medical knowledge graph in Chinese approach applied in smart healthcare based on IoT and WoT is presented, using deep neural networks combined with self-attention to generate medical knowledge graph to make it more convenient for performing disease diagnosis and providing treatment advisement. Although great success has been made in the medical knowledge graph in recent studies, the issue of comprehensive medical knowledge graph in Chinese appropriate for telemedicine or mobile devices have been ignored. In our study, it is a working theory which is based on semantic mobile computing and deep learning. When several experiments have been carried out, it is demonstrated that it has better performance in generating various types of medical knowledge graph in Chinese, which is similar to that of the state-of-the-art. Also, it works well in the accuracy and comprehensive, which is much higher and highly consisted with the predictions of the theoretical model. It proves to be inspiring and encouraging that our work involving studies of medical knowledge graph in Chinese, which can stimulate the smart healthcare development.

scholarly journals Multi-agent reinforcement learning for character control

2021 ◽  
Author(s):  
Cheng Li ◽  
Levi Fussell ◽  
Taku Komura
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
Computer Games ◽  
State Of The Art ◽  
Research Topic ◽  
Future Directions ◽  
Computer Animations ◽  
Survey Papers ◽  
Simultaneous Control ◽  
Multi Agent

AbstractSimultaneous control of multiple characters has been a research topic that has been extensively pursued for applications in computer games and computer animations, for applications such as crowd simulation, controlling two characters carrying objects or fighting with one another and controlling a team of characters playing collective sports. With the advance in deep learning and reinforcement learning, there is a growing interest in applying multi-agent reinforcement learning for intelligently controlling the characters to produce realistic movements. In this paper we will survey the state-of-the-art MARL techniques that are applicable for character control. We will then survey papers that make use of MARL for multi-character control and then discuss about the possible future directions of research.

scholarly journals Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari

2018 ◽  
Author(s):  
Patryk Chrabąszcz ◽  
Ilya Loshchilov ◽  
Frank Hutter
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
State Of The Art ◽  
The State ◽  
Evolution Strategies ◽  
Learning Problems ◽  
Local Minima ◽  
Natural Evolution ◽  
The Many ◽  
Made In

Evolution Strategies (ES) have recently been demonstrated to be a viable alternative to reinforcement learning (RL) algorithms on a set of challenging deep learning problems, including Atari games and MuJoCo humanoid locomotion benchmarks. While the ES algorithms in that work belonged to the specialized class of natural evolution strategies (which resemble approximate gradient RL algorithms, such as REINFORCE), we demonstrate that even a very basic canonical ES algorithm can achieve the same or even better performance. This success of a basic ES algorithm suggests that the state-of-the-art can be advanced further by integrating the many advances made in the field of ES in the last decades.We also demonstrate that ES algorithms have very different performance characteristics than traditional RL algorithms: on some games, they learn to exploit the environment and perform much better while on others they can get stuck in suboptimal local minima. Combining their strengths and weaknesses with those of traditional RL algorithms is therefore likely to lead to new advances in the state-of-the-art for solving RL problems.

scholarly journals A deep learning model for gas storage optimization

2021 ◽  
Author(s):  
Nicolas Curin ◽  
Michael Kettler ◽  
Xi Kleisinger-Yu ◽  
Vlatka Komaric ◽  
Thomas Krabichler ◽  
...  
Keyword(s):  
Risk Management ◽  
Monte Carlo ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
State Of The Art ◽  
Gas Storage ◽  
High Dimensional ◽  
Forward Market ◽  
Storage Optimization ◽  
Deep Learning Model

AbstractTo the best of our knowledge, the application of deep learning in the field of quantitative risk management is still a relatively recent phenomenon. In this article, we utilize techniques inspired by reinforcement learning in order to optimize the operation plans of underground natural gas storage facilities. We provide a theoretical framework and assess the performance of the proposed method numerically in comparison to a state-of-the-art least-squares Monte-Carlo approach. Due to the inherent intricacy originating from the high-dimensional forward market as well as the numerous constraints and frictions, the optimization exercise can hardly be tackled by means of traditional techniques.

scholarly journals A State-of-the-Art Survey on Deep Learning Theory and Architectures

Electronics ◽  
2019 ◽  
Vol 8 (3) ◽  
pp. 292 ◽  
Author(s):  
Md Zahangir Alom ◽  
Tarek M. Taha ◽  
Chris Yakopcic ◽  
Stefan Westberg ◽  
Paheding Sidike ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Language Processing ◽  
Large Scale ◽  
Medical Information ◽  
State Of The Art ◽  
Generative Models ◽  
Learning Approaches

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.

scholarly journals Adversarial Attacks for Deep Learning-Based Infrared Object Detection

2021 ◽  
Vol 24 (6) ◽  
pp. 591-601
Author(s):  
Hoseong Kim ◽  
Jaeguk Hyun ◽  
Hyunjung Yoo ◽  
Chunho Kim ◽  
Hyunho Jeon
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Object Detection ◽  
Image Recognition ◽  
Rapid Growth ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Visible Image ◽  
Adversarial Attack

Recently, infrared object detection(IOD) has been extensively studied due to the rapid growth of deep neural networks(DNN). Adversarial attacks using imperceptible perturbation can dramatically deteriorate the performance of DNN. However, most adversarial attack works are focused on visible image recognition(VIR), and there are few methods for IOD. We propose deep learning-based adversarial attacks for IOD by expanding several state-of-the-art adversarial attacks for VIR. We effectively validate our claim through comprehensive experiments on two challenging IOD datasets, including FLIR and MSOD.

scholarly journals Adaptive Region Embedding for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7314-7321
Author(s):  
Liuyu Xiang ◽  
Xiaoming Jin ◽  
Lan Yi ◽  
Guiguang Ding
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Context Information ◽  
Great Success ◽  
Recurrent Networks ◽  
Learning Models ◽  
Benchmark Datasets

Deep learning models such as convolutional neural networks and recurrent networks are widely applied in text classification. In spite of their great success, most deep learning models neglect the importance of modeling context information, which is crucial to understanding texts. In this work, we propose the Adaptive Region Embedding to learn context representation to improve text classification. Specifically, a metanetwork is learned to generate a context matrix for each region, and each word interacts with its corresponding context matrix to produce the regional representation for further classification. Compared to previous models that are designed to capture context information, our model contains less parameters and is more flexible. We extensively evaluate our method on 8 benchmark datasets for text classification. The experimental results prove that our method achieves state-of-the-art performances and effectively avoids word ambiguity.

scholarly journals Learning 2-Opt Heuristics for Routing Problems via Deep Reinforcement Learning

2021 ◽  
Vol 2 (5) ◽  
Author(s):  
Paulo da Costa ◽  
Jason Rhuggenaath ◽  
Yingqian Zhang ◽  
Alp Akcay ◽  
Uzay Kaymak
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
State Of The Art ◽  
Gradient Algorithm ◽  
Routing Problems ◽  
Routing Problem ◽  
Local Search Heuristic ◽  
Policy Gradient ◽  
Previous State ◽  
The Traveling Salesman Problem

AbstractRecent works using deep learning to solve routing problems such as the traveling salesman problem (TSP) have focused on learning construction heuristics. Such approaches find good quality solutions but require additional procedures such as beam search and sampling to improve solutions and achieve state-of-the-art performance. However, few studies have focused on improvement heuristics, where a given solution is improved until reaching a near-optimal one. In this work, we propose to learn a local search heuristic based on 2-opt operators via deep reinforcement learning. We propose a policy gradient algorithm to learn a stochastic policy that selects 2-opt operations given a current solution. Moreover, we introduce a policy neural network that leverages a pointing attention mechanism, which can be easily extended to more general k-opt moves. Our results show that the learned policies can improve even over random initial solutions and approach near-optimal solutions faster than previous state-of-the-art deep learning methods for the TSP. We also show we can adapt the proposed method to two extensions of the TSP: the multiple TSP and the Vehicle Routing Problem, achieving results on par with classical heuristics and learned methods.

Sign in / Sign up

Export Citation Format

Share Document