Generative Adversarial Network for Imitation Learning from Single Demonstration

Imitation learning is an effective method for training an autonomous agent to accomplish a task by imitating expert behaviors in their demonstrations. However, traditional imitation learning methods require a large number of expert demonstrations in order to learn a complex behavior. Such a disadvantage has limited the potential of imitation learning in complex tasks where the expert demonstrations are not sufficient. In order to address the problem, we propose a Generative Adversarial Network-based model which is designed to learn optimal policies using only a single demonstration. The proposed model is evaluated on two simulated tasks in comparison with other methods. The results show that our proposed model is capable of completing considered tasks despite the limitation in the number of expert demonstrations, which clearly indicate the potential of our model.

Download Full-text

Domain Adaptation for Imitation Learning Using Generative Adversarial Network

Sensors ◽

10.3390/s21144718 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4718

Author(s):

Tho Nguyen Duc ◽

Chanh Minh Tran ◽

Phan Xuan Tan ◽

Eiji Kamioka

Keyword(s):

Optimal Policy ◽

Domain Adaptation ◽

Imitation Learning ◽

High Dimensional ◽

Autonomous Agent ◽

Generative Adversarial Network ◽

Reward Function ◽

Domain Specific ◽

Adversarial Network ◽

Domain Configuration

Imitation learning is an effective approach for an autonomous agent to learn control policies when an explicit reward function is unavailable, using demonstrations provided from an expert. However, standard imitation learning methods assume that the agents and the demonstrations provided by the expert are in the same domain configuration. Such an assumption has made the learned policies difficult to apply in another distinct domain. The problem is formalized as domain adaptive imitation learning, which is the process of learning how to perform a task optimally in a learner domain, given demonstrations of the task in a distinct expert domain. We address the problem by proposing a model based on Generative Adversarial Network. The model aims to learn both domain-shared and domain-specific features and utilizes it to find an optimal policy across domains. The experimental results show the effectiveness of our model in a number of tasks ranging from low to complex high-dimensional.

Download Full-text

Generative Adversarial Networks-Based Semi-Supervised Automatic Modulation Recognition for Cognitive Radio Networks

Sensors ◽

10.3390/s18113913 ◽

2018 ◽

Vol 18 (11) ◽

pp. 3913 ◽

Cited By ~ 6

Author(s):

Mingxuan Li ◽

Ou Li ◽

Guangyi Liu ◽

Ce Zhang

Keyword(s):

Deep Learning ◽

Cognitive Radio ◽

Supervised Learning ◽

Rapid Development ◽

Generative Adversarial Networks ◽

Modulation Recognition ◽

Learning Methods ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Automatic Modulation Recognition

With the recently explosive growth of deep learning, automatic modulation recognition has undergone rapid development. Most of the newly proposed methods are dependent on large numbers of labeled samples. We are committed to using fewer labeled samples to perform automatic modulation recognition in the cognitive radio domain. Here, a semi-supervised learning method based on adversarial training is proposed which is called signal classifier generative adversarial network. Most of the prior methods based on this technology involve computer vision applications. However, we improve the existing network structure of a generative adversarial network by adding the encoder network and a signal spatial transform module, allowing our framework to address radio signal processing tasks more efficiently. These two technical improvements effectively avoid nonconvergence and mode collapse problems caused by the complexity of the radio signals. The results of simulations show that compared with well-known deep learning methods, our method improves the classification accuracy on a synthetic radio frequency dataset by 0.1% to 12%. In addition, we verify the advantages of our method in a semi-supervised scenario and obtain a significant increase in accuracy compared with traditional semi-supervised learning methods.

Download Full-text

IntroVNMT: An Introspective Model for Variational Neural Machine Translation

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6411 ◽

2020 ◽

Vol 34 (05) ◽

pp. 8830-8837

Author(s):

Xin Sheng ◽

Linli Xu ◽

Junliang Guo ◽

Jingchang Liu ◽

Ruoyu Zhao ◽

...

Keyword(s):

Machine Translation ◽

Latent Variables ◽

Image Synthesis ◽

Target Language ◽

Generative Adversarial Network ◽

Neural Machine Translation ◽

Adversarial Network ◽

Proposed Model ◽

Model Training ◽

High Level

We propose a novel introspective model for variational neural machine translation (IntroVNMT) in this paper, inspired by the recent successful application of introspective variational autoencoder (IntroVAE) in high quality image synthesis. Different from the vanilla variational NMT model, IntroVNMT is capable of improving itself introspectively by evaluating the quality of the generated target sentences according to the high-level latent variables of the real and generated target sentences. As a consequence of introspective training, the proposed model is able to discriminate between the generated and real sentences of the target language via the latent variables generated by the encoder of the model. In this way, IntroVNMT is able to generate more realistic target sentences in practice. In the meantime, IntroVNMT inherits the advantages of the variational autoencoders (VAEs), and the model training process is more stable than the generative adversarial network (GAN) based models. Experimental results on different translation tasks demonstrate that the proposed model can achieve significant improvements over the vanilla variational NMT model.

Download Full-text

Measuring Traffic Volumes Using an Autoencoder with No Need to Tag Images with Labels

Electronics ◽

10.3390/electronics9050702 ◽

2020 ◽

Vol 9 (5) ◽

pp. 702

Author(s):

Seungbin Roh ◽

Johyun Shin ◽

Keemin Sohn

Keyword(s):

Input Image ◽

Video Frame ◽

Generative Adversarial Network ◽

Detection Algorithms ◽

Adversarial Network ◽

Simpler Algorithm ◽

Proposed Model ◽

Traffic Volumes ◽

Step Algorithm ◽

Almost All

Almost all vision technologies that are used to measure traffic volume use a two-step procedure that involves tracking and detecting. Object detection algorithms such as YOLO and Fast-RCNN have been successfully applied to detecting vehicles. The tracking of vehicles requires an additional algorithm that can trace the vehicles that appear in a previous video frame to their appearance in a subsequent frame. This two-step algorithm prevails in the field but requires substantial computation resources for training, testing, and evaluation. The present study devised a simpler algorithm based on an autoencoder that requires no labeled data for training. An autoencoder was trained on the pixel intensities of a virtual line placed on images in an unsupervised manner. The last hidden node of the former encoding portion of the autoencoder generates a scalar signal that can be used to judge whether a vehicle is passing. A cycle-consistent generative adversarial network (CycleGAN) was used to transform an original input photo of complex vehicle images and backgrounds into a simple illustration input image that enhances the performance of the autoencoder in judging the presence of a vehicle. The proposed model is much lighter and faster than a YOLO-based model, and accuracy of the proposed model is equivalent to, or better than, a YOLO-based model. In measuring traffic volumes, the proposed approach turned out to be robust in terms of both accuracy and efficiency.

Download Full-text

Joint Entity and Event Extraction with Generative Adversarial Imitation Learning

Data Intelligence ◽

10.1162/dint_a_00014 ◽

2019 ◽

Vol 1 (2) ◽

pp. 99-120 ◽

Cited By ~ 6

Author(s):

Tongtao Zhang ◽

Heng Ji ◽

Avirup Sil

Keyword(s):

State Of The Art ◽

Ground Truth ◽

Event Extraction ◽

Imitation Learning ◽

Learning Method ◽

Inverse Reinforcement Learning ◽

Generative Adversarial Network ◽

Adversarial Network ◽

The Difference ◽

New Framework

We propose a new framework for entity and event extraction based on generative adversarial imitation learning—an inverse reinforcement learning method using a generative adversarial network (GAN). We assume that instances and labels yield to various extents of difficulty and the gains and penalties (rewards) are expected to be diverse. We utilize discriminators to estimate proper rewards according to the difference between the labels committed by the ground-truth (expert) and the extractor (agent). Our experiments demonstrate that the proposed framework outperforms state-of-the-art methods.

Download Full-text

Impact of GAN-based lesion-focused medical image super-resolution on the robustness of radiomic features

Scientific Reports ◽

10.1038/s41598-021-00898-z ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Erick Costa de Farias ◽

Christian di Noia ◽

Changhee Han ◽

Evis Sala ◽

Mauro Castelli ◽

...

Keyword(s):

Biomarker Discovery ◽

Super Resolution ◽

Principal Component ◽

Medical Decision ◽

Perceptual Quality ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Proposed Model ◽

Spatial Pyramid Pooling ◽

Image Super Resolution

AbstractRobust machine learning models based on radiomic features might allow for accurate diagnosis, prognosis, and medical decision-making. Unfortunately, the lack of standardized radiomic feature extraction has hampered their clinical use. Since the radiomic features tend to be affected by low voxel statistics in regions of interest, increasing the sample size would improve their robustness in clinical studies. Therefore, we propose a Generative Adversarial Network (GAN)-based lesion-focused framework for Computed Tomography (CT) image Super-Resolution (SR); for the lesion (i.e., cancer) patch-focused training, we incorporate Spatial Pyramid Pooling (SPP) into GAN-Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE). At $$2\times $$ 2 × SR, the proposed model achieved better perceptual quality with less blurring than the other considered state-of-the-art SR methods, while producing comparable results at $$4\times $$ 4 × SR. We also evaluated the robustness of our model’s radiomic feature in terms of quantization on a different lung cancer CT dataset using Principal Component Analysis (PCA). Intriguingly, the most important radiomic features in our PCA-based analysis were the most robust features extracted on the GAN-super-resolved images. These achievements pave the way for the application of GAN-based image Super-Resolution techniques for studies of radiomics for robust biomarker discovery.

Download Full-text

Data augmentation-based conditional Wasserstein generative adversarial network-gradient penalty for XSS attack detection system

PeerJ Computer Science ◽

10.7717/peerj-cs.328 ◽

2020 ◽

Vol 6 ◽

pp. e328

Author(s):

Fawaz Mahiuob Mohammed Mokbal ◽

Dan Wang ◽

Xiaoxi Wang ◽

Lihua Fu

Keyword(s):

Web Applications ◽

Data Augmentation ◽

Detection System ◽

Attack Detection ◽

Generative Adversarial Network ◽

Minority Class ◽

Adversarial Network ◽

Proposed Model ◽

Worldwide Web ◽

Resource Data

The rapid growth of the worldwide web and accompanied opportunities of web applications in various aspects of life have attracted the attention of organizations, governments, and individuals. Consequently, web applications have increasingly become the target of cyberattacks. Notably, cross-site scripting (XSS) attacks on web applications are increasing and have become the critical focus of information security experts’ reports. Machine learning (ML) technique has significantly advanced and shown impressive results in the area of cybersecurity. However, XSS training datasets are often limited and significantly unbalanced, which does not meet well-developed ML algorithms’ requirements and potentially limits the detection system efficiency. Furthermore, XSS attacks have multiple payload vectors that execute in different ways, resulting in many real threats passing through the detection system undetected. In this study, we propose a conditional Wasserstein generative adversarial network with a gradient penalty to enhance the XSS detection system in a low-resource data environment. The proposed method integrates a conditional generative adversarial network and Wasserstein generative adversarial network with a gradient penalty to obtain necessary data from directivity, which improves the strength of the security system over unbalance data. The proposed method generates synthetic samples of minority class that have identical distribution as real XSS attack scenarios. The augmented data were used to train a new boosting model and subsequently evaluated the model using a real test dataset. Experiments on two unbalanced XSS attack datasets demonstrate that the proposed model generates valid and reliable samples. Furthermore, the samples were indistinguishable from real XSS data and significantly enhanced the detection of XSS attacks compared with state-of-the-art methods.

Download Full-text

Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18126216 ◽

2021 ◽

Vol 18 (12) ◽

pp. 6216

Author(s):

Mooseop Kim ◽

YunKyung Park ◽

KyeongDeok Moon ◽

Chi Yoon Jeong

Keyword(s):

Visual Information ◽

Experimental Results ◽

Auditory Sensitivity ◽

Sensory Substitution ◽

Sensitivity Analyses ◽

Auditory Signal ◽

Behavioral Experiments ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Proposed Model

Visual-auditory sensory substitution has demonstrated great potential to help visually impaired and blind groups to recognize objects and to perform basic navigational tasks. However, the high latency between visual information acquisition and auditory transduction may contribute to the lack of the successful adoption of such aid technologies in the blind community; thus far, substitution methods have remained only laboratory-scale research or pilot demonstrations. This high latency for data conversion leads to challenges in perceiving fast-moving objects or rapid environmental changes. To reduce this latency, prior analysis of auditory sensitivity is necessary. However, existing auditory sensitivity analyses are subjective because they were conducted using human behavioral analysis. Therefore, in this study, we propose a cross-modal generative adversarial network-based evaluation method to find an optimal auditory sensitivity to reduce transmission latency in visual-auditory sensory substitution, which is related to the perception of visual information. We further conducted a human-based assessment to evaluate the effectiveness of the proposed model-based analysis in human behavioral experiments. We conducted experiments with three participant groups, including sighted users (SU), congenitally blind (CB) and late-blind (LB) individuals. Experimental results from the proposed model showed that the temporal length of the auditory signal for sensory substitution could be reduced by 50%. This result indicates the possibility of improving the performance of the conventional vOICe method by up to two times. We confirmed that our experimental results are consistent with human assessment through behavioral experiments. Analyzing auditory sensitivity with deep learning models has the potential to improve the efficiency of sensory substitution.

Download Full-text

Improving representation learning in autoencoders via multidimensional interpolation and dual regularizations

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/453 ◽

2019 ◽

Author(s):

Sheng Qian ◽

Guanyue Li ◽

Wen-Ming Cao ◽

Cheng Liu ◽

Si Wu ◽

...

Keyword(s):

Representation Learning ◽

Data Interpolation ◽

Generative Adversarial Network ◽

Maximum Mean Discrepancy ◽

Adversarial Network ◽

Multidimensional Interpolation ◽

Proposed Model ◽

Data Representations ◽

Latent Representations ◽

Improve Representation

Autoencoders enjoy a remarkable ability to learn data representations. Research on autoencoders shows that the effectiveness of data interpolation can reflect the performance of representation learning. However, existing interpolation methods in autoencoders do not have enough capability of traversing a possible region between two datapoints on a data manifold, and the distribution of interpolated latent representations is not considered.To address these issues, we aim to fully exert the potential of data interpolation and further improve representation learning in autoencoders. Specifically, we propose the multidimensional interpolation to increase the capability of data interpolation by randomly setting interpolation coefficients for each dimension of latent representations. In addition, we regularize autoencoders in both the latent and the data spaces by imposing a prior on latent representations in the Maximum Mean Discrepancy (MMD) framework and encouraging generated datapoints to be realistic in the Generative Adversarial Network (GAN) framework. Compared to representative models, our proposed model has empirically shown that representation learning exhibits better performance on downstream tasks on multiple benchmarks.

Download Full-text

Unpaired Underwater Image Enhancement Based on CycleGAN

Information ◽

10.3390/info13010001 ◽

2021 ◽

Vol 13 (1) ◽

pp. 1

Author(s):

Rong Du ◽

Weiwei Li ◽

Shudong Chen ◽

Congying Li ◽

Yong Zhang

Keyword(s):

Image Enhancement ◽

Paired Data ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Underwater Image ◽

Training Stage ◽

Proposed Model ◽

Degraded Image ◽

Enhancement Method ◽

Degraded Images

Underwater image enhancement recovers degraded underwater images to produce corresponding clear images. Image enhancement methods based on deep learning usually use paired data to train the model, while such paired data, e.g., the degraded images and the corresponding clear images, are difficult to capture simultaneously in the underwater environment. In addition, how to retain the detailed information well in the enhanced image is another critical problem. To solve such issues, we propose a novel unpaired underwater image enhancement method via a cycle generative adversarial network (UW-CycleGAN) to recover the degraded underwater images. Our proposed UW-CycleGAN model includes three main modules: (1) A content loss regularizer is adopted into the generator in CycleGAN, which constrains the detailed information existing in one degraded image to remain in the corresponding generated clear image; (2) A blur-promoting adversarial loss regularizer is introduced into the discriminator to reduce the blur and noise in the generated clear images; (3) We add the DenseNet block to the generator to retain more information of each feature map in the training stage. Finally, experimental results on two unpaired underwater image datasets produced satisfactory performance compared to the state-of-the-art image enhancement methods, which proves the effectiveness of the proposed model.

Download Full-text