OptAGAN: Entropy-based Finetuning on Text VAE-GAN

Word Generation ◽

Adversarial Networks ◽

Current State ◽

Latent Space ◽

Variational Autoencoder ◽

Maximum Likelihood Methods ◽

Transfer learning through large pre-trained models has changed the landscape of current applications in natural language processing (NLP). Recently Optimus, a variational autoencoder (VAE) which combines two pre-trained models, BERT and GPT-2, has been released, and its combination with generative adversarial networks (GANs) has been shown to produce novel, yet very human-looking text. The Optimus and GANs combination avoids the troublesome application of GANs to the discrete domain of text, and prevents the exposure bias of standard maximum likelihood methods. We combine the training of GANs in the latent space, with the finetuning of the decoder of Optimus for single word generation. This approach lets us model both the high-level features of the sentences, and the low-level word-by-word generation. We finetune using reinforcement learning (RL) by exploiting the structure of GPT-2 and by adding entropy-based intrinsically motivated rewards to balance between quality and diversity. We benchmark the results of the VAE-GAN model, and show the improvements brought by our RL finetuning on three widely used datasets for text generation, with results that greatly surpass the current state-of-the-art for the quality of the generated texts.

Improvement of Learning Stability of Generative Adversarial Network Using Variational Learning

Applied Sciences ◽

10.3390/app10134528 ◽

2020 ◽

Vol 10 (13) ◽

pp. 4528

Author(s):

Je-Yeol Lee ◽

Sang-Il Choi

Keyword(s):

Generative Models ◽

Learning Processes ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Secondary Network ◽

Variational Autoencoder ◽

Variational Learning

In this paper, we propose a new network model using variational learning to improve the learning stability of generative adversarial networks (GAN). The proposed method can be easily applied to improve the learning stability of GAN-based models that were developed for various purposes, given that the variational autoencoder (VAE) is used as a secondary network while the basic GAN structure is maintained. When the gradient of the generator vanishes in the learning process of GAN, the proposed method receives gradient information from the decoder of the VAE that maintains gradient stably, so that the learning processes of the generator and discriminator are not halted. The experimental results of the MNIST and the CelebA datasets verify that the proposed method improves the learning stability of the networks by overcoming the vanishing gradient problem of the generator, and maintains the excellent data quality of the conventional GAN-based generative models.

Single Image De-Raining Using Spinning Detail Perceptual Generative Adversarial Networks

Journal of Advanced Computational Intelligence and Intelligent Informatics ◽

10.20965/jaciii.2020.p0811 ◽

2020 ◽

Vol 24 (7) ◽

pp. 811-819

Author(s):

Kaizheng Chen ◽

◽

Yaping Dai ◽

Zhiyang Jia ◽

Kaoru Hirota

Keyword(s):

Perceptual Quality ◽

Training Procedure ◽

Single Image ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Real World Datasets ◽

In this paper, Spinning Detail Perceptual Generative Adversarial Networks (SDP-GAN) is proposed for single image de-raining. The proposed method adopts the Generative Adversarial Network (GAN) framework and consists of two following networks: the rain streaks generative network G and the discriminative network D. To reduce the background interference, we propose a rain streaks generative network which not only focuses on the high frequency detail map of rainy image, but also directly reduces the mapping range from input to output. To further improve the perceptual quality of generated images, we modify the perceptual loss by extracting high-level features from discriminative network D, rather than pre-trained networks. Furthermore, we introduce a new training procedure based on the notion of self spinning to improve the final de-raining performance. Extensive experiments on the synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods.

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

A Deep Generative Model for Code Switched Text

10.24963/ijcai.2019/719 ◽

2019 ◽

Cited By ~ 1

Author(s):

Bidisha Samanta ◽

Sharmila Reddy ◽

Hussain Jagirdar ◽

Niloy Ganguly ◽

Soumen Chakrabarti

Keyword(s):

State Of The Art ◽

Generative Models ◽

Code Switching ◽

Language Models ◽

Data Intensive ◽

Adversarial Networks ◽

Latent Space ◽

Variational Autoencoder ◽

Multilingual Societies

Code-switching, the interleaving of two or more languages within a sentence or discourse is pervasive in multilingual societies. Accurate language models for code-switched text are critical for NLP tasks. State-of-the-art data-intensive neural language models are difficult to train well from scarce language-labeled code-switched text. A potential solution is to use deep generative models to synthesize large volumes of realistic code-switched text. Although generative adversarial networks and variational autoencoders can synthesize plausible monolingual text from continuous latent space, they cannot adequately address code-switched text, owing to their informal style and complex interplay between the constituent languages. We introduce VACS, a novel variational autoencoder architecture specifically tailored to code-switching phenomena. VACS encodes to and decodes from a two-level hierarchical representation, which models syntactic contextual signals in the lower level, and language switching signals in the upper layer. Sampling representations from the prior and decoding them produced well-formed, diverse code-switched sentences. Extensive experiments show that using synthetic code-switched text with natural monolingual data results in significant (33.06\%) drop in perplexity.

Automatic Target Recognition for Low Resolution Foliage Penetrating SAR Images Using CNNs and GANs

Remote Sensing ◽

10.3390/rs13040596 ◽

2021 ◽

Vol 13 (4) ◽

pp. 596

Author(s):

David Vint ◽

Matthew Anderson ◽

Yuhao Yang ◽

Christos Ilioudis ◽

Gaetano Di Caterina ◽

...

Keyword(s):

Target Recognition ◽

Automatic Target Recognition ◽

Low Resolution ◽

Sar Images ◽

Adversarial Networks ◽

Technological Advances ◽

Dataset Size ◽

Resolution Imaging ◽

In recent years, the technological advances leading to the production of high-resolution Synthetic Aperture Radar (SAR) images has enabled more and more effective target recognition capabilities. However, high spatial resolution is not always achievable, and, for some particular sensing modes, such as Foliage Penetrating Radars, low resolution imaging is often the only option. In this paper, the problem of automatic target recognition in Low Resolution Foliage Penetrating (FOPEN) SAR is addressed through the use of Convolutional Neural Networks (CNNs) able to extract both low and high level features of the imaged targets. Additionally, to address the issue of limited dataset size, Generative Adversarial Networks are used to enlarge the training set. Finally, a Receiver Operating Characteristic (ROC)-based post-classification decision approach is used to reduce classification errors and measure the capability of the classifier to provide a reliable output. The effectiveness of the proposed framework is demonstrated through the use of real SAR FOPEN data.

Towards Accuracy Enhancement of Age Group Classification Using Generative Adversarial Networks

Journal of Integrated Design and Process Science ◽

10.3233/jid-210019 ◽

2021 ◽

pp. 1-17

Author(s):

Khaled ELKarazle ◽

Valliappan Raman ◽

Patrick Then

Keyword(s):

Age Estimation ◽

Super Resolution ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Original Dataset ◽

Age Group Classification ◽

Facial Images

Age estimation models can be employed in many applications, including soft biometrics, content access control, targeted advertising, and many more. However, as some facial images are taken in unrestrained conditions, the quality relegates, which results in the loss of several essential ageing features. This study investigates how introducing a new layer of data processing based on a super-resolution generative adversarial network (SRGAN) model can influence the accuracy of age estimation by enhancing the quality of both the training and testing samples. Additionally, we introduce a novel convolutional neural network (CNN) classifier to distinguish between several age classes. We train one of our classifiers on a reconstructed version of the original dataset and compare its performance with an identical classifier trained on the original version of the same dataset. Our findings reveal that the classifier which trains on the reconstructed dataset produces better classification accuracy, opening the door for more research into building data-centric machine learning systems.

A deep learning approach to capture the essence of Candida albicans morphologies

10.1101/2021.06.10.445299 ◽

2021 ◽

Author(s):

Van Bettauer ◽

Anna CBP Costa ◽

Raha Parvizi Omran ◽

Samira Massahi ◽

Eftyhios Kirbizakis ◽

...

Keyword(s):

Deep Learning ◽

Learning Strategy ◽

Developmental Trajectories ◽

Learning Approach ◽

Adversarial Networks ◽

Latent Space ◽

Contrast Microscopy ◽

Opportunistic Human Pathogen ◽

Community Meeting

We present deep learning-based approaches for exploring the complex array of morphologies exhibited by the opportunistic human pathogen C. albicans. Our system entitled Candescence automatically detects C. albicans cells from Differential Image Contrast microscopy, and labels each detected cell with one of nine vegetative, mating-competent or filamentous morphologies. The software is based upon a fully convolutional one-stage object detector and exploits a novel cumulative curriculum-based learning strategy that stratifies our images by difficulty from simple vegetative forms to more complex filamentous architectures. Candescence achieves very good performance on this difficult learning set which has substantial intermixing between the predicted classes. To capture the essence of each C. albicans morphology, we develop models using generative adversarial networks and identify subcomponents of the latent space which control technical variables, developmental trajectories or morphological switches. We envision Candescence as a community meeting point for quantitative explorations of C. albicans morphology.

Fractional Wavelet-Based Generative Scattering Networks

Frontiers in Neurorobotics ◽

10.3389/fnbot.2021.752752 ◽

2021 ◽

Vol 15 ◽

Author(s):

Jiasong Wu ◽

Xiang Qiu ◽

Jing Zhang ◽

Fuzhi Wu ◽

Youyong Kong ◽

...

Keyword(s):

Dimensionality Reduction ◽

Reduction Method ◽

Gaussian White Noise ◽

Principal Component ◽

Experimental Results ◽

Image Generation ◽

Adversarial Networks ◽

Dimensionality Reduction Method

Generative adversarial networks and variational autoencoders (VAEs) provide impressive image generation from Gaussian white noise, but both are difficult to train, since they need a generator (or encoder) and a discriminator (or decoder) to be trained simultaneously, which can easily lead to unstable training. To solve or alleviate these synchronous training problems of generative adversarial networks (GANs) and VAEs, researchers recently proposed generative scattering networks (GSNs), which use wavelet scattering networks (ScatNets) as the encoder to obtain features (or ScatNet embeddings) and convolutional neural networks (CNNs) as the decoder to generate an image. The advantage of GSNs is that the parameters of ScatNets do not need to be learned, while the disadvantage of GSNs is that their ability to obtain representations of ScatNets is slightly weaker than that of CNNs. In addition, the dimensionality reduction method of principal component analysis (PCA) can easily lead to overfitting in the training of GSNs and, therefore, affect the quality of generated images in the testing process. To further improve the quality of generated images while keeping the advantages of GSNs, this study proposes generative fractional scattering networks (GFRSNs), which use more expressive fractional wavelet scattering networks (FrScatNets), instead of ScatNets as the encoder to obtain features (or FrScatNet embeddings) and use similar CNNs of GSNs as the decoder to generate an image. Additionally, this study develops a new dimensionality reduction method named feature-map fusion (FMF) instead of performing PCA to better retain the information of FrScatNets,; it also discusses the effect of image fusion on the quality of the generated image. The experimental results obtained on the CIFAR-10 and CelebA datasets show that the proposed GFRSNs can lead to better generated images than the original GSNs on testing datasets. The experimental results of the proposed GFRSNs with deep convolutional GAN (DCGAN), progressive GAN (PGAN), and CycleGAN are also given.

Encyclopedia of Organizational Knowledge, Administration, and Technology - Advances in Logistics, Operations, and Management Science ◽

Embodied Conversation

10.4018/978-1-7998-3473-1.ch076 ◽

2021 ◽

pp. 1091-1107

Author(s):

Andrej Zgank ◽

Izidor Mlakar ◽

Uros Berglez ◽

Danilo Zimsek ◽

Matej Borko ◽

...

Keyword(s):

Language Processing ◽

Automatic Speech Recognition ◽

Speech Synthesis ◽

Conversational Agents ◽

Embodied Conversational Agent ◽

Computer Interfaces ◽

Human Computer Interfaces ◽

Text To Speech Synthesis ◽

The chapter presents an overview of human-computer interfaces, which are a crucial element of an ambient intelligence solution. The focus is given to the embodied conversational agents, which are needed to communicate with users in a most natural way. Different input and output modalities, with supporting methods, to process the captured information (e.g., automatic speech recognition, gesture recognition, natural language processing, dialog processing, text to speech synthesis, etc.), have the crucial role to provide the high level of quality of experience to the user. As an example, usage of embodied conversational agent for e-Health domain is proposed.

Digital Transformation

The Data Imperative ◽

10.1093/oso/9780198840817.003.0001 ◽

2020 ◽

pp. 1-18

Author(s):

Henri Schildt

Keyword(s):

Language Processing ◽

Digital Transformation ◽

Digital Data ◽

Adversarial Networks ◽

Geographic Boundaries ◽

Technological Advances ◽

Data Flows ◽

Speed Up ◽

Systematic Shift

The introductory chapter to the book The Data Imperative examines how technological advances together with a new managerial mindset are driving digital transformation. While early business information systems were often self-contained and designed to solve specific problems, contemporary systems are highly interconnected and integrated. Corporations can use data flows to coordinate diverse processes and activities across organizational and geographic boundaries. The chapter explains how digital transformation involves a systematic shift from predominant reliance on human knowledge and skills to digital data flows and smart algorithms. Artificial intelligence techniques, such as generative adversarial networks and advanced natural language processing, and 5G wireless technologies create new opportunities to replace human routines with algorithmic processing. Data will continue to break down organizational silos, enable deeper collaboration across company boundaries, and speed up the development of new services.

Multi-Turn Chatbot Based on Query-Context Attentions and Dual Wasserstein Generative Adversarial Networks

Applied Sciences ◽

10.3390/app9183908 ◽

2019 ◽

Vol 9 (18) ◽

pp. 3908 ◽

Cited By ~ 3

Author(s):

Jintae Kim ◽

Shinhyeok Oh ◽

Oh-Woog Kwon ◽

Harksoo Kim

Keyword(s):

Performance Measures ◽

State Of The Art ◽

Attention Mechanism ◽

Training Method ◽

Adversarial Networks ◽

Proposed Model ◽

Previous State ◽

Vector Representations

To generate proper responses to user queries, multi-turn chatbot models should selectively consider dialogue histories. However, previous chatbot models have simply concatenated or averaged vector representations of all previous utterances without considering contextual importance. To mitigate this problem, we propose a multi-turn chatbot model in which previous utterances participate in response generation using different weights. The proposed model calculates the contextual importance of previous utterances by using an attention mechanism. In addition, we propose a training method that uses two types of Wasserstein generative adversarial networks to improve the quality of responses. In experiments with the DailyDialog dataset, the proposed model outperformed the previous state-of-the-art models based on various performance measures.