scholarly journals DR-VIDAL - Doubly Robust Variational Information-theoretic Deep Adversarial Learning for Counterfactual Prediction and Treatment Effect Estimation

2021 ◽  
Author(s):  
SHANTANU GHOSH ◽  
Zheng Feng ◽  
Jiang Bian ◽  
Kevin Butler ◽  
Mattia Prosperi
Keyword(s):  
Latent Variables ◽  
Unbiased Estimation ◽  
Joint Models ◽  
Generative Adversarial Network ◽  
Adversarial Learning ◽  
Information Theoretic ◽  
Adversarial Network ◽  
Causal Assumptions ◽  
Real World Datasets ◽  
Doubly Robust

Abstract Determining causal effects of interventions onto outcomes from observational (non-randomized) data, e.g., treatment repurposing using electronic health records, is challenging due to underlying bias. Causal deep learning has improved over traditional techniques for estimating individualized treatment effects. We present the Doubly Robust Variational Information-theoretic Deep Adversarial Learning (DR-VIDAL), a novel generative framework that combines two joint models of treatment and outcome, ensuring an unbiased estimation even when one of the two is misspecified. DR-VIDAL uses a variational autoencoder (VAE) to factorize confounders into latent variables according to causal assumptions; then, an information-theoretic generative adversarial network (Info-GAN) is used to generate counterfactuals; finally, a doubly robust block incorporates propensity matching/weighting into predictions. On synthetic and real-world datasets, DR-VIDAL achieves better performance than other non-generative and generative methods. In conclusion, DR-VIDAL uniquely fuses causal assumptions, VAE, Info-GAN, and doubly robustness into a comprehensive, performant framework. Code is available at: https://bitbucket.org/goingdeep2406/dr-vidal/src/master/

scholarly journals Information-Based Boundary Equilibrium Generative Adversarial Networks with Interpretable Representation Learning

2018 ◽  
Vol 2018 ◽  
pp. 1-14 ◽  
Author(s):  
Junghoon Hah ◽  
Woojin Lee ◽  
Jaewook Lee ◽  
Saerom Park
Keyword(s):  
Latent Variables ◽  
Representation Learning ◽  
Wasserstein Distance ◽  
Generative Adversarial Networks ◽  
Small Subset ◽  
Generative Adversarial Network ◽  
Information Theoretic ◽  
Adversarial Network ◽  
Boundary Equilibrium ◽  
Input Variables

This paper describes a new image generation algorithm based on generative adversarial network. With an information-theoretic extension to the autoencoder-based discriminator, this new algorithm is able to learn interpretable representations from the input images. Our model not only adversarially minimizes the Wasserstein distance-based losses of the discriminator and generator but also maximizes the mutual information between small subset of the latent variables and the observation. We also train our model with proportional control theory to keep the equilibrium between the discriminator and the generator balanced, and as a result, our generative adversarial network can mitigate the convergence problem. Through the experiments on real images, we validate our proposed method, which can manipulate the generated images as desired by controlling the latent codes of input variables. In addition, the visual qualities of produced images are effectively maintained, and the model can stably converge to the equilibrium. However, our model has a difficulty in learning disentangling factors because our model does not regularize the independence between the interpretable factors. Therefore, in the future, we will develop a generative model that can learn disentangling factors.

scholarly journals IntroVNMT: An Introspective Model for Variational Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8830-8837
Author(s):  
Xin Sheng ◽  
Linli Xu ◽  
Junliang Guo ◽  
Jingchang Liu ◽  
Ruoyu Zhao ◽  
...  
Keyword(s):  
Machine Translation ◽  
Latent Variables ◽  
Image Synthesis ◽  
Target Language ◽  
Generative Adversarial Network ◽  
Neural Machine Translation ◽  
Adversarial Network ◽  
Proposed Model ◽  
Model Training ◽  
High Level

We propose a novel introspective model for variational neural machine translation (IntroVNMT) in this paper, inspired by the recent successful application of introspective variational autoencoder (IntroVAE) in high quality image synthesis. Different from the vanilla variational NMT model, IntroVNMT is capable of improving itself introspectively by evaluating the quality of the generated target sentences according to the high-level latent variables of the real and generated target sentences. As a consequence of introspective training, the proposed model is able to discriminate between the generated and real sentences of the target language via the latent variables generated by the encoder of the model. In this way, IntroVNMT is able to generate more realistic target sentences in practice. In the meantime, IntroVNMT inherits the advantages of the variational autoencoders (VAEs), and the model training process is more stable than the generative adversarial network (GAN) based models. Experimental results on different translation tasks demonstrate that the proposed model can achieve significant improvements over the vanilla variational NMT model.

scholarly journals JANE: Jointly Adversarial Network Embedding

2020 ◽  
Author(s):  
Liang Yang ◽  
Yuexue Wang ◽  
Junhua Gu ◽  
Chuan Wang ◽  
Xiaochun Cao ◽  
...  
Keyword(s):  
Link Prediction ◽  
Real Data ◽  
Semantic Space ◽  
Network Embedding ◽  
Generative Adversarial Network ◽  
Adversarial Learning ◽  
Adversarial Network ◽  
Node Clustering ◽  
Topology Information ◽  
Embedding Methods

Motivated by the capability of Generative Adversarial Network on exploring the latent semantic space and capturing semantic variations in the data distribution, adversarial learning has been adopted in network embedding to improve the robustness. However, this important ability is lost in existing adversarially regularized network embedding methods, because their embedding results are directly compared to the samples drawn from perturbation (Gaussian) distribution without any rectification from real data. To overcome this vital issue, a novel Joint Adversarial Network Embedding (JANE) framework is proposed to jointly distinguish the real and fake combinations of the embeddings, topology information and node features. JANE contains three pluggable components, Embedding module, Generator module and Discriminator module. The overall objective function of JANE is defined in a min-max form, which can be optimized via alternating stochastic gradient. Extensive experiments demonstrate the remarkable superiority of the proposed JANE on link prediction (3% gains in both AUC and AP) and node clustering (5% gain in F1 score).

scholarly journals Data Augmentation for Electricity Theft Detection Using Conditional Variational Auto-Encoder

Energies ◽  
2020 ◽  
Vol 13 (17) ◽  
pp. 4291
Author(s):  
Xuejiao Gong ◽  
Bo Tang ◽  
Ruijin Zhu ◽  
Wenlong Liao ◽  
Like Song
Keyword(s):  
Latent Variables ◽  
Data Augmentation ◽  
Sampling Technique ◽  
Smart Meters ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Electricity Theft ◽  
Adversarial Network ◽  
Low Dimensional ◽  
Power Curves

Due to the strong concealment of electricity theft and the limitation of inspection resources, the number of power theft samples mastered by the power department is insufficient, which limits the accuracy of power theft detection. Therefore, a data augmentation method for electricity theft detection based on the conditional variational auto-encoder (CVAE) is proposed. Firstly, the stealing power curves are mapped into low dimensional latent variables by using the encoder composed of convolutional layers, and the new stealing power curves are reconstructed by the decoder composed of deconvolutional layers. Then, five typical attack models are proposed, and the convolutional neural network is constructed as a classifier according to the data characteristics of stealing power curves. Finally, the effectiveness and adaptability of the proposed method is verified by a smart meters’ data set from London. The simulation results show that the CVAE can take into account the shapes and distribution characteristics of samples at the same time, and the generated stealing power curves have the best effect on the performance improvement of the classifier than the traditional augmentation methods such as the random oversampling method, synthetic minority over-sampling technique, and conditional generative adversarial network. Moreover, it is suitable for different classifiers.

scholarly journals Posterior-GAN: Towards Informative and Coherent Response Generation with Posterior Generative Adversarial Network

2020 ◽  
Vol 34 (05) ◽  
pp. 7708-7715
Author(s):  
Shaoxiong Feng ◽  
Hongshen Chen ◽  
Kan Li ◽  
Dawei Yin
Keyword(s):  
Likelihood Estimation ◽  
Model Learning ◽  
Generative Adversarial Network ◽  
Adversarial Learning ◽  
Loosely Coupled ◽  
Learning Framework ◽  
Adversarial Network ◽  
The Future ◽  
Quality Response ◽  
The Given

Neural conversational models learn to generate responses by taking into account the dialog history. These models are typically optimized over the query-response pairs with a maximum likelihood estimation objective. However, the query-response tuples are naturally loosely coupled, and there exist multiple responses that can respond to a given query, which leads the conversational model learning burdensome. Besides, the general dull response problem is even worsened when the model is confronted with meaningless response training instances. Intuitively, a high-quality response not only responds to the given query but also links up to the future conversations, in this paper, we leverage the query-response-future turn triples to induce the generated responses that consider both the given context and the future conversations. To facilitate the modeling of these triples, we further propose a novel encoder-decoder based generative adversarial learning framework, Posterior Generative Adversarial Network (Posterior-GAN), which consists of a forward and a backward generative discriminator to cooperatively encourage the generated response to be informative and coherent by two complementary assessment perspectives. Experimental results demonstrate that our method effectively boosts the informativeness and coherence of the generated response on both automatic and human evaluation, which verifies the advantages of considering two assessment perspectives.

scholarly journals Multifeature Named Entity Recognition in Information Security Based on Adversarial Learning

2019 ◽  
Vol 2019 ◽  
pp. 1-9
Author(s):  
Han Zhang ◽  
Yuanbo Guo ◽  
Tao Li
Keyword(s):  
Information Security ◽  
Large Scale ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Generative Adversarial Network ◽  
Adversarial Learning ◽  
New Approach ◽  
Named Entity ◽  
Adversarial Network ◽  
Security Research

In order to obtain high quality and large-scale labelled data for information security research, we propose a new approach that combines a generative adversarial network with the BiLSTM-Attention-CRF model to obtain labelled data from crowd annotations. We use the generative adversarial network to find common features in crowd annotations and then consider them in conjunction with the domain dictionary feature and sentence dependency feature as additional features to be introduced into the BiLSTM-Attention-CRF model, which is then used to carry out named entity recognition in crowdsourcing. Finally, we create a dataset to evaluate our models using information security data. The experimental results show that our model has better performance than the other baseline models.

scholarly journals Speech Enhancement Using Generative Adversarial Network by Distilling Knowledge from Statistical Method

2019 ◽  
Vol 9 (16) ◽  
pp. 3396 ◽  
Author(s):  
Jianfeng Wu ◽  
Yongzhu Hua ◽  
Shengying Yang ◽  
Hongshuai Qin ◽  
Huibin Qin
Keyword(s):  
Neural Network ◽  
Statistical Method ◽  
Speech Enhancement ◽  
Data Sets ◽  
Generative Adversarial Network ◽  
Adversarial Learning ◽  
Noisy Speech ◽  
Adversarial Network ◽  
Knowledge Distillation ◽  
Enhancement Algorithm

This paper presents a new deep neural network (DNN)-based speech enhancement algorithm by integrating the distilled knowledge from the traditional statistical-based method. Unlike the other DNN-based methods, which usually train many different models on the same data and then average their predictions, or use a large number of noise types to enlarge the simulated noisy speech, the proposed method does not train a whole ensemble of models and does not require a mass of simulated noisy speech. It first trains a discriminator network and a generator network simultaneously using the adversarial learning method. Then, the discriminator network and generator network are re-trained by distilling knowledge from the statistical method, which is inspired by the knowledge distillation in a neural network. Finally, the generator network is fine-tuned using real noisy speech. Experiments on CHiME4 data sets demonstrate that the proposed method achieves a more robust performance than the compared DNN-based method in terms of perceptual speech quality.

scholarly journals Generative Adversarial Phonology: Modeling unsupervised allophonic learning with neural networks

2019 ◽  
Author(s):  
Gasper Begus
Keyword(s):  
Neural Networks ◽  
Latent Variables ◽  
Network Architecture ◽  
Speech Disorders ◽  
Levels Of Abstraction ◽  
Generative Adversarial Network ◽  
Internal Representations ◽  
Adversarial Network ◽  
Training Interval ◽  
Speech Data

Training deep neural networks on well-understood dependencies in speech data can provide new insights into how they learn internal representations. This paper argues that acquisition of speech can be modeled as a dependency between random space and generated speech data in the Generative Adversarial Network architecture and proposes a methodology to uncover the network's internal representations that correspond to phonetic and phonological properties. The Generative Adversarial architecture is uniquely appropriate for modeling phonetic and phonological learning because the network is trained on unannotated raw acoustic data and learning is unsupervised without any language-specific assumptions or pre-assumed levels of abstraction. A Generative Adversarial Network was trained on an allophonic distribution in English, in which voiceless stops surface as aspirated word-initially before stressed vowels, except if preceded by a sibilant [s]. The network successfully learns the allophonic alternation: the network's generated speech signal contains the conditional distribution of aspiration duration. The paper proposes a technique for establishing the network's internal representations that identifies latent variables that correspond to, for example, presence of [s] and its spectral properties. By manipulating these variables, we actively control the presence of [s] and its frication amplitude in the generated outputs. This suggests that the network learns to use latent variables as an approximation of phonetic and phonological representations. Crucially, we observe that the dependencies learned in training extend beyond the training interval, which allows for additional exploration of learning representations. The paper also discusses how the network's architecture and innovative outputs resemble and differ from linguistic behavior in language acquisition, speech disorders, and speech errors, and how well-understood dependencies in speech data can help us interpret how neural networks learn their representations.

scholarly journals Single Image De-Raining Using Spinning Detail Perceptual Generative Adversarial Networks

2020 ◽  
Vol 24 (7) ◽  
pp. 811-819
Author(s):  
Kaizheng Chen ◽  
◽  
Yaping Dai ◽  
Zhiyang Jia ◽  
Kaoru Hirota
Keyword(s):  
Generative Adversarial Networks ◽  
Perceptual Quality ◽  
Training Procedure ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Real World Datasets ◽  
High Level

In this paper, Spinning Detail Perceptual Generative Adversarial Networks (SDP-GAN) is proposed for single image de-raining. The proposed method adopts the Generative Adversarial Network (GAN) framework and consists of two following networks: the rain streaks generative network G and the discriminative network D. To reduce the background interference, we propose a rain streaks generative network which not only focuses on the high frequency detail map of rainy image, but also directly reduces the mapping range from input to output. To further improve the perceptual quality of generated images, we modify the perceptual loss by extracting high-level features from discriminative network D, rather than pre-trained networks. Furthermore, we introduce a new training procedure based on the notion of self spinning to improve the final de-raining performance. Extensive experiments on the synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document