scholarly journals GAN-EM: GAN Based EM Learning Framework

2019 ◽  
Author(s):  
Wentian Zhao ◽  
Shaojie Wang ◽  
Zhihuai Xie ◽  
Jing Shi ◽  
Chenliang Xu
Keyword(s):  
Maximum Likelihood ◽  
Latent Variables ◽  
Latent Variable ◽  
Supervised Classification ◽  
Real Data ◽  
Gaussian Mixture ◽  
Generative Models ◽  
Image Clustering ◽  
Class Label ◽  
Learning Framework

Expectation maximization (EM) algorithm is to find maximum likelihood solution for models having latent variables. A typical example is Gaussian Mixture Model (GMM) which requires Gaussian assumption, however, natural images are highly non-Gaussian so that GMM cannot be applied to perform image clustering task on pixel space. To overcome such limitation, we propose a GAN based EM learning framework that can maximize the likelihood of images and estimate the latent variables. We call this model GAN-EM, which is a framework for image clustering, semi-supervised classification and dimensionality reduction. In M-step, we design a novel loss function for discriminator of GAN to perform maximum likelihood estimation (MLE) on data with soft class label assignments. Specifically, a conditional generator captures data distribution for K classes, and a discriminator tells whether a sample is real or fake for each class. Since our model is unsupervised, the class label of real data is regarded as latent variable, which is estimated by an additional network (E-net) in E-step. The proposed GAN-EM achieves state-of-the-art clustering and semi-supervised classification results on MNIST, SVHN and CelebA, as well as comparable quality of generated images to other recently developed generative models.

scholarly journals Learning Latent Variable Gaussian Graphical Model for Biomolecular Network with Low Sample Complexity

2016 ◽  
Vol 2016 ◽  
pp. 1-13 ◽  
Author(s):  
Yanbo Wang ◽  
Quan Liu ◽  
Bo Yuan
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Graphical Model ◽  
Real Data ◽  
Sample Complexity ◽  
Gaussian Graphical Model ◽  
Alternating Direction ◽  
Biomolecular Network ◽  
Ill Posed ◽  
Cancer Networks

Learning a Gaussian graphical model with latent variables is ill posed when there is insufficient sample complexity, thus having to be appropriately regularized. A common choice is convexl1plus nuclear norm to regularize the searching process. However, the best estimator performance is not always achieved with these additive convex regularizations, especially when the sample complexity is low. In this paper, we consider a concave additive regularization which does not require the strong irrepresentable condition. We use concave regularization to correct the intrinsic estimation biases from Lasso and nuclear penalty as well. We establish the proximity operators for our concave regularizations, respectively, which induces sparsity and low rankness. In addition, we extend our method to also allow the decomposition of fused structure-sparsity plus low rankness, providing a powerful tool for models with temporal information. Specifically, we develop a nontrivial modified alternating direction method of multipliers with at least local convergence. Finally, we use both synthetic and real data to validate the excellence of our method. In the application of reconstructing two-stage cancer networks, “the Warburg effect” can be revealed directly.

scholarly journals A Flow-Based Deep Latent Variable Model for Speech Spectrogram Modeling and Enhancement

2020 ◽  
Author(s):  
Aditya Arie Nugraha ◽  
Kouhei Sekiguchi ◽  
Kazuyoshi Yoshii
Keyword(s):  
Speech Enhancement ◽  
Latent Variables ◽  
Latent Variable ◽  
Generative Models ◽  
Latent Variable Model ◽  
Variable Model ◽  
Variational Autoencoder ◽  
Latent Representations ◽  
Low Dimensional ◽  
Better Than

This paper describes a deep latent variable model of speech power spectrograms and its application to semi-supervised speech enhancement with a deep speech prior. By integrating two major deep generative models, a variational autoencoder (VAE) and a normalizing flow (NF), in a mutually-beneficial manner, we formulate a flexible latent variable model called the NF-VAE that can extract low-dimensional latent representations from high-dimensional observations, akin to the VAE, and does not need to explicitly represent the distribution of the observations, akin to the NF. In this paper, we consider a variant of NF called the generative flow (GF a.k.a. Glow) and formulate a latent variable model called the GF-VAE. We experimentally show that the proposed GF-VAE is better than the standard VAE at capturing fine-structured harmonics of speech spectrograms, especially in the high-frequency range. A similar finding is also obtained when the GF-VAE and the VAE are used to generate speech spectrograms from latent variables randomly sampled from the standard Gaussian distribution. Lastly, when these models are used as speech priors for statistical multichannel speech enhancement, the GF-VAE outperforms the VAE and the GF.

Inpainting forgery detection using hybrid generative/discriminative approach based on bounded generalized Gaussian mixture model

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Abdullah Alharbi ◽  
Wajdi Alhakami ◽  
Sami Bourouis ◽  
Fatma Najar ◽  
Nizar Bouguila
Keyword(s):  
Gaussian Mixture Model ◽  
Mixture Model ◽  
Detection Method ◽  
Gaussian Mixture ◽  
Generative Models ◽  
Support Vector ◽  
Learning Framework ◽  
Probabilistic Support ◽  
Discriminative Approach ◽  
Vector Machines

We propose in this paper a novel reliable detection method to recognize forged inpainting images. Detecting potential forgeries and authenticating the content of digital images is extremely challenging and important for many applications. The proposed approach involves developing new probabilistic support vector machines (SVMs) kernels from a flexible generative statistical model named “bounded generalized Gaussian mixture model”. The developed learning framework has the advantage to combine properly the benefits of both discriminative and generative models and to include prior knowledge about the nature of data. It can effectively recognize if an image is a tampered one and also to identify both forged and authentic images. The obtained results confirmed that the developed framework has good performance under numerous inpainted images.

scholarly journals Generalized ridge estimators adapted in structural equation models

2020 ◽  
Vol 43 ◽  
pp. e49929
Author(s):  
Gislene Araujo Pereira ◽  
Mariana Resende ◽  
Marcelo Ângelo Cirillo
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Structural Equation ◽  
Structural Equation Models ◽  
Real Data ◽  
Alternative Methods ◽  
Estimation Methods ◽  
Specification Error ◽  
Maximum Likelihood Methods ◽  
Pls Method

Multicollinearity is detected via regression models, where independent variables are strongly correlated. Since they entail linear relations between observed or latent variables, the structural equation models (SEM) are subject to the multicollinearity effect, whose numerous consequences include the singularity between the inverse matrices used in estimation methods. Given to this behavior, it is natural to understand that the suitability of these estimators to structural equation models show the same features, either in the simulation results that validate the estimators in different multicollinearity degrees, or in their application to real data. Due to the multicollinearity overview arose from the fact that the matrices inversion is impracticable, the usage of numerical procedures demanded by the maximum likelihood methods leads to numerical singularity problems. An alternative could be the use of the Partial Least Squares (PLS) method, however, it is demanded that the observed variables are built by assuming a positive correlation with the latent variable. Thus, theoretically, it is expected that the load signals are positive, however, there are no restrictions to these signals in the algorithms used in the PLS method. This fact implies in corrective areas, such as the observed variables removal or new formulations of the theoretical model. In view of this problem, this paper aimed to propose adaptations of six generalized ridge estimators as alternative methods to estimate SEM parameters. The conclusion is that the evaluated estimators presented the same performance in terms of accuracy, precision while considering the scenarios represented by model without specification error and model with specification error, different levels of multicollinearity and sample sizes.

scholarly journals On the curved exponential family in the Stochatic Approximation Expectation Maximization Algorithm

2021 ◽  
Author(s):  
Vianney Debavelaere ◽  
Stéphanie Allassonnière
Keyword(s):  
Maximum Likelihood ◽  
Expectation Maximization ◽  
Latent Variables ◽  
Latent Variable ◽  
Exponential Family ◽  
Expectation Maximization Algorithm ◽  
Computation Time ◽  
Initial Model ◽  
Joint Likelihood ◽  
Curved Exponential Family

The Expectation-Maximization Algorithm (EM) is a widely used method allowing to estimate the maximum likelihood of  models involving latent variables. When the Expectation step cannot be computed easily, one can use stochastic versions of the EM such as the Stochastic Approximation EM. This algorithm, however, has the drawback to require the joint likelihood to belong to the curved exponential family. To overcome this problem, \cite{kuhn2005maximum} introduced a rewriting of the model which ``exponentializes'' it by considering the parameter as an additional latent variable following a Normal distribution centered on the newly defined parameters and with fixed variance. The likelihood of this new exponentialized model now belongs to the curved exponential family. Although often used, there is no guarantee that the estimated mean is close to the  maximum likelihood estimate of the initial model. In this paper, we quantify the error done in this estimation while considering the exponentialized model instead of the initial one. By verifying those results on an example, we see that a trade-off must be made between the speed of convergence and the tolerated error. Finally, we propose a new algorithm allowing a better estimation of the parameter in a reasonable computation time to reduce the bias.

scholarly journals A Flow-Based Deep Latent Variable Model for Speech Spectrogram Modeling and Enhancement

2020 ◽  
Author(s):  
Aditya Arie Nugraha ◽  
Kouhei Sekiguchi ◽  
Kazuyoshi Yoshii
Keyword(s):  
Speech Enhancement ◽  
Latent Variables ◽  
Latent Variable ◽  
Generative Models ◽  
Latent Variable Model ◽  
Variable Model ◽  
Variational Autoencoder ◽  
Latent Representations ◽  
Low Dimensional ◽  
Better Than

This paper describes a deep latent variable model of speech power spectrograms and its application to semi-supervised speech enhancement with a deep speech prior. By integrating two major deep generative models, a variational autoencoder (VAE) and a normalizing flow (NF), in a mutually-beneficial manner, we formulate a flexible latent variable model called the NF-VAE that can extract low-dimensional latent representations from high-dimensional observations, akin to the VAE, and does not need to explicitly represent the distribution of the observations, akin to the NF. In this paper, we consider a variant of NF called the generative flow (GF a.k.a. Glow) and formulate a latent variable model called the GF-VAE. We experimentally show that the proposed GF-VAE is better than the standard VAE at capturing fine-structured harmonics of speech spectrograms, especially in the high-frequency range. A similar finding is also obtained when the GF-VAE and the VAE are used to generate speech spectrograms from latent variables randomly sampled from the standard Gaussian distribution. Lastly, when these models are used as speech priors for statistical multichannel speech enhancement, the GF-VAE outperforms the VAE and the GF.

Maximum Likelihood Analysis of a Two-Level Nonlinear Structural Equation Model With Fixed Covariates

2005 ◽  
Vol 30 (1) ◽  
pp. 1-26 ◽  
Author(s):  
Sik-Yum Lee ◽  
Xin-Yuan Song
Keyword(s):  
Maximum Likelihood ◽  
Structural Equation Model ◽  
Latent Variables ◽  
Structural Equation ◽  
Model Comparison ◽  
Real Data ◽  
Equation Model ◽  
Data Set ◽  
Nonlinear Structural Equation ◽  
Monte Carlo Em

In this article, a maximum likelihood (ML) approach for analyzing a rather general two-level structural equation model is developed for hierarchically structured data that are very common in educational and/or behavioral research. The proposed two-level model can accommodate nonlinear causal relations among latent variables as well as effects of fixed covariate in its various components. Methods for computing the ML estimates, and the Bayesian information criterion (BIC) for model comparison are established on the basis of powerful tools in statistical computing such as the Monte Carlo EM algorithm, Gibbs sampler, Metropolis–Hastings algorithm, conditional maximization, bridge sampling, and path sampling. The newly developed procedures are illustrated by results obtained from a simulation study and analysis of a real data set in education.

scholarly journals Classification with Rejection: Scaling Generative Classifiers with Supervised Deep Infomax

2020 ◽  
Author(s):  
Xin Wang ◽  
Siu Ming Yiu
Keyword(s):  
Representation Learning ◽  
Generative Models ◽  
Probabilistic Constraints ◽  
Class Label ◽  
Learning Framework ◽  
Level Data ◽  
Data Representations ◽  
Comparable Performance ◽  
Adversarial Examples ◽  
High Level

Deep Infomax (DIM) is an unsupervised representation learning framework by maximizing the mutual information between the inputs and the outputs of an encoder, while probabilistic constraints are imposed on the outputs. In this paper, we propose Supervised Deep InfoMax (SDIM), which introduces supervised probabilistic constraints to the encoder outputs. The supervised probabilistic constraints are equivalent to a generative classifier on high-level data representations, where class conditional log-likelihoods of samples can be evaluated. Unlike other works building generative classifiers with conditional generative models, SDIMs scale on complex datasets, and can achieve comparable performance with discriminative counterparts. With SDIM, we could perform classification with rejection. Instead of always reporting a class label, SDIM only makes predictions when test samples' largest class conditional surpass some pre-chosen thresholds, otherwise they will be deemed as out of the data distributions, and be rejected. Our experiments show that SDIM with rejection policy can effectively reject illegal inputs, including adversarial examples and out-of-distribution samples.

scholarly journals An Improved Flexible Partial Histogram Bayes Learning Algorithm

2018 ◽  
Vol 11 (3) ◽  
pp. 975 ◽  
Author(s):  
Haider O. Lawend ◽  
Anuar Muad ◽  
Aini Hussain
Keyword(s):  
Supervised Classification ◽  
Nearest Neighbor ◽  
Learning Algorithm ◽  
Synthetic Data ◽  
Real Data ◽  
Gaussian Mixture ◽  
Classification Technique ◽  
Classification Tasks ◽  
Speed And Accuracy ◽  
Standard Learning

<em>This paper presents a proposed supervised classification technique namely flexible partial histogram Bayes (fPHBayes) learning algorithm. In our previous work, partial histogram Bayes (PHBayes) learning algorithm showed some advantages in the aspects of speed and accuracy in classification tasks. However, its accuracy declines when dealing with small number of instances or when the class feature distributes in wide area. In this work, the proposed fPHBayes solves these limitations in order to increase the classification accuracy. fPHBayes was analyzed and compared with PHBayes and other standard learning algorithms like first nearest neighbor, nearest subclass mean, nearest class mean, naive Bayes and Gaussian mixture model classifier. The experiments were performed using both real data and synthetic data considering different number of instances and different variances of Gaussians. The results showed that fPHBayes is more accurate and flexible to deal with different number of instances and different variances of Gaussians as compared to PHBayes.</em>

scholarly journals Disentangling Information in Artificial Images of Plant Seedlings Using Semi-Supervised GAN

2019 ◽  
Vol 11 (22) ◽  
pp. 2671
Author(s):  
Simon Leminen Madsen ◽  
Anders Krogh Mortensen ◽  
Rasmus Nyholm Jørgensen ◽  
Henrik Karstoft
Keyword(s):  
Latent Variables ◽  
Visual Recognition ◽  
Assessment Tool ◽  
Real Data ◽  
Generative Models ◽  
Training Data ◽  
Classification Model ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Plant Seedlings

Lack of annotated data for training of deep learning systems is a challenge for many visual recognition tasks. This is especially true for domain-specific applications, such as plant detection and recognition, where the annotation process can be both time-consuming and error-prone. Generative models can be used to alleviate this issue by producing artificial data that mimic properties of real data. This work presents a semi-supervised generative adversarial network (GAN) model to produce artificial samples of plant seedlings. By applying the semi-supervised approach, we are able to produce visually distinct samples for nine unique plant species using a single GAN model, while still maintaining a relatively high visual variance in the produced samples for each species. Additionally, we are able to control the appearance of the generated samples with respect to rotation and size through a set of latent variables, despite these not being annotated features in the training data. The generated samples resemble the intended species with an average recognition accuracy of ∼64.3%, evaluated using an external state-of-the-art plant seedling classification model. Additionally, we explore the potential of using the GAN model’s discriminator as a quality assessment tool to remove poor representations of plant seedlings from the artificial samples.

Sign in / Sign up

Export Citation Format

Share Document