Bayesian analysis of two-part nonlinear latent variable model: Semiparametric method

2021 ◽  
pp. 1471082X2110592
Author(s):  
Jian-Wei Gou ◽  
Ye-Mao Xia ◽  
De-Peng Jiang
Keyword(s):  
Factor Analysis ◽  
Latent Variables ◽  
Latent Variable ◽  
Latent Variable Model ◽  
Continuous Data ◽  
Model Assessment ◽  
Continuous Variables ◽  
Continuous Part ◽  
Variable Model ◽  
Log Normal

Two-part model (TPM) is a widely appreciated statistical method for analyzing semi-continuous data. Semi-continuous data can be viewed as arising from two distinct stochastic processes: one governs the occurrence or binary part of data and the other determines the intensity or continuous part. In the regression setting with the semi-continuous outcome as functions of covariates, the binary part is commonly modelled via logistic regression and the continuous component via a log-normal model. The conventional TPM, still imposes assumptions such as log-normal distribution of the continuous part, with no unobserved heterogeneity among the response, and no collinearity among covariates, which are quite often unrealistic in practical applications. In this article, we develop a two-part nonlinear latent variable model (TPNLVM) with mixed multiple semi-continuous and continuous variables. The semi-continuous variables are treated as indicators of the latent factor analysis along with other manifest variables. This reduces the dimensionality of the regression model and alleviates the potential multicollinearity problems. Our TPNLVM can accommodate the nonlinear relationships among latent variables extracted from the factor analysis. To downweight the influence of distribution deviations and extreme observations, we develop a Bayesian semiparametric analysis procedure. The conventional parametric assumptions on the related distributions are relaxed and the Dirichlet process (DP) prior is used to improve model fitting. By taking advantage of the discreteness of DP, our method is effective in capturing the heterogeneity underlying population. Within the Bayesian paradigm, posterior inferences including parameters estimates and model assessment are carried out through Markov Chains Monte Carlo (MCMC) sampling method. To facilitate posterior sampling, we adapt the Polya-Gamma stochastic representation for the logistic model. Using simulation studies, we examine properties and merits of our proposed methods and illustrate our approach by evaluating the effect of treatment on cocaine use and examining whether the treatment effect is moderated by psychiatric problems.

scholarly journals A Flow-Based Deep Latent Variable Model for Speech Spectrogram Modeling and Enhancement

2020 ◽  
Author(s):  
Aditya Arie Nugraha ◽  
Kouhei Sekiguchi ◽  
Kazuyoshi Yoshii
Keyword(s):  
Speech Enhancement ◽  
Latent Variables ◽  
Latent Variable ◽  
Generative Models ◽  
Latent Variable Model ◽  
Variable Model ◽  
Variational Autoencoder ◽  
Latent Representations ◽  
Low Dimensional ◽  
Better Than

This paper describes a deep latent variable model of speech power spectrograms and its application to semi-supervised speech enhancement with a deep speech prior. By integrating two major deep generative models, a variational autoencoder (VAE) and a normalizing flow (NF), in a mutually-beneficial manner, we formulate a flexible latent variable model called the NF-VAE that can extract low-dimensional latent representations from high-dimensional observations, akin to the VAE, and does not need to explicitly represent the distribution of the observations, akin to the NF. In this paper, we consider a variant of NF called the generative flow (GF a.k.a. Glow) and formulate a latent variable model called the GF-VAE. We experimentally show that the proposed GF-VAE is better than the standard VAE at capturing fine-structured harmonics of speech spectrograms, especially in the high-frequency range. A similar finding is also obtained when the GF-VAE and the VAE are used to generate speech spectrograms from latent variables randomly sampled from the standard Gaussian distribution. Lastly, when these models are used as speech priors for statistical multichannel speech enhancement, the GF-VAE outperforms the VAE and the GF.

scholarly journals Choosing the number of factors in independent factor analysis model

2005 ◽  
Vol 2 (2) ◽  
Author(s):  
Cinzia Viroli
Keyword(s):  
Factor Analysis ◽  
Latent Variables ◽  
Latent Variable ◽  
Information Criteria ◽  
Ratio Test ◽  
Independent Factor ◽  
Analysis Model ◽  
Variable Model ◽  
Factor Analysis Model ◽  
Independent Factor Analysis

Independent Factor Analysis (IFA) has recently been proposed in the signal processing literature as a way to model a set of observed variables through linear combinations of hidden independent ones plus a noise term. Despite the peculiarity of its origin the method can be framed within the latent variable model domain and some parallels with the ordinary factor analysis can be drawn. If no prior information on the latent structure is available a relevant issue concerns the correct specification of the model. In this work some methods to detect the number of significant latent variables are investigated. Moreover, since the method defines a probability density function for the latent variables by mixtures of gaussians, the correct number of mixture components must also be determined. This issue will be treated according to two main approaches. The first one amounts to carry out a likelihood ratio test. The other one is based on a penalized form of the likelihood, that leads to the so called information criteria. Some simulations and empirical results on real data sets are finally presented.

scholarly journals A tractable latent variable model for nonlinear dimensionality reduction

2020 ◽  
Vol 117 (27) ◽  
pp. 15403-15408
Author(s):  
Lawrence K. Saul
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Linear Equations ◽  
Three Dimensional ◽  
Coarse Graining ◽  
Graph Laplacian ◽  
Latent Variable Model ◽  
Nonlinear Dimensionality Reduction ◽  
Variable Model ◽  
Low Dimensional

We propose a latent variable model to discover faithful low-dimensional representations of high-dimensional data. The model computes a low-dimensional embedding that aims to preserve neighborhood relationships encoded by a sparse graph. The model both leverages and extends current leading approaches to this problem. Like t-distributed Stochastic Neighborhood Embedding, the model can produce two- and three-dimensional embeddings for visualization, but it can also learn higher-dimensional embeddings for other uses. Like LargeVis and Uniform Manifold Approximation and Projection, the model produces embeddings by balancing two goals—pulling nearby examples closer together and pushing distant examples further apart. Unlike these approaches, however, the latent variables in our model provide additional structure that can be exploited for learning. We derive an Expectation–Maximization procedure with closed-form updates that monotonically improve the model’s likelihood: In this procedure, embeddings are iteratively adapted by solving sparse, diagonally dominant systems of linear equations that arise from a discrete graph Laplacian. For large problems, we also develop an approximate coarse-graining procedure that avoids the need for negative sampling of nonadjacent nodes in the graph. We demonstrate the model’s effectiveness on datasets of images and text.

Comparison of partial least square algorithms in hierarchical latent variable model with missing data

SIMULATION ◽  
2020 ◽  
Vol 96 (10) ◽  
pp. 825-839
Author(s):  
Hao Cheng
Keyword(s):  
Missing Data ◽  
Quantile Regression ◽  
Latent Variables ◽  
Latent Variable ◽  
Partial Least Square ◽  
Least Square ◽  
Latent Variable Model ◽  
Data Handling ◽  
Variable Model ◽  
Handling Methods

Missing data is almost inevitable for various reasons in many applications. For hierarchical latent variable models, there usually exist two kinds of missing data problems. One is manifest variables with incomplete observations, the other is latent variables which cannot be observed directly. Missing data in manifest variables can be handled by different methods. For latent variables, there exist several kinds of partial least square (PLS) algorithms which have been widely used to estimate the value of latent variables. In this paper, we not only combine traditional linear regression type PLS algorithms with missing data handling methods, but also introduce quantile regression to improve the performances of PLS algorithms when the relationships among manifest and latent variables are not fixed according to the explored quantile of interest. Thus, we can get the overall view of variables’ relationships at different levels. The main challenges lie in how to introduce quantile regression in PLS algorithms correctly and how well the PLS algorithms perform when missing manifest variables occur. By simulation studies, we compare all the PLS algorithms with missing data handling methods in different settings, and finally build a business sophistication hierarchical latent variable model based on real data.

scholarly journals A Flow-Based Deep Latent Variable Model for Speech Spectrogram Modeling and Enhancement

2020 ◽  
Author(s):  
Aditya Arie Nugraha ◽  
Kouhei Sekiguchi ◽  
Kazuyoshi Yoshii
Keyword(s):  
Speech Enhancement ◽  
Latent Variables ◽  
Latent Variable ◽  
Generative Models ◽  
Latent Variable Model ◽  
Variable Model ◽  
Variational Autoencoder ◽  
Latent Representations ◽  
Low Dimensional ◽  
Better Than

This paper describes a deep latent variable model of speech power spectrograms and its application to semi-supervised speech enhancement with a deep speech prior. By integrating two major deep generative models, a variational autoencoder (VAE) and a normalizing flow (NF), in a mutually-beneficial manner, we formulate a flexible latent variable model called the NF-VAE that can extract low-dimensional latent representations from high-dimensional observations, akin to the VAE, and does not need to explicitly represent the distribution of the observations, akin to the NF. In this paper, we consider a variant of NF called the generative flow (GF a.k.a. Glow) and formulate a latent variable model called the GF-VAE. We experimentally show that the proposed GF-VAE is better than the standard VAE at capturing fine-structured harmonics of speech spectrograms, especially in the high-frequency range. A similar finding is also obtained when the GF-VAE and the VAE are used to generate speech spectrograms from latent variables randomly sampled from the standard Gaussian distribution. Lastly, when these models are used as speech priors for statistical multichannel speech enhancement, the GF-VAE outperforms the VAE and the GF.

scholarly journals Revealing unobserved factors underlying cortical activity using a rectified latent variable model applied to neural population recordings

2016 ◽  
Author(s):  
Matthew R. Whiteway ◽  
Daniel A. Butts
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Large Population ◽  
Component Analysis ◽  
Latent Variable Model ◽  
Neural Population ◽  
Model Parameters ◽  
Variable Model ◽  
Sensory Stimuli ◽  
External Stimuli

ABSTRACTThe activity of sensory cortical neurons is not only driven by external stimuli, but is also shaped by other sources of input to the cortex. Unlike external stimuli these other sources of input are challenging to experimentally control or even observe, and as a result contribute to variability of neuronal responses to sensory stimuli. However, such sources of input are likely not “noise”, and likely play an integral role in sensory cortex function. Here, we introduce the rectified latent variable model (RLVM) in order to identify these sources of input using simultaneously recorded cortical neuron populations. The RLVM is novel in that it employs non-negative (rectified) latent variables, and is able to be much less restrictive in the mathematical constraints on solutions due to the use an autoencoder neural network to initialize model parameters. We show the RLVM outperforms principal component analysis, factor analysis and independent component analysis across a variety of measures using simulated data. We then apply this model to the 2-photon imaging of hundreds of simultaneously recorded neurons in mouse primary somatosensory cortex during a tactile discrimination task. Across many experiments, the RLVM identifies latent variables related to both the tactile stimulation as well as non-stimulus aspects of the behavioral task, with a majority of activity explained by the latter. These results suggest that properly identifying such latent variables is necessary for a full understanding of sensory cortical function, and demonstrates novel methods for leveraging large population recordings to this end.

scholarly journals Revealing unobserved factors underlying cortical activity with a rectified latent variable model applied to neural population recordings

2017 ◽  
Vol 117 (3) ◽  
pp. 919-936 ◽  
Author(s):  
Matthew R. Whiteway ◽  
Daniel A. Butts
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Large Population ◽  
Component Analysis ◽  
Latent Variable Model ◽  
Neural Population ◽  
Model Parameters ◽  
Variable Model ◽  
Sensory Stimuli ◽  
External Stimuli

The activity of sensory cortical neurons is not only driven by external stimuli but also shaped by other sources of input to the cortex. Unlike external stimuli, these other sources of input are challenging to experimentally control, or even observe, and as a result contribute to variability of neural responses to sensory stimuli. However, such sources of input are likely not “noise” and may play an integral role in sensory cortex function. Here we introduce the rectified latent variable model (RLVM) in order to identify these sources of input using simultaneously recorded cortical neuron populations. The RLVM is novel in that it employs nonnegative (rectified) latent variables and is much less restrictive in the mathematical constraints on solutions because of the use of an autoencoder neural network to initialize model parameters. We show that the RLVM outperforms principal component analysis, factor analysis, and independent component analysis, using simulated data across a range of conditions. We then apply this model to two-photon imaging of hundreds of simultaneously recorded neurons in mouse primary somatosensory cortex during a tactile discrimination task. Across many experiments, the RLVM identifies latent variables related to both the tactile stimulation as well as nonstimulus aspects of the behavioral task, with a majority of activity explained by the latter. These results suggest that properly identifying such latent variables is necessary for a full understanding of sensory cortical function and demonstrate novel methods for leveraging large population recordings to this end. NEW & NOTEWORTHY The rapid development of neural recording technologies presents new opportunities for understanding patterns of activity across neural populations. Here we show how a latent variable model with appropriate nonlinear form can be used to identify sources of input to a neural population and infer their time courses. Furthermore, we demonstrate how these sources are related to behavioral contexts outside of direct experimental control.

scholarly journals TPLVM: Portfolio Construction by Student’s t-Process Latent Variable Model

Mathematics ◽  
2020 ◽  
Vol 8 (3) ◽  
pp. 449 ◽  
Author(s):  
Yusuke Uchiyama ◽  
Kei Nakagawa
Keyword(s):  
Gaussian Process ◽  
Asset Allocation ◽  
Latent Variables ◽  
Latent Variable ◽  
Minimum Variance ◽  
Latent Variable Model ◽  
Portfolio Construction ◽  
Variable Model ◽  
Student’S T ◽  
Non Gaussian

Optimal asset allocation is a key topic in modern finance theory. To realize the optimal asset allocation on investor’s risk aversion, various portfolio construction methods have been proposed. Recently, the applications of machine learning are rapidly growing in the area of finance. In this article, we propose the Student’s t-process latent variable model (TPLVM) to describe non-Gaussian fluctuations of financial timeseries by lower dimensional latent variables. Subsequently, we apply the TPLVM to portfolio construction as an alternative of existing nonlinear factor models. To test the performance of the proposed method, we construct minimum-variance portfolios of global stock market indices based on the TPLVM or Gaussian process latent variable model. By comparing these portfolios, we confirm the proposed portfolio outperforms that of the existing Gaussian process latent variable model.

Probability-Based and Measurement-Related Hypotheses With Full Restriction for Investigations by Means of Confirmatory Factor Analysis

Methodology ◽  
2011 ◽  
Vol 7 (4) ◽  
pp. 157-164
Author(s):  
Karl Schweizer
Keyword(s):  
Factor Analysis ◽  
Confirmatory Factor Analysis ◽  
Cognitive Processing ◽  
Latent Variables ◽  
Repeated Measures ◽  
Latent Variable ◽  
Model Fit ◽  
Repeated Measures Data ◽  
Confirmatory Factor ◽  
And Performance

Probability-based and measurement-related hypotheses for confirmatory factor analysis of repeated-measures data are investigated. Such hypotheses comprise precise assumptions concerning the relationships among the true components associated with the levels of the design or the items of the measure. Measurement-related hypotheses concentrate on the assumed processes, as, for example, transformation and memory processes, and represent treatment-dependent differences in processing. In contrast, probability-based hypotheses provide the opportunity to consider probabilities as outcome predictions that summarize the effects of various influences. The prediction of performance guided by inexact cues serves as an example. In the empirical part of this paper probability-based and measurement-related hypotheses are applied to working-memory data. Latent variables according to both hypotheses contribute to a good model fit. The best model fit is achieved for the model including latent variables that represented serial cognitive processing and performance according to inexact cues in combination with a latent variable for subsidiary processes.

Sign in / Sign up

Export Citation Format

Share Document