scholarly journals Learning Latent Variable Gaussian Graphical Model for Biomolecular Network with Low Sample Complexity

2016 ◽  
Vol 2016 ◽  
pp. 1-13 ◽  
Author(s):  
Yanbo Wang ◽  
Quan Liu ◽  
Bo Yuan
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Graphical Model ◽  
Real Data ◽  
Sample Complexity ◽  
Gaussian Graphical Model ◽  
Alternating Direction ◽  
Biomolecular Network ◽  
Ill Posed ◽  
Cancer Networks

Learning a Gaussian graphical model with latent variables is ill posed when there is insufficient sample complexity, thus having to be appropriately regularized. A common choice is convexl1plus nuclear norm to regularize the searching process. However, the best estimator performance is not always achieved with these additive convex regularizations, especially when the sample complexity is low. In this paper, we consider a concave additive regularization which does not require the strong irrepresentable condition. We use concave regularization to correct the intrinsic estimation biases from Lasso and nuclear penalty as well. We establish the proximity operators for our concave regularizations, respectively, which induces sparsity and low rankness. In addition, we extend our method to also allow the decomposition of fused structure-sparsity plus low rankness, providing a powerful tool for models with temporal information. Specifically, we develop a nontrivial modified alternating direction method of multipliers with at least local convergence. Finally, we use both synthetic and real data to validate the excellence of our method. In the application of reconstructing two-stage cancer networks, “the Warburg effect” can be revealed directly.

scholarly journals Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity

Biometrika ◽  
2020 ◽  
Author(s):  
S Na ◽  
M Kolar ◽  
O Koyejo
Keyword(s):  
Graphical Models ◽  
Latent Variable ◽  
Graphical Model ◽  
Brain Connectivity ◽  
Statistical Error ◽  
Real Data ◽  
Low Rank ◽  
Conditional Dependence ◽  
Gradient Descent Algorithm ◽  
The Difference

Abstract Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation. Motivated by modern applications, this manuscript considers an extended setting where each group is generated by a latent variable Gaussian graphical model. Due to the existence of latent factors, the differential network is decomposed into sparse and low-rank components, both of which are symmetric indefinite matrices. We estimate these two components simultaneously using a two-stage procedure: (i) an initialization stage, which computes a simple, consistent estimator, and (ii) a convergence stage, implemented using a projected alternating gradient descent algorithm applied to a nonconvex objective, initialized using the output of the first stage. We prove that given the initialization, the estimator converges linearly with a nontrivial, minimax optimal statistical error. Experiments on synthetic and real data illustrate that the proposed nonconvex procedure outperforms existing methods.

scholarly journals Generalized ridge estimators adapted in structural equation models

2020 ◽  
Vol 43 ◽  
pp. e49929
Author(s):  
Gislene Araujo Pereira ◽  
Mariana Resende ◽  
Marcelo Ângelo Cirillo
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Structural Equation ◽  
Structural Equation Models ◽  
Real Data ◽  
Alternative Methods ◽  
Estimation Methods ◽  
Specification Error ◽  
Maximum Likelihood Methods ◽  
Pls Method

Multicollinearity is detected via regression models, where independent variables are strongly correlated. Since they entail linear relations between observed or latent variables, the structural equation models (SEM) are subject to the multicollinearity effect, whose numerous consequences include the singularity between the inverse matrices used in estimation methods. Given to this behavior, it is natural to understand that the suitability of these estimators to structural equation models show the same features, either in the simulation results that validate the estimators in different multicollinearity degrees, or in their application to real data. Due to the multicollinearity overview arose from the fact that the matrices inversion is impracticable, the usage of numerical procedures demanded by the maximum likelihood methods leads to numerical singularity problems. An alternative could be the use of the Partial Least Squares (PLS) method, however, it is demanded that the observed variables are built by assuming a positive correlation with the latent variable. Thus, theoretically, it is expected that the load signals are positive, however, there are no restrictions to these signals in the algorithms used in the PLS method. This fact implies in corrective areas, such as the observed variables removal or new formulations of the theoretical model. In view of this problem, this paper aimed to propose adaptations of six generalized ridge estimators as alternative methods to estimate SEM parameters. The conclusion is that the evaluated estimators presented the same performance in terms of accuracy, precision while considering the scenarios represented by model without specification error and model with specification error, different levels of multicollinearity and sample sizes.

scholarly journals GAN-EM: GAN Based EM Learning Framework

2019 ◽  
Author(s):  
Wentian Zhao ◽  
Shaojie Wang ◽  
Zhihuai Xie ◽  
Jing Shi ◽  
Chenliang Xu
Keyword(s):  
Maximum Likelihood ◽  
Latent Variables ◽  
Latent Variable ◽  
Supervised Classification ◽  
Real Data ◽  
Gaussian Mixture ◽  
Generative Models ◽  
Image Clustering ◽  
Class Label ◽  
Learning Framework

Expectation maximization (EM) algorithm is to find maximum likelihood solution for models having latent variables. A typical example is Gaussian Mixture Model (GMM) which requires Gaussian assumption, however, natural images are highly non-Gaussian so that GMM cannot be applied to perform image clustering task on pixel space. To overcome such limitation, we propose a GAN based EM learning framework that can maximize the likelihood of images and estimate the latent variables. We call this model GAN-EM, which is a framework for image clustering, semi-supervised classification and dimensionality reduction. In M-step, we design a novel loss function for discriminator of GAN to perform maximum likelihood estimation (MLE) on data with soft class label assignments. Specifically, a conditional generator captures data distribution for K classes, and a discriminator tells whether a sample is real or fake for each class. Since our model is unsupervised, the class label of real data is regarded as latent variable, which is estimated by an additional network (E-net) in E-step. The proposed GAN-EM achieves state-of-the-art clustering and semi-supervised classification results on MNIST, SVHN and CelebA, as well as comparable quality of generated images to other recently developed generative models.

scholarly journals DNA Methylation Network Estimation with Sparse Latent Gaussian Graphical Model

2018 ◽  
Author(s):  
Bernard Ng ◽  
Sina Jafarzadeh ◽  
Daniel Cole ◽  
Anna Goldenberg ◽  
Sara Mostafavi
Keyword(s):  
Gene Expression ◽  
Dna Methylation ◽  
Latent Variables ◽  
Latent Class ◽  
Graphical Model ◽  
Expression Patterns ◽  
Methylation Data ◽  
Gaussian Graphical Model ◽  
Cpg Sites ◽  
Network Estimation

AbstractInferring molecular interaction networks from genomics data is important for advancing our understanding of biological processes. Whereas considerable research effort has been placed on inferring such networks from gene expression data, network estimation from DNA methylation data has received very little attention due to the substantially higher dimensionality and complications with result interpretation for non-genic regions. To combat these challenges, we propose here an approach based on sparse latent Gaussian graphical model (SLGGM). The core idea is to perform network estimation on q latent variables as opposed to d CpG sites, with q<<d. To impose a correspondence between the latent variables and genes, we use the distance between CpG sites and transcription starting sites of the genes to generate a prior on the CpG sites’ latent class membership. We evaluate this approach on synthetic data, and show on real data that the gene network estimated from DNA methylation data significantly explains gene expression patterns in unseen datasets.

scholarly journals Multi-Partitions Subspace Clustering

Mathematics ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 597 ◽  
Author(s):  
Vincent Vandewalle
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Subspace Clustering ◽  
Real Data ◽  
Model Choice ◽  
Model Based Clustering ◽  
Model Based ◽  
Choice Strategy ◽  
Factorial Discriminant Analysis ◽  
Bic Criterion

In model based clustering, it is often supposed that only one clustering latent variable explains the heterogeneity of the whole dataset. However, in many cases several latent variables could explain the heterogeneity of the data at hand. Finding such class variables could result in a richer interpretation of the data. In the continuous data setting, a multi-partition model based clustering is proposed. It assumes the existence of several latent clustering variables, each one explaining the heterogeneity of the data with respect to some clustering subspace. It allows to simultaneously find the multi-partitions and the related subspaces. Parameters of the model are estimated through an EM algorithm relying on a probabilistic reinterpretation of the factorial discriminant analysis. A model choice strategy relying on the BIC criterion is proposed to select to number of subspaces and the number of clusters by subspace. The obtained results are thus several projections of the data, each one conveying its own clustering of the data. Model’s behavior is illustrated on simulated and real data.

Probability-Based and Measurement-Related Hypotheses With Full Restriction for Investigations by Means of Confirmatory Factor Analysis

Methodology ◽  
2011 ◽  
Vol 7 (4) ◽  
pp. 157-164
Author(s):  
Karl Schweizer
Keyword(s):  
Factor Analysis ◽  
Confirmatory Factor Analysis ◽  
Cognitive Processing ◽  
Latent Variables ◽  
Repeated Measures ◽  
Latent Variable ◽  
Model Fit ◽  
Repeated Measures Data ◽  
Confirmatory Factor ◽  
And Performance

Probability-based and measurement-related hypotheses for confirmatory factor analysis of repeated-measures data are investigated. Such hypotheses comprise precise assumptions concerning the relationships among the true components associated with the levels of the design or the items of the measure. Measurement-related hypotheses concentrate on the assumed processes, as, for example, transformation and memory processes, and represent treatment-dependent differences in processing. In contrast, probability-based hypotheses provide the opportunity to consider probabilities as outcome predictions that summarize the effects of various influences. The prediction of performance guided by inexact cues serves as an example. In the empirical part of this paper probability-based and measurement-related hypotheses are applied to working-memory data. Latent variables according to both hypotheses contribute to a good model fit. The best model fit is achieved for the model including latent variables that represented serial cognitive processing and performance according to inexact cues in combination with a latent variable for subsidiary processes.

Conceptualizing Protective Family Context and Its effect on Substance Use: Comparisons Across Diverse Ethnic-Racial Youth

2019 ◽  
Author(s):  
Kevin Constante ◽  
Edward Huntley ◽  
Emma Schillinger ◽  
Christine Wagner ◽  
Daniel Keating
Keyword(s):  
Substance Use ◽  
Measurement Invariance ◽  
Latent Variables ◽  
Latent Variable ◽  
Path Model ◽  
Family Context ◽  
Partial Metric ◽  
Racial Groups ◽  
Protective Methods ◽  
Family Variables

Background: Although family behaviors are known to be important for buffering youth against substance use, research in this area often evaluates a particular type of family interaction and how it shapes adolescents’ behaviors, when it is likely that youth experience the co-occurrence of multiple types of family behaviors that may be protective. Methods: The current study (N = 1716, 10th and 12th graders, 55% female) examined associations between protective family context, a latent variable comprised of five different measures of family behaviors, and past 12 months substance use: alcohol, cigarettes, marijuana, and e-cigarettes. Results: A multi-group measurement invariance assessment supported protective family context as a coherent latent construct with partial (metric) measurement invariance among Black, Latinx, and White youth. A multi-group path model indicated that protective family context was significantly associated with less substance use for all youth, but of varying magnitudes across ethnic-racial groups. Conclusion: These results emphasize the importance of evaluating psychometric properties of family-relevant latent variables on the basis of group membership in order to draw appropriate inferences on how such family variables relate to substance use among diverse samples.

scholarly journals Interpretable Variational Graph Autoencoder with Noninformative Prior

2021 ◽  
Vol 13 (2) ◽  
pp. 51
Author(s):  
Lili Sun ◽  
Xueyan Liu ◽  
Min Zhao ◽  
Bo Yang
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Expert Knowledge ◽  
Structural Information ◽  
Standard Normal Distribution ◽  
Noninformative Prior ◽  
Latent Space ◽  
Distribution Parameters ◽  
Standard Normal ◽  
Low Dimensional

Variational graph autoencoder, which can encode structural information and attribute information in the graph into low-dimensional representations, has become a powerful method for studying graph-structured data. However, most existing methods based on variational (graph) autoencoder assume that the prior of latent variables obeys the standard normal distribution which encourages all nodes to gather around 0. That leads to the inability to fully utilize the latent space. Therefore, it becomes a challenge on how to choose a suitable prior without incorporating additional expert knowledge. Given this, we propose a novel noninformative prior-based interpretable variational graph autoencoder (NPIVGAE). Specifically, we exploit the noninformative prior as the prior distribution of latent variables. This prior enables the posterior distribution parameters to be almost learned from the sample data. Furthermore, we regard each dimension of a latent variable as the probability that the node belongs to each block, thereby improving the interpretability of the model. The correlation within and between blocks is described by a block–block correlation matrix. We compare our model with state-of-the-art methods on three real datasets, verifying its effectiveness and superiority.

scholarly journals Remote Sensing Image Stripe Detecting and Destriping Using the Joint Sparsity Constraint with Iterative Support Detection

2019 ◽  
Vol 11 (6) ◽  
pp. 608 ◽  
Author(s):  
Yun-Jia Sun ◽  
Ting-Zhu Huang ◽  
Tian-Hui Ma ◽  
Yong Chen
Keyword(s):  
Remote Sensing ◽  
Real Data ◽  
Total Variation Regularization ◽  
Joint Sparsity ◽  
Alternating Direction ◽  
Proposed Model ◽  
Comparison Results ◽  
Wide Range ◽  
Image Edges ◽  
Full Consideration

Remote sensing images have been applied to a wide range of fields, but they are often degraded by various types of stripes, which affect the image visual quality and limit the subsequent processing tasks. Most existing destriping methods fail to exploit the stripe properties adequately, leading to suboptimal performance. Based on a full consideration of the stripe properties, we propose a new destriping model to achieve stripe detection and stripe removal simultaneously. In this model, we adopt the unidirectional total variation regularization to depict the directional property of stripes and the weighted ℓ 2 , 1 -norm regularization to depict the joint sparsity of stripes. Then, we combine the alternating direction method of multipliers and iterative support detection to solve the proposed model effectively. Comparison results on simulated and real data suggest that the proposed method can remove and detect stripes effectively while preserving image edges and details.

Sign in / Sign up

Export Citation Format

Share Document