scholarly journals Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity

Biometrika ◽  
2020 ◽  
Author(s):  
S Na ◽  
M Kolar ◽  
O Koyejo
Keyword(s):  
Graphical Models ◽  
Latent Variable ◽  
Graphical Model ◽  
Brain Connectivity ◽  
Statistical Error ◽  
Real Data ◽  
Low Rank ◽  
Conditional Dependence ◽  
Gradient Descent Algorithm ◽  
The Difference

Abstract Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation. Motivated by modern applications, this manuscript considers an extended setting where each group is generated by a latent variable Gaussian graphical model. Due to the existence of latent factors, the differential network is decomposed into sparse and low-rank components, both of which are symmetric indefinite matrices. We estimate these two components simultaneously using a two-stage procedure: (i) an initialization stage, which computes a simple, consistent estimator, and (ii) a convergence stage, implemented using a projected alternating gradient descent algorithm applied to a nonconvex objective, initialized using the output of the first stage. We prove that given the initialization, the estimator converges linearly with a nontrivial, minimax optimal statistical error. Experiments on synthetic and real data illustrate that the proposed nonconvex procedure outperforms existing methods.

scholarly journals Efficient Regularization Parameter Selection for Latent Variable Graphical Models via Bi-Level Optimization

2019 ◽  
Author(s):  
Joachim Giesen ◽  
Frank Nussbaum ◽  
Christopher Schneider
Keyword(s):  
Graphical Models ◽  
Latent Variable ◽  
Graphical Model ◽  
Regularization Parameter ◽  
Relative Weight ◽  
Low Rank ◽  
Semidefinite Program ◽  
Wide Range ◽  
Regularization Parameters ◽  
Feasible Solutions

Latent variable graphical models are an extension of Gaussian graphical models that decompose the precision matrix into a sparse and a low-rank component. These models can be learned with theoretical guarantees from data via a semidefinite program. This program features two regularization terms, one for promoting sparsity and one for promoting a low rank. In practice, however, it is not straightforward to learn a good model since the model highly depends on the regularization parameters that control the relative weight of the loss function and the two regularization terms. Selecting good regularization parameters can be modeled as a bi-level optimization problem, where the upper level optimizes some form of generalization error and the lower level provides a description of the solution gamut. The solution gamut is the set of feasible solutions for all possible values of the regularization parameters. In practice, it is often not feasible to describe the solution gamut efficiently. Hence, algorithmic schemes for approximating solution gamuts have been devised. One such scheme is Benson's generic vector optimization algorithm that comes with approximation guarantees. So far Benson's algorithm has not been used in conjunction with semidefinite programs like the latent variable graphical Lasso. Here, we develop an adaptive variant of Benson's algorithm for the semidefinite case and show that it keeps the known approximation and run time guarantees. Furthermore, Benson's algorithm turns out to be practically more efficient for the latent variable graphical model than the existing solution gamut approximation scheme on a wide range of data sets.

scholarly journals Learning Latent Variable Gaussian Graphical Model for Biomolecular Network with Low Sample Complexity

2016 ◽  
Vol 2016 ◽  
pp. 1-13 ◽  
Author(s):  
Yanbo Wang ◽  
Quan Liu ◽  
Bo Yuan
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Graphical Model ◽  
Real Data ◽  
Sample Complexity ◽  
Gaussian Graphical Model ◽  
Alternating Direction ◽  
Biomolecular Network ◽  
Ill Posed ◽  
Cancer Networks

Learning a Gaussian graphical model with latent variables is ill posed when there is insufficient sample complexity, thus having to be appropriately regularized. A common choice is convexl1plus nuclear norm to regularize the searching process. However, the best estimator performance is not always achieved with these additive convex regularizations, especially when the sample complexity is low. In this paper, we consider a concave additive regularization which does not require the strong irrepresentable condition. We use concave regularization to correct the intrinsic estimation biases from Lasso and nuclear penalty as well. We establish the proximity operators for our concave regularizations, respectively, which induces sparsity and low rankness. In addition, we extend our method to also allow the decomposition of fused structure-sparsity plus low rankness, providing a powerful tool for models with temporal information. Specifically, we develop a nontrivial modified alternating direction method of multipliers with at least local convergence. Finally, we use both synthetic and real data to validate the excellence of our method. In the application of reconstructing two-stage cancer networks, “the Warburg effect” can be revealed directly.

scholarly journals GGMnonreg: Non-Regularized Gaussian Graphical Models in R

2021 ◽  
Author(s):  
Donald Ray Williams
Keyword(s):  
Graphical Models ◽  
Brain Connectivity ◽  
Dependence Structure ◽  
Gaussian Graphical Models ◽  
Graphical Modeling ◽  
Conditional Dependence ◽  
Common Task ◽  
The Social ◽  
Cortical Regions ◽  
Low Dimensional

Studying complex relations in multivariate datasets is a common task across the sciences. Cognitive neuroscientists model brain connectivity with the goal of unearthing functional and structural associations betweencortical regions. In clinical psychology, researchers wish to better understand the intri-cate web of symptom interrelations that underlie mental health disorders. To this end, graphical modeling has emerged as an oft-used tool in the chest of scientific inquiry. Thebasic idea is to characterize multivariate relations by learning the conditional dependence structure. The cortical regions or symptoms are nodes and the featured connections linking nodes are edges that graphically represent the conditional dependence structure. Graphical modeling is quite common in fields with wide data, that is, when there are more variables (p) thanobservations (n). Accordingly, many regularization-based approaches have been developed for those kinds of data. More recently, graphical modeling has emerged in psychology, where the data is typically long or low-dimensional. The primary purpose of GGMnonreg is to provide methods that were specifically designed for low-dimensional data (e.g., those common in the social-behavioral sciences), for which there is a dearth of methodology.

scholarly journals Changes in EEG Power Spectral Density and Cortical Connectivity in Healthy and Tetraplegic Patients during a Motor Imagery Task

2009 ◽  
Vol 2009 ◽  
pp. 1-12 ◽  
Author(s):  
Filippo Cona ◽  
Melissa Zavaglia ◽  
Laura Astolfi ◽  
Fabio Babiloni ◽  
Mauro Ursino
Keyword(s):  
Brain Connectivity ◽  
Real Data ◽  
Element Model ◽  
Inhibitory Neurons ◽  
Imagery Task ◽  
Power Spectral ◽  
Neural Mass ◽  
The Difference ◽  
Cortical Regions ◽  
Shell Boundary

Knowledge of brain connectivity is an important aspect of modern neuroscience, to understand how the brain realizes its functions. In this work, neural mass models including four groups of excitatory and inhibitory neurons are used to estimate the connectivity among three cortical regions of interests (ROIs) during a foot-movement task. Real data were obtained via high-resolution scalp EEGs on two populations: healthy volunteers and tetraplegic patients. A 3-shell Boundary Element Model of the head was used to estimate the cortical current density and to derive cortical EEGs in the three ROIs. The model assumes that each ROI can generate an intrinsic rhythm in the beta range, and receives rhythms in the alpha and gamma ranges from other two regions. Connectivity strengths among the ROIs were estimated by means of an original genetic algorithm that tries to minimize several cost functions of the difference between real and model power spectral densities. Results show that the stronger connections are those from the cingulate cortex to the primary and supplementary motor areas, thus emphasizing the pivotal role played by theCMA_Lduring the task. Tetraplegic patients exhibit higher connectivity strength on average, with significant statistical differences in some connections. The results are commented and virtues and limitations of the proposed method discussed.

scholarly journals ACE: adaptive cluster expansion for maximum entropy graphical model inference

2016 ◽  
Author(s):  
John P Barton ◽  
Eleonora De Leonardis ◽  
Alice Coucke ◽  
Simona Cocco
Keyword(s):  
Graphical Models ◽  
Cluster Expansion ◽  
Markov Random Fields ◽  
Graphical Model ◽  
Statistical Error ◽  
Artificial Data ◽  
Potts Models ◽  
Pseudo Likelihood ◽  
Correlation Data ◽  
Ace Algorithm

Graphical models are often employed to interpret patterns of correlations observed in data through a network of interactions between the variables. Recently, Ising/Potts models, also known as Markov random fields, have been productively applied to diverse problems in biology, including the prediction of structural contacts from protein sequence data and the description of neural activity patterns. However, inference of such models is a challenging computational problem that cannot be solved exactly. Here we describe the adaptive cluster expansion (ACE) method to quickly and accurately infer Ising or Potts models based on correlation data. ACE avoids overfitting by constructing a sparse network of interactions sufficient to reproduce the observed correlation data within the statistical error expected due to finite sampling. When convergence of the ACE algorithm is slow, we combine it with a Boltzmann Machine Learning algorithm (BML). We illustrate this method on a variety of biological and artificial data sets and compare it to state-of-the-art approximate methods such as Gaussian and pseudo-likelihood inference.We show that ACE accurately reproduces the true parameters of the underlying model when they are known, and yields accurate statistical descriptions of both biological and artificial data. Models inferred by ACE have substantially better statistical performance compared to those obtained from faster Gaussian and pseudo-likelihood methods, which only precisely recover the structure of the interaction network.

Doubly functional graphical models in high dimensions

Biometrika ◽  
2020 ◽  
Vol 107 (2) ◽  
pp. 415-431
Author(s):  
Xinghao Qiao ◽  
Cheng Qian ◽  
Gareth M James ◽  
Shaojun Guo
Keyword(s):  
Graphical Models ◽  
Functional Data ◽  
Graphical Model ◽  
Convergence Rates ◽  
High Dimensions ◽  
Conditional Dependence ◽  
Large P Small N ◽  
Concentration Bounds ◽  
Dependence Relationship ◽  
Functional Principal Components

Summary We consider estimating a functional graphical model from multivariate functional observations. In functional data analysis, the classical assumption is that each function has been measured over a densely sampled grid. However, in practice the functions have often been observed, with measurement error, at a relatively small number of points. We propose a class of doubly functional graphical models to capture the evolving conditional dependence relationship among a large number of sparsely or densely sampled functions. Our approach first implements a nonparametric smoother to perform functional principal components analysis for each curve, then estimates a functional covariance matrix and finally computes sparse precision matrices, which in turn provide the doubly functional graphical model. We derive some novel concentration bounds, uniform convergence rates and model selection properties of our estimator for both sparsely and densely sampled functional data in the high-dimensional large-$p$, small-$n$ regime. We demonstrate via simulations that the proposed method significantly outperforms possible competitors. Our proposed method is applied to a brain imaging dataset.

scholarly journals GGMncv: Nonconvex Penalized Gaussian Graphical Models in R

2020 ◽  
Author(s):  
Donald Ray Williams
Keyword(s):  
Graphical Models ◽  
Graphical Model ◽  
Dependence Structure ◽  
Penalized Least Squares ◽  
Gaussian Graphical Model ◽  
Gaussian Graphical Models ◽  
Personality Psychology ◽  
Conditional Dependence ◽  
Graphical Lasso ◽  
Common Task

Studying complex relations in multivariate datasets is a common task across the sciences. Recently, the Gaussian graphical model has emerged as an increasingly popular model for characterizing the conditional dependence structure of random variables. Although the graphical lasso ($\ell_1$-regularization) is the most well-known estimator, it has several drawbacks that make it less than ideal for model selection. There are now alternative forms of regularization that were developed specifically to overcome issues inherent to the $\ell_1$-penalty.To date, however, these alternatives have been slow to work their way into software for research workers. To address this dearth of software, I developed the package \textbf{GGMncv} that includes a variety of nonconvex penalties, two algorithms for their estimation, plotting capabilities, and an approach for making statistical inference. As an added bonus, \textbf{GGMncv} can be used for nonconvex penalized least squares. After describing the various nonconvex penalties, the functionality of \textbf{GGMncv} is demonstrated through examples using a dataset from personality psychology.

scholarly journals Calibrating the CreditRisk+ Model at Different Time Scales and in Presence of Temporal Autocorrelation †

Mathematics ◽  
2021 ◽  
Vol 9 (14) ◽  
pp. 1679
Author(s):  
Jacopo Giacomelli ◽  
Luca Passalacqua
Keyword(s):  
Time Series ◽  
Default Risk ◽  
Real Life ◽  
Statistical Error ◽  
Real Data ◽  
Sampling Period ◽  
Calibration Procedure ◽  
Industry Standards ◽  
Default Rate

The CreditRisk+ model is one of the industry standards for the valuation of default risk in credit loans portfolios. The calibration of CreditRisk+ requires, inter alia, the specification of the parameters describing the structure of dependence among default events. This work addresses the calibration of these parameters. In particular, we study the dependence of the calibration procedure on the sampling period of the default rate time series, that might be different from the time horizon onto which the model is used for forecasting, as it is often the case in real life applications. The case of autocorrelated time series and the role of the statistical error as a function of the time series period are also discussed. The findings of the proposed calibration technique are illustrated with the support of an application to real data.

scholarly journals Supervised dimensionality reduction for big data

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Joshua T. Vogelstein ◽  
Eric W. Bridgeford ◽  
Minh Tang ◽  
Da Zheng ◽  
Christopher Douville ◽  
...  
Keyword(s):  
Dimensionality Reduction ◽  
Data Science ◽  
Real Data ◽  
Low Rank ◽  
Conditional Moment ◽  
Desktop Computer ◽  
Reduction Techniques ◽  
Reduction Methods ◽  
The Individual ◽  
Low Dimensional

AbstractTo solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees. We introduce an approach to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest version, Linear Optimal Low-rank projection, incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that Linear Optimal Low-Rank Projection and its generalizations lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of more than 150 million features, and several genomics datasets with more than 500,000 features, Linear Optimal Low-Rank Projection outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.

scholarly journals Low-Rank Approximation of Difference between Correlation Matrices Using Inner Product

2021 ◽  
Vol 11 (10) ◽  
pp. 4582
Author(s):  
Kensuke Tanioka ◽  
Satoru Hiwa
Keyword(s):  
Dimensional Reduction ◽  
Low Rank ◽  
Inner Product ◽  
Low Rank Approximation ◽  
Correlation Matrices ◽  
Rank Matrix ◽  
Clustering Approach ◽  
Rank Approximation ◽  
The Difference ◽  
Low Rank Matrix

In the domain of functional magnetic resonance imaging (fMRI) data analysis, given two correlation matrices between regions of interest (ROIs) for the same subject, it is important to reveal relatively large differences to ensure accurate interpretation. However, clustering results based only on differences tend to be unsatisfactory and interpreting the features tends to be difficult because the differences likely suffer from noise. Therefore, to overcome these problems, we propose a new approach for dimensional reduction clustering. Methods: Our proposed dimensional reduction clustering approach consists of low-rank approximation and a clustering algorithm. The low-rank matrix, which reflects the difference, is estimated from the inner product of the difference matrix, not only from the difference. In addition, the low-rank matrix is calculated based on the majorize–minimization (MM) algorithm such that the difference is bounded within the range −1 to 1. For the clustering process, ordinal k-means is applied to the estimated low-rank matrix, which emphasizes the clustering structure. Results: Numerical simulations show that, compared with other approaches that are based only on differences, the proposed method provides superior performance in recovering the true clustering structure. Moreover, as demonstrated through a real-data example of brain activity measured via fMRI during the performance of a working memory task, the proposed method can visually provide interpretable community structures consisting of well-known brain functional networks, which can be associated with the human working memory system. Conclusions: The proposed dimensional reduction clustering approach is a very useful tool for revealing and interpreting the differences between correlation matrices, even when the true differences tend to be relatively small.

Sign in / Sign up

Export Citation Format

Share Document