scholarly journals Approximating posteriors with high-dimensional nuisance parameters via integrated rotated Gaussian approximation

Biometrika ◽  
2020 ◽  
Author(s):  
W Van Den Boom ◽  
G Reeves ◽  
D B Dunson
Keyword(s):  
Gaussian Approximation ◽  
Broad Class ◽  
Nuisance Parameter ◽  
Real Data ◽  
Nuisance Parameters ◽  
High Dimensional ◽  
Data Sets ◽  
Posterior Distributions ◽  
Approximation Accuracy ◽  
Dimensional Parameter

Abstract Posterior computation for high-dimensional data with many parameters can be challenging. This article focuses on a new method for approximating posterior distributions of a low- to moderate-dimensional parameter in the presence of a high-dimensional or otherwise computationally challenging nuisance parameter. The focus is on regression models and the key idea is to separate the likelihood into two components through a rotation. One component involves only the nuisance parameters, which can then be integrated out using a novel type of Gaussian approximation. We provide theory on approximation accuracy that holds for a broad class of forms of the nuisance component and priors. Applying our method to simulated and real data sets shows that it can outperform state-of-the-art posterior approximation approaches.

Multi-Instance Dimensionality Reduction via Sparsity and Orthogonality

2018 ◽  
Vol 30 (12) ◽  
pp. 3281-3308
Author(s):  
Hong Zhu ◽  
Li-Zhi Liao ◽  
Michael K. Ng
Keyword(s):  
Dimensionality Reduction ◽  
Optimization Problem ◽  
Augmented Lagrangian ◽  
Main Idea ◽  
Real Data ◽  
Learning Performance ◽  
High Dimensional ◽  
Data Sets ◽  
Outer Loop ◽  
Orthogonality Constraints

We study a multi-instance (MI) learning dimensionality-reduction algorithm through sparsity and orthogonality, which is especially useful for high-dimensional MI data sets. We develop a novel algorithm to handle both sparsity and orthogonality constraints that existing methods do not handle well simultaneously. Our main idea is to formulate an optimization problem where the sparse term appears in the objective function and the orthogonality term is formed as a constraint. The resulting optimization problem can be solved by using approximate augmented Lagrangian iterations as the outer loop and inertial proximal alternating linearized minimization (iPALM) iterations as the inner loop. The main advantage of this method is that both sparsity and orthogonality can be satisfied in the proposed algorithm. We show the global convergence of the proposed iterative algorithm. We also demonstrate that the proposed algorithm can achieve high sparsity and orthogonality requirements, which are very important for dimensionality reduction. Experimental results on both synthetic and real data sets show that the proposed algorithm can obtain learning performance comparable to that of other tested MI learning algorithms.

Mahalanobis distance informed by clustering

2018 ◽  
Vol 8 (2) ◽  
pp. 377-406
Author(s):  
Almog Lahav ◽  
Ronen Talmon ◽  
Yuval Kluger
Keyword(s):  
Mahalanobis Distance ◽  
High Dimensional Data ◽  
Hidden Variables ◽  
Real Data ◽  
Risk Groups ◽  
High Dimensional ◽  
Data Sets ◽  
Kaplan Meier ◽  
Data Points ◽  
Survival Plot

Abstract A fundamental question in data analysis, machine learning and signal processing is how to compare between data points. The choice of the distance metric is specifically challenging for high-dimensional data sets, where the problem of meaningfulness is more prominent (e.g. the Euclidean distance between images). In this paper, we propose to exploit a property of high-dimensional data that is usually ignored, which is the structure stemming from the relationships between the coordinates. Specifically, we show that organizing similar coordinates in clusters can be exploited for the construction of the Mahalanobis distance between samples. When the observable samples are generated by a nonlinear transformation of hidden variables, the Mahalanobis distance allows the recovery of the Euclidean distances in the hidden space. We illustrate the advantage of our approach on a synthetic example where the discovery of clusters of correlated coordinates improves the estimation of the principal directions of the samples. Our method was applied to real data of gene expression for lung adenocarcinomas (lung cancer). By using the proposed metric we found a partition of subjects to risk groups with a good separation between their Kaplan–Meier survival plot.

Specification tests for covariance structures in high-dimensional statistical models

Biometrika ◽  
2020 ◽  
Author(s):  
X Guo ◽  
C Y Tang
Keyword(s):  
Statistical Models ◽  
Nuisance Parameter ◽  
Additive Model ◽  
Covariance Structure ◽  
Real Data ◽  
Theoretical Development ◽  
High Dimensional ◽  
Test Statistics ◽  
Estimation Errors ◽  
The Impact

Summary We consider testing the covariance structure in statistical models. We focus on developing such tests when the random vectors of interest are not directly observable and have to be derived via estimated models. Additionally, the covariance specification may involve extra nuisance parameters which also need to be estimated. In a generic additive model setting, we develop and investigate test statistics based on the maximum discrepancy measure calculated from the residuals. To approximate the distributions of the test statistics under the null hypothesis, new multiplier bootstrap procedures with dedicated adjustments that incorporate the model and nuisance parameter estimation errors are proposed. Our theoretical development elucidates the impact due to the estimation errors with high-dimensional data and demonstrates the validity of our tests. Simulations and real data examples confirm our theory and demonstrate the performance of the proposed tests.

scholarly journals High dimensional nuisance parameters: an example from parametric survival analysis

2020 ◽  
Vol 3 (2) ◽  
pp. 119-148
Author(s):  
H. S. Battey ◽  
D. R. Cox
Keyword(s):  
Survival Analysis ◽  
Random Effects ◽  
Nuisance Parameter ◽  
Nuisance Parameters ◽  
High Dimensional ◽  
Open Problems ◽  
Complex Dependence ◽  
Alternative Approaches ◽  
Parametric Survival ◽  
Parametric Survival Analysis

AbstractParametric statistical problems involving both large amounts of data and models with many parameters raise issues that are explicitly or implicitly differential geometric. When the number of nuisance parameters is comparable to the sample size, alternative approaches to inference on interest parameters treat the nuisance parameters either as random variables or as arbitrary constants. The two approaches are compared in the context of parametric survival analysis, with emphasis on the effects of misspecification of the random effects distribution. Notably, we derive a detailed expression for the precision of the maximum likelihood estimator of an interest parameter when the assumed random effects model is erroneous, recovering simply derived results based on the Fisher information in the correctly specified situation but otherwise illustrating complex dependence on other aspects. Methods of assessing model adequacy are given. The results are both directly applicable and illustrate general principles of inference when there is a high-dimensional nuisance parameter. Open problems with an information geometrical bearing are outlined.

LASSO-Type Variable Selection Methods for High-Dimensional Data

2013 ◽  
Vol 444-445 ◽  
pp. 604-609
Author(s):  
Guang Hui Fu ◽  
Pan Wang
Keyword(s):  
Variable Selection ◽  
High Dimensional Data ◽  
Real Data ◽  
Oracle Property ◽  
High Dimensional ◽  
Data Sets ◽  
Group Effect ◽  
Selection Methods ◽  
Variable Selection Method ◽  
Type Variable

LASSO is a very useful variable selection method for high-dimensional data , But it does not possess oracle property [Fan and Li, 200 and group effect [Zou and Hastie, 200. In this paper, we firstly review four improved LASSO-type methods which satisfy oracle property and (or) group effect, and then give another two new ones called WFEN and WFAEN. The performance on both the simulation and real data sets shows that WFEN and WFAEN are competitive with other LASSO-type methods.

scholarly journals Measuring Bayesian Robustness Using Rényi Divergence

Stats ◽  
2021 ◽  
Vol 4 (2) ◽  
pp. 251-268
Author(s):  
Luai Al-Labadi ◽  
Forough Fazeli Asl ◽  
Ce Wang
Keyword(s):  
Real Data ◽  
Data Sets ◽  
Posterior Distributions ◽  
Bayesian Robustness ◽  
Rényi Divergence

This paper deals with measuring the Bayesian robustness of classes of contaminated priors. Two different classes of priors in the neighborhood of the elicited prior are considered. The first one is the well-known ϵ-contaminated class, while the second one is the geometric mixing class. The proposed measure of robustness is based on computing the curvature of Rényi divergence between posterior distributions. Examples are used to illustrate the results by using simulated and real data sets.

scholarly journals Cluster Weighted Model Based on TSNE Algorithm for High-Dimensional Data

2021 ◽  
Author(s):  
Kehinde Olobatuyi
Keyword(s):  
Mixture Models ◽  
Dimensional Space ◽  
High Dimensional Data ◽  
Expectation Maximization Algorithm ◽  
Real Data ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
Dimensionality Reduction Technique ◽  
Weighted Model

Abstract Similar to many Machine Learning models, both accuracy and speed of the Cluster weighted models (CWMs) can be hampered by high-dimensional data, leading to previous works on a parsimonious technique to reduce the effect of ”Curse of dimensionality” on mixture models. In this work, we review the background study of the cluster weighted models (CWMs). We further show that parsimonious technique is not sufficient for mixture models to thrive in the presence of huge high-dimensional data. We discuss a heuristic for detecting the hidden components by choosing the initial values of location parameters using the default values in the ”FlexCWM” R package. We introduce a dimensionality reduction technique called T-distributed stochastic neighbor embedding (TSNE) to enhance the parsimonious CWMs in high-dimensional space. Originally, CWMs are suited for regression but for classification purposes, all multi-class variables are transformed logarithmically with some noise. The parameters of the model are obtained via expectation maximization algorithm. The effectiveness of the discussed technique is demonstrated using real data sets from different fields.

scholarly journals Jackknife model averaging prediction methods for complex phenotypes with gene expression levels by integrating external pathway information

2018 ◽  
Author(s):  
Xinghao Yu ◽  
Lishun Xiao ◽  
Ping Zeng ◽  
Shuiping Huang
Keyword(s):  
Predictive Accuracy ◽  
Disease Risk ◽  
Model Fitting ◽  
Model Averaging ◽  
Real Data ◽  
Genetic Data ◽  
Model Specification ◽  
High Dimensional ◽  
Data Sets ◽  
Pathway Information

AbstractMotivationIn the past few years many novel prediction approaches have been proposed and widely employed in high dimensional genetic data for disease risk evaluation. However, those approaches typically ignore in model fitting the important group structures or functional classifications that naturally exists in genetic data.MethodsIn the present study, we applied a novel model averaging approach, called Jackknife Model Averaging Prediction (JMAP), for high dimensional genetic risk prediction while incorporating KEGG pathway information into the model specification. JMAP selects the optimal weights across candidate models by minimizing a cross-validation criterion in a jackknife way. Compared with previous approaches, one of the primary features of JMAP is to allow model weights to vary from 0 to 1 but without the limitation that the summation of weights is equal to one. We evaluated the performance of JMAP using extensive simulation studies and compared it with existing methods. We finally applied JMAP to five real cancer datasets that are publicly available from TCGA.ResultsThe simulations showed that, compared with other existing approaches, JMAP performed best or are among the best methods across a range of scenarios. For example, among 14 out of 16 simulation settings with PVE=0.3, JMAP has an average of 0.075 higher prediction accuracy compared with gsslasso. We further found that in the simulation the model weights for the true candidate models have much smaller chances to be zero compared with those for the null candidate models and are substantially greater in magnitude. In the real data application, JMAP also behaves comparably or better compared with the other methods for both continuous and binary phenotypes. For example, for the COAD, CRC and PAAD data sets, the average gains of predictive accuracy of JMAP are 0.019, 0.064 and 0.052 compared with gsslasso.ConclusionThe proposed method JMAP is a novel method that can provide more accurate phenotypic prediction while incorporating external useful group information.

scholarly journals Human-in-the-loop Active Covariance Learning for Improving Prediction in Small Data Sets

2019 ◽  
Author(s):  
Homayun Afrabandpey ◽  
Tomi Peltola ◽  
Samuel Kaski
Keyword(s):  
Expert Knowledge ◽  
Covariance Structure ◽  
Predictive Performance ◽  
Real Data ◽  
High Dimensional ◽  
Strong Line ◽  
Small Data ◽  
Data Sets ◽  
Sequential Decision ◽  
Human In The Loop

Learning predictive models from small high-dimensional data sets is a key problem in high-dimensional statistics. Expert knowledge elicitation can help, and a strong line of work focuses on directly eliciting informative prior distributions for parameters. This either requires considerable statistical expertise or is laborious, as the emphasis has been on accuracy and not on efficiency of the process. Another line of work queries about importance of features one at a time, assuming them to be independent and hence missing covariance information. In contrast, we propose eliciting expert knowledge about pairwise feature similarities, to borrow statistical strength in the predictions, and using sequential decision making techniques to minimize the effort of the expert. Empirical results demonstrate improvement in predictive performance on both simulated and real data, in high-dimensional linear regression tasks, where we learn the covariance structure with a Gaussian process, based on sequential elicitation.

Sign in / Sign up

Export Citation Format

Share Document