scholarly journals Learning Markov Models Via Low-Rank Optimization

2021 ◽  
Author(s):  
Ziwei Zhu ◽  
Xudong Li ◽  
Mengdi Wang ◽  
Anru Zhang
Keyword(s):  
Markov Models ◽  
Supply And Demand ◽  
Likelihood Estimation ◽  
Low Rank ◽  
Programming Algorithm ◽  
High Dimensional ◽  
High Dimensions ◽  
Taxi Service ◽  
Low Dimensional ◽  
Nuclear Norm Regularization

Taming high-dimensional Markov models In “Learning Markov models via low-rank optimization”, Z. Zhu, X. Li, M. Wang, and A. Zhang focus on learning a high-dimensional Markov model with low-dimensional latent structure from a single trajectory of states. To overcome the curse of high dimensions, the authors propose to equip the standard MLE (maximum-likelihood estimation) with either nuclear norm regularization or rank constraint. They show that both approaches can estimate the full transition matrix accurately using a trajectory of length that is merely proportional to the number of states. To solve the rank-constrained MLE, which is a nonconvex problem, the authors develop a new DC (difference) programming algorithm. Finally, they apply the proposed methods to analyze taxi trips on the Manhattan island and partition the island based on the destination preference of customers; this partition can help balance supply and demand of taxi service and optimize the allocation of traffic resources.

scholarly journals Bootstrapping spectral statistics in high dimensions

Biometrika ◽  
2019 ◽  
Vol 106 (4) ◽  
pp. 781-801 ◽  
Author(s):  
Miles E Lopes ◽  
Andrew Blandino ◽  
Alexander Aue
Keyword(s):  
Parametric Bootstrap ◽  
Covariance Matrices ◽  
Asymptotic Formulas ◽  
High Dimensional ◽  
High Dimensions ◽  
Full Data ◽  
Sample Covariance ◽  
Practical Standpoint ◽  
Spectral Statistics ◽  
Low Dimensional

Summary Statistics derived from the eigenvalues of sample covariance matrices are called spectral statistics, and they play a central role in multivariate testing. Although bootstrap methods are an established approach to approximating the laws of spectral statistics in low-dimensional problems, such methods are relatively unexplored in the high-dimensional setting. The aim of this article is to focus on linear spectral statistics as a class of prototypes for developing a new bootstrap in high dimensions, a method we refer to as the spectral bootstrap. In essence, the proposed method originates from the parametric bootstrap and is motivated by the fact that in high dimensions it is difficult to obtain a nonparametric approximation to the full data-generating distribution. From a practical standpoint, the method is easy to use and allows the user to circumvent the difficulties of complex asymptotic formulas for linear spectral statistics. In addition to proving the consistency of the proposed method, we present encouraging empirical results in a variety of settings. Lastly, and perhaps most interestingly, we show through simulations that the method can be applied successfully to statistics outside the class of linear spectral statistics, such as the largest sample eigenvalue and others.

Implications of High-Dimensional Geometry for Structural Reliability Analysis and a Novel Linear Response Surface Method Based on SVM

2015 ◽  
Vol 12 (04) ◽  
pp. 1540016 ◽  
Author(s):  
Umberto Alibrandi ◽  
Amir Alani ◽  
C. G. Koh
Keyword(s):  
Response Surface ◽  
High Dimension ◽  
Linear Response ◽  
Structural Reliability ◽  
High Dimensional ◽  
Support Vector ◽  
High Dimensions ◽  
Vector Method ◽  
Reliability Method ◽  
Low Dimensional

The geometry of high-dimensional spaces is very different from low dimensional spaces and possesses some counter-intuitive features. It is shown that, for high dimensions, the sampling points fall far away from the origin and concentrate within an intersection between a very thin shell and a suitable equatorial slab. The well-known First-Order Reliability Method (FORM), originally formulated for low dimensions, may work well in many engineering problems of high dimension. But it is not able to reveal the level of achieved accuracy. Considering the features of high-dimensional geometry, a novel linear response surface based on Support Vector Method (SVM) is proposed for structural reliability problems of high dimension. The method is shown to outperform FORM for structural reliability problems of high dimension in terms of robustness and accuracy.

scholarly journals Estimating marginal likelihoods from the posterior draws through a geometric identity

2020 ◽  
Vol 26 (3) ◽  
pp. 205-221
Author(s):  
Johannes Reichl
Keyword(s):  
Bayesian Modeling ◽  
Marginal Likelihood ◽  
Likelihood Estimation ◽  
Simple Approach ◽  
High Dimensional ◽  
Marginal Likelihoods ◽  
Random Intercept ◽  
Low Dimensional ◽  
Excellent Stability ◽  
Marginal Likelihood Estimation

AbstractThis article develops a new estimator of the marginal likelihood that requires only a sample of the posterior distribution as the input from the analyst. This sample may come from any sampling scheme, such as Gibbs sampling or Metropolis–Hastings sampling. The presented approach can be implemented generically in almost any application of Bayesian modeling and significantly decreases the computational burdens associated with marginal likelihood estimation compared to existing techniques. The functionality of this method is demonstrated in the context of probit and logit regressions, on two mixtures of normals models, and also on a high-dimensional random intercept probit. Simulation results show that the simple approach presented here achieves excellent stability in low-dimensional models, and also clearly outperforms existing methods when the number of coefficients in the model increases.

scholarly journals Randomized maximum likelihood based posterior sampling

2021 ◽  
Author(s):  
Yuming Ba ◽  
Jana de Wiljes ◽  
Dean S. Oliver ◽  
Sebastian Reich
Keyword(s):  
Inverse Problems ◽  
Flow Problem ◽  
Low Rank ◽  
High Dimensional ◽  
Posterior Distributions ◽  
High Dimensions ◽  
Bayesian Inverse Problems ◽  
Posterior Sampling ◽  
Multimodal Distributions ◽  
Randomized Maximum Likelihood

AbstractMinimization of a stochastic cost function is commonly used for approximate sampling in high-dimensional Bayesian inverse problems with Gaussian prior distributions and multimodal posterior distributions. The density of the samples generated by minimization is not the desired target density, unless the observation operator is linear, but the distribution of samples is useful as a proposal density for importance sampling or for Markov chain Monte Carlo methods. In this paper, we focus on applications to sampling from multimodal posterior distributions in high dimensions. We first show that sampling from multimodal distributions is improved by computing all critical points instead of only minimizers of the objective function. For applications to high-dimensional geoscience inverse problems, we demonstrate an efficient approximate weighting that uses a low-rank Gauss-Newton approximation of the determinant of the Jacobian. The method is applied to two toy problems with known posterior distributions and a Darcy flow problem with multiple modes in the posterior.

scholarly journals High-dimensional Similarity Learning via Dual-sparse Random Projection

2018 ◽  
Author(s):  
Dezhong Yao ◽  
Peilin Zhao ◽  
Tuan-Anh Nguyen Pham ◽  
Gao Cong
Keyword(s):  
Dimensional Space ◽  
Computational Cost ◽  
Optimal Solution ◽  
Random Projection ◽  
Projection Methods ◽  
Low Rank ◽  
High Dimensional ◽  
Similarity Learning ◽  
Reduced Space ◽  
Low Dimensional

We investigate how to adopt dual random projection for high-dimensional similarity learning. For a high-dimensional similarity learning problem, projection is usually adopted to map high-dimensional features into low-dimensional space, in order to reduce the computational cost. However, dimensionality reduction method sometimes results in unstable performance due to the suboptimal solution in original space. In this paper, we propose a dual random projection framework for similarity learning to recover the original optimal solution from subspace optimal solution. Previous dual random projection methods usually make strong assumptions about the data, which need to be low rank or have a large margin. Those assumptions limit dual random projection applications in similarity learning. Thus, we adopt a dual-sparse regularized random projection method that introduces a sparse regularizer into the reduced dual problem. As the original dual solution is a sparse one, applying a sparse regularizer in the reduced space relaxes the low-rank assumption. Experimental results show that our method enjoys higher effectiveness and efficiency than state-of-the-art solutions.

Preface to the special issue, CMAM 2011, no. 3.

2011 ◽  
Vol 11 (3) ◽  
pp. 272
Author(s):  
Ivan Gavrilyuk ◽  
Boris Khoromskij ◽  
Eugene Tyrtyshnikov
Keyword(s):  
Separation Of Variables ◽  
Numerical Algorithms ◽  
Multilevel Methods ◽  
High Dimensional ◽  
Special Issue ◽  
High Dimensions ◽  
Guiding Principle ◽  
Tensor Methods ◽  
Closed World ◽  
The One

Abstract In the recent years, multidimensional numerical simulations with tensor-structured data formats have been recognized as the basic concept for breaking the "curse of dimensionality". Modern applications of tensor methods include the challenging high-dimensional problems of material sciences, bio-science, stochastic modeling, signal processing, machine learning, and data mining, financial mathematics, etc. The guiding principle of the tensor methods is an approximation of multivariate functions and operators with some separation of variables to keep the computational process in a low parametric tensor-structured manifold. Tensors structures had been wildly used as models of data and discussed in the contexts of differential geometry, mechanics, algebraic geometry, data analysis etc. before tensor methods recently have penetrated into numerical computations. On the one hand, the existing tensor representation formats remained to be of a limited use in many high-dimensional problems because of lack of sufficiently reliable and fast software. On the other hand, for moderate dimensional problems (e.g. in "ab-initio" quantum chemistry) as well as for selected model problems of very high dimensions, the application of traditional canonical and Tucker formats in combination with the ideas of multilevel methods has led to the new efficient algorithms. The recent progress in tensor numerical methods is achieved with new representation formats now known as "tensor-train representations" and "hierarchical Tucker representations". Note that the formats themselves could have been picked up earlier in the literature on the modeling of quantum systems. Until 2009 they lived in a closed world of those quantum theory publications and never trespassed the territory of numerical analysis. The tremendous progress during the very recent years shows the new tensor tools in various applications and in the development of these tools and study of their approximation and algebraic properties. This special issue treats tensors as a base for efficient numerical algorithms in various modern applications and with special emphases on the new representation formats.

scholarly journals Classification of Brainwaves for Sleep Stages by High-Dimensional FFT Features from EEG Signals

2020 ◽  
Vol 10 (5) ◽  
pp. 1797 ◽  
Author(s):  
Mera Kartika Delimayanti ◽  
Bedy Purnama ◽  
Ngoc Giang Nguyen ◽  
Mohammad Reza Faisal ◽  
Kunti Robiatul Mahmudah ◽  
...  
Keyword(s):  
Machine Learning ◽  
Sleep Stage ◽  
Machine Learning Algorithms ◽  
High Dimensional ◽  
Sleep Stages ◽  
Eeg Signals ◽  
Stage Classification ◽  
Sleep Stage Classification ◽  
Low Dimensional

Manual classification of sleep stage is a time-consuming but necessary step in the diagnosis and treatment of sleep disorders, and its automation has been an area of active study. The previous works have shown that low dimensional fast Fourier transform (FFT) features and many machine learning algorithms have been applied. In this paper, we demonstrate utilization of features extracted from EEG signals via FFT to improve the performance of automated sleep stage classification through machine learning methods. Unlike previous works using FFT, we incorporated thousands of FFT features in order to classify the sleep stages into 2–6 classes. Using the expanded version of Sleep-EDF dataset with 61 recordings, our method outperformed other state-of-the art methods. This result indicates that high dimensional FFT features in combination with a simple feature selection is effective for the improvement of automated sleep stage classification.

scholarly journals A Nonlinear Maximum Correntropy Information Filter for High-Dimensional Neural Decoding

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 743
Author(s):  
Xi Liu ◽  
Shuhang Chen ◽  
Xiang Shen ◽  
Xiang Zhang ◽  
Yiwen Wang
Keyword(s):  
State Estimation ◽  
Measurement Model ◽  
High Dimensional ◽  
Neural Firing ◽  
The Neural Network ◽  
Information Filter ◽  
Critical Technology ◽  
Dimensional Measurements ◽  
Non Gaussian ◽  
Low Dimensional

Neural signal decoding is a critical technology in brain machine interface (BMI) to interpret movement intention from multi-neural activity collected from paralyzed patients. As a commonly-used decoding algorithm, the Kalman filter is often applied to derive the movement states from high-dimensional neural firing observation. However, its performance is limited and less effective for noisy nonlinear neural systems with high-dimensional measurements. In this paper, we propose a nonlinear maximum correntropy information filter, aiming at better state estimation in the filtering process for a noisy high-dimensional measurement system. We reconstruct the measurement model between the high-dimensional measurements and low-dimensional states using the neural network, and derive the state estimation using the correntropy criterion to cope with the non-Gaussian noise and eliminate large initial uncertainty. Moreover, analyses of convergence and robustness are given. The effectiveness of the proposed algorithm is evaluated by applying it on multiple segments of neural spiking data from two rats to interpret the movement states when the subjects perform a two-lever discrimination task. Our results demonstrate better and more robust state estimation performance when compared with other filters.

scholarly journals Supervised dimensionality reduction for big data

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Joshua T. Vogelstein ◽  
Eric W. Bridgeford ◽  
Minh Tang ◽  
Da Zheng ◽  
Christopher Douville ◽  
...  
Keyword(s):  
Dimensionality Reduction ◽  
Data Science ◽  
Real Data ◽  
Low Rank ◽  
Conditional Moment ◽  
Desktop Computer ◽  
Reduction Techniques ◽  
Reduction Methods ◽  
The Individual ◽  
Low Dimensional

AbstractTo solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees. We introduce an approach to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest version, Linear Optimal Low-rank projection, incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that Linear Optimal Low-Rank Projection and its generalizations lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of more than 150 million features, and several genomics datasets with more than 500,000 features, Linear Optimal Low-Rank Projection outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.

scholarly journals Markov-Modulated Continuous-Time Markov Chains to Identify Site- and Branch-Specific Evolutionary Variation in BEAST

2020 ◽  
Vol 70 (1) ◽  
pp. 181-189
Author(s):  
Guy Baele ◽  
Mandev S Gill ◽  
Paul Bastide ◽  
Philippe Lemey ◽  
Marc A Suchard
Keyword(s):  
High Performance ◽  
Markov Models ◽  
Marginal Likelihood ◽  
Likelihood Estimation ◽  
Phylogenetic Inference ◽  
Time Variability ◽  
Substitution Process ◽  
Wide Range ◽  
Markov Modulated ◽  
Over Time

Abstract Markov models of character substitution on phylogenies form the foundation of phylogenetic inference frameworks. Early models made the simplifying assumption that the substitution process is homogeneous over time and across sites in the molecular sequence alignment. While standard practice adopts extensions that accommodate heterogeneity of substitution rates across sites, heterogeneity in the process over time in a site-specific manner remains frequently overlooked. This is problematic, as evolutionary processes that act at the molecular level are highly variable, subjecting different sites to different selective constraints over time, impacting their substitution behavior. We propose incorporating time variability through Markov-modulated models (MMMs), which extend covarion-like models and allow the substitution process (including relative character exchange rates as well as the overall substitution rate) at individual sites to vary across lineages. We implement a general MMM framework in BEAST, a popular Bayesian phylogenetic inference software package, allowing researchers to compose a wide range of MMMs through flexible XML specification. Using examples from bacterial, viral, and plastid genome evolution, we show that MMMs impact phylogenetic tree estimation and can substantially improve model fit compared to standard substitution models. Through simulations, we show that marginal likelihood estimation accurately identifies the generative model and does not systematically prefer the more parameter-rich MMMs. To mitigate the increased computational demands associated with MMMs, our implementation exploits recent developments in BEAGLE, a high-performance computational library for phylogenetic inference. [Bayesian inference; BEAGLE; BEAST; covarion, heterotachy; Markov-modulated models; phylogenetics.]

Sign in / Sign up

Export Citation Format

Share Document