scholarly journals Parametric Manifold Learning of Gaussian Mixture Models

2019 ◽  
Author(s):  
Ziquan Liu ◽  
Lei Yu ◽  
Janet H. Hsiao ◽  
Antoni B. Chan
Keyword(s):  
Manifold Learning ◽  
Probability Distributions ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Monte Carlo Sampling ◽  
High Dimensional ◽  
Dimensional Manifold ◽  
Inverse Mapping ◽  
Principal Axes ◽  
Leibler Divergence

The Gaussian Mixture Model (GMM) is among the most widely used parametric probability distributions for representing data. However, it is complicated to analyze the relationship among GMMs since they lie on a high-dimensional manifold. Previous works either perform clustering of GMMs, which learns a limited discrete latent representation, or kernel-based embedding of GMMs, which is not interpretable due to difficulty in computing the inverse mapping. In this paper, we propose Parametric Manifold Learning of GMMs (PML-GMM), which learns a parametric mapping from a low-dimensional latent space to a high-dimensional GMM manifold. Similar to PCA, the proposed mapping is parameterized by the principal axes for the component weights, means, and covariances, which are optimized to minimize the reconstruction loss measured using Kullback-Leibler divergence (KLD). As the KLD between two GMMs is intractable, we approximate the objective function by a variational upper bound, which is optimized by an EM-style algorithm. Moreover, We derive an efficient solver by alternating optimization of subproblems and exploit Monte Carlo sampling to escape from local minima. We demonstrate the effectiveness of PML-GMM through experiments on synthetic, eye-fixation, flow cytometry, and social check-in data.

scholarly journals Measure of Similarity between GMMs by Embedding of the Parameter Space That Preserves KL Divergence

Mathematics ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 957
Author(s):  
Branislav Popović ◽  
Lenka Cepova ◽  
Robert Cep ◽  
Marko Janev ◽  
Lidija Krstanović
Keyword(s):  
Computational Complexity ◽  
Parameter Space ◽  
Recognition Accuracy ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Dimensional Manifold ◽  
Dimensional Parameter ◽  
Trade Off ◽  
Measure Of Similarity ◽  
Lower Dimensional

In this work, we deliver a novel measure of similarity between Gaussian mixture models (GMMs) by neighborhood preserving embedding (NPE) of the parameter space, that projects components of GMMs, which by our assumption lie close to lower dimensional manifold. By doing so, we obtain a transformation from the original high-dimensional parameter space, into a much lower-dimensional resulting parameter space. Therefore, resolving the distance between two GMMs is reduced to (taking the account of the corresponding weights) calculating the distance between sets of lower-dimensional Euclidean vectors. Much better trade-off between the recognition accuracy and the computational complexity is achieved in comparison to measures utilizing distances between Gaussian components evaluated in the original parameter space. The proposed measure is much more efficient in machine learning tasks that operate on large data sets, as in such tasks, the required number of overall Gaussian components is always large. Artificial, as well as real-world experiments are conducted, showing much better trade-off between recognition accuracy and computational complexity of the proposed measure, in comparison to all baseline measures of similarity between GMMs tested in this paper.

scholarly journals Classification of Infrared Objects in Manifold Space Using Kullback-Leibler Divergence of Gaussian Distributions of Image Points

Symmetry ◽  
2020 ◽  
Vol 12 (3) ◽  
pp. 434 ◽  
Author(s):  
Huilin Ge ◽  
Zhiyu Zhu ◽  
Kang Lou ◽  
Wei Wei ◽  
Runbang Liu ◽  
...  
Keyword(s):  
Manifold Learning ◽  
Gaussian Distribution ◽  
Classification Accuracy ◽  
Infrared Image ◽  
Data Sets ◽  
Dimensional Manifold ◽  
Infrared Images ◽  
Leibler Divergence ◽  
Data Points ◽  
Low Dimensional

Infrared image recognition technology can work day and night and has a long detection distance. However, the infrared objects have less prior information and external factors in the real-world environment easily interfere with them. Therefore, infrared object classification is a very challenging research area. Manifold learning can be used to improve the classification accuracy of infrared images in the manifold space. In this article, we propose a novel manifold learning algorithm for infrared object detection and classification. First, a manifold space is constructed with each pixel of the infrared object image as a dimension. Infrared images are represented as data points in this constructed manifold space. Next, we simulate the probability distribution information of infrared data points with the Gaussian distribution in the manifold space. Then, based on the Gaussian distribution information in the manifold space, the distribution characteristics of the data points of the infrared image in the low-dimensional space are derived. The proposed algorithm uses the Kullback-Leibler (KL) divergence to minimize the loss function between two symmetrical distributions, and finally completes the classification in the low-dimensional manifold space. The efficiency of the algorithm is validated on two public infrared image data sets. The experiments show that the proposed method has a 97.46% classification accuracy and competitive speed in regards to the analyzed data sets.

scholarly journals Regularized Parameter Estimation in High-Dimensional Gaussian Mixture Models

2011 ◽  
Vol 23 (6) ◽  
pp. 1605-1622 ◽  
Author(s):  
Lingyan Ruan ◽  
Ming Yuan ◽  
Hui Zou
Keyword(s):  
Parameter Estimation ◽  
Mixture Models ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Real Data ◽  
Gaussian Mixture ◽  
High Dimensional ◽  
Model Based Clustering ◽  
Text Type ◽  
Effective Dimensionality

Finite gaussian mixture models are widely used in statistics thanks to their great flexibility. However, parameter estimation for gaussian mixture models with high dimensionality can be challenging because of the large number of parameters that need to be estimated. In this letter, we propose a penalized likelihood estimator to address this difficulty. The [Formula: see text]-type penalty we impose on the inverse covariance matrices encourages sparsity on its entries and therefore helps to reduce the effective dimensionality of the problem. We show that the proposed estimate can be efficiently computed using an expectation-maximization algorithm. To illustrate the practical merits of the proposed method, we consider its applications in model-based clustering and mixture discriminant analysis. Numerical experiments with both simulated and real data show that the new method is a valuable tool for high-dimensional data analysis.

Failure Mode Recognition Clustering Algorithm Based on Manifold Learning

2012 ◽  
Vol 263-266 ◽  
pp. 2126-2130 ◽  
Author(s):  
Zhi Gang Lou ◽  
Hong Zhao Liu
Keyword(s):  
Manifold Learning ◽  
Clustering Algorithm ◽  
Learning Algorithm ◽  
Spectral Characteristics ◽  
Normal Operation ◽  
High Dimensional ◽  
Dimensional Manifold ◽  
Pattern Clustering ◽  
Oil Pipeline ◽  
Adjustable Valve

Manifold learning is a new unsupervised learning method. Its main purpose is to find the inherent law of generated data sets. Be used for high dimensional nonlinear fault samples for learning, in order to identify embedded in high dimensional data space in the low dimensional manifold, can be effective data found the essential characteristics of fault identification. In many types of fault, sometimes often failure and normal operation of the equipment of some operation similar to misjudgment, such as oil pipeline transportation process, pipeline regulating pump, adjustable valve, pump switch, normal operation and pipeline leakage fault condition similar spectral characteristics, thus easy for pipeline leakage cause mistakes. This paper uses the manifold learning algorithm for fault pattern clustering recognition, and through experiments on the algorithm is evaluated.

Spectral Clustering Algorithm of Uncertain Objects

2013 ◽  
Vol 300-301 ◽  
pp. 1058-1061
Author(s):  
Tong He
Keyword(s):  
Mixture Models ◽  
Experimental Evaluation ◽  
Spectral Clustering ◽  
Clustering Algorithm ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Bayesian Probability ◽  
Leibler Divergence ◽  
Effectiveness And Efficiency ◽  
Spectral Clustering Algorithm

By extending classical spectral clustering algorithm, a new clustering algorithm of uncertain objects is proposed in this paper. In the algorithm, each uncertain object is represented as a Gaussian mixture model, and Kullback-Leibler divergence and Bayesian probability are respectively used as similarity measure between Gaussian mixture models. In an extensive experimental evaluation, we not only show the effectiveness and efficiency of the new algorithm and compare it with CLARANS algorithm of uncertain objects.

Sign in / Sign up

Export Citation Format

Share Document