MPI Implementation of Expectation Maximization Algorithm for Gaussian Mixture Models

2015 ◽  
pp. 517-523 ◽  
Author(s):  
Ayush Kapoor ◽  
Harsh Hemani ◽  
N. Sakthivel ◽  
S. Chaturvedi
Keyword(s):  
Mixture Models ◽  
Expectation Maximization ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Gaussian Mixture ◽  
Mpi Implementation

scholarly journals Data Assimilation with Gaussian Mixture Models Using the Dynamically Orthogonal Field Equations. Part I: Theory and Scheme

2013 ◽  
Vol 141 (6) ◽  
pp. 1737-1760 ◽  
Author(s):  
Thomas Sondergaard ◽  
Pierre F. J. Lermusiaux
Keyword(s):  
Data Assimilation ◽  
Mixture Models ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Planck Equation ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Field Equations ◽  
Dynamical Equations ◽  
Prior Probabilities

Abstract This work introduces and derives an efficient, data-driven assimilation scheme, focused on a time-dependent stochastic subspace that respects nonlinear dynamics and captures non-Gaussian statistics as it occurs. The motivation is to obtain a filter that is applicable to realistic geophysical applications, but that also rigorously utilizes the governing dynamical equations with information theory and learning theory for efficient Bayesian data assimilation. Building on the foundations of classical filters, the underlying theory and algorithmic implementation of the new filter are developed and derived. The stochastic Dynamically Orthogonal (DO) field equations and their adaptive stochastic subspace are employed to predict prior probabilities for the full dynamical state, effectively approximating the Fokker–Planck equation. At assimilation times, the DO realizations are fit to semiparametric Gaussian Mixture Models (GMMs) using the Expectation-Maximization algorithm and the Bayesian Information Criterion. Bayes’s law is then efficiently carried out analytically within the evolving stochastic subspace. The resulting GMM-DO filter is illustrated in a very simple example. Variations of the GMM-DO filter are also provided along with comparisons with related schemes.

scholarly journals Regularized Parameter Estimation in High-Dimensional Gaussian Mixture Models

2011 ◽  
Vol 23 (6) ◽  
pp. 1605-1622 ◽  
Author(s):  
Lingyan Ruan ◽  
Ming Yuan ◽  
Hui Zou
Keyword(s):  
Parameter Estimation ◽  
Mixture Models ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Real Data ◽  
Gaussian Mixture ◽  
High Dimensional ◽  
Model Based Clustering ◽  
Text Type ◽  
Effective Dimensionality

Finite gaussian mixture models are widely used in statistics thanks to their great flexibility. However, parameter estimation for gaussian mixture models with high dimensionality can be challenging because of the large number of parameters that need to be estimated. In this letter, we propose a penalized likelihood estimator to address this difficulty. The [Formula: see text]-type penalty we impose on the inverse covariance matrices encourages sparsity on its entries and therefore helps to reduce the effective dimensionality of the problem. We show that the proposed estimate can be efficiently computed using an expectation-maximization algorithm. To illustrate the practical merits of the proposed method, we consider its applications in model-based clustering and mixture discriminant analysis. Numerical experiments with both simulated and real data show that the new method is a valuable tool for high-dimensional data analysis.

Video Image Segmentation Using Gaussian Mixture Models Based on the Differential Evolution-Based Parameter Estimation

2011 ◽  
Vol 474-476 ◽  
pp. 442-447
Author(s):  
Zhi Gao Zeng ◽  
Li Xin Ding ◽  
Sheng Qiu Yi ◽  
San You Zeng ◽  
Zi Hua Qiu
Keyword(s):  
Image Segmentation ◽  
Differential Evolution ◽  
Mixture Models ◽  
Clustering Algorithms ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Image Data ◽  
Gaussian Mixture ◽  
Parameters Estimation ◽  
Video Image

In order to improve the accuracy of the image segmentation in video surveillance sequences and to overcome the limits of the traditional clustering algorithms that can not accurately model the image data sets which Contains noise data, the paper presents an automatic and accurate video image segmentation algorithm, according to the spatial properties, which uses the Gaussian mixture models to segment the image. But the expectation-maximization algorithm is very sensitive to initial values, and easy to fall into local optimums, so the paper presents a differential evolution-based parameters estimation for Gaussian mixture models. The experiment result shows that the segmentation accuracy has been improved greatly than by the traditional segmentation algorithms.

Mixture Model Approach for Acoustic Emission Control of Cylindrical Pressure Equipment

2005 ◽  
Vol 128 (3) ◽  
pp. 479-483
Author(s):  
Hani Hamdan ◽  
Gérard Govaert
Keyword(s):  
Acoustic Emission ◽  
Gaussian Mixture Model ◽  
Mixture Model ◽  
Expectation Maximization ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Real Data ◽  
Gaussian Mixture ◽  
Mixture Model Approach ◽  
Model Approach

In this paper, we present a new and original mixture model approach for acoustic emission (AE) data clustering. AE techniques have been used in a variety of applications in industrial plants. These techniques can provide the most sophisticated monitoring test and can generally be done with the plant/pressure equipment operating at several conditions. Since the AE clusters may present several constraints (different proportions, volumes, orientations, and shapes), we propose to base the AE cluster analysis on Gaussian mixture models, which will be, in such situations, a powerful approach. Furthermore, the diagonal Gaussian mixture model seems to be well adapted to the detection and monitoring of defect classes since the weldings of cylindrical pressure equipment are lengthened horizontally and vertically (cluster shapes lengthened along the axes). The EM (Expectation-Maximization) algorithm applied to a diagonal Gaussian mixture model provides a satisfactory solution but the real time constraints imposed in our problem make the application of this algorithm impossible if the number of points becomes too big. The solution that we propose is to use the CEM (Classification Expectation-Maximization) algorithm, which converges faster and generates comparable solutions in terms of resulting partition. The practical results on real data are very satisfactory from the experts point of view.

scholarly journals Efficient Greedy Learning of Gaussian Mixture Models

2003 ◽  
Vol 15 (2) ◽  
pp. 469-485 ◽  
Author(s):  
J. J. Verbeek ◽  
N. Vlassis ◽  
B. Kröse
Keyword(s):  
Mixture Models ◽  
Density Estimation ◽  
Expectation Maximization ◽  
State Of The Art ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Optimal Number ◽  
Gaussian Mixtures ◽  
Data Points ◽  
Time Linear

This article concerns the greedy learning of gaussian mixtures. In the greedy approach, mixture components are inserted into the mixture one aftertheother.We propose a heuristic for searching for the optimal component to insert. In a randomized manner, a set of candidate new components is generated. For each of these candidates, we find the locally optimal new component and insert it into the existing mixture. The resulting algorithm resolves the sensitivity to initialization of state-of-the-art methods, like expectation maximization, and has running time linear in the number of data points and quadratic in the (final) number of mixture components. Due to its greedy nature, the algorithm can be particularly useful when the optimal number of mixture components is unknown. Experimental results comparing the proposed algorithm to other methods on density estimation and texture segmentation are provided.

Evolving Gaussian Mixture Models with Splitting and Merging Mutation Operators

2016 ◽  
Vol 24 (2) ◽  
pp. 293-317 ◽  
Author(s):  
Thiago Ferreira Covões ◽  
Eduardo Raul Hruschka ◽  
Joydeep Ghosh
Keyword(s):  
Em Algorithm ◽  
Mixture Models ◽  
Expectation Maximization ◽  
Complexity Analysis ◽  
A Priori ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Critical Parameters ◽  
Mutation Operators ◽  
Split And Merge

This paper describes the evolutionary split and merge for expectation maximization (ESM-EM) algorithm and eight of its variants, which are based on the use of split and merge operations to evolve Gaussian mixture models. Asymptotic time complexity analysis shows that the proposed algorithms are competitive with the state-of-the-art genetic-based expectation maximization (GA-EM) algorithm. Experiments performed in 35 data sets showed that ESM-EM can be computationally more efficient than the widely used multiple runs of EM (for different numbers of components and initializations). Moreover, a variant of ESM-EM free from critical parameters was shown to be able to provide competitive results with GA-EM, even when GA-EM parameters were fine-tuned a priori.

Sign in / Sign up

Export Citation Format

Share Document