SMEM Algorithm Is Not Fully Compatible with Maximum-Likelihood Framework

2002 ◽  
Vol 14 (6) ◽  
pp. 1261-1266 ◽  
Author(s):  
Akihiro Minagawa ◽  
Norio Tagawa ◽  
Toshiyuki Tanaka
Keyword(s):  
Maximum Likelihood ◽  
Em Algorithm ◽  
Mixture Models ◽  
Expectation Maximization ◽  
Evaluation Method ◽  
Local Optimum ◽  
The Em Algorithm ◽  
Optimum Property ◽  
Split And Merge ◽  
Searching Method

The expectation-maximization (EM) algorithm with split-and-merge operations (SMEM algorithm) proposed by Ueda, Nakano, Ghahramani, and Hinton (2000) is a nonlocal searching method, applicable to mixture models, for relaxing the local optimum property of the EM algorithm. In this article, we point out that the SMEM algorithm uses the acceptance-rejection evaluation method, which may pick up a distribution with smaller likelihood, and demonstrate that an increase in likelihood can then be guaranteed only by comparing log likelihoods.

scholarly journals Fast Computation of the EM Algorithm for Mixture Models

2021 ◽  
Author(s):  
Masahiro Kuroda
Keyword(s):  
Maximum Likelihood ◽  
Em Algorithm ◽  
Mixture Models ◽  
Computation Time ◽  
Likelihood Estimation ◽  
Heterogeneous Data ◽  
Maximum Likelihood Estimates ◽  
The Em Algorithm ◽  
Clustering And Classification ◽  
Number Of Iterations

Mixture models become increasingly popular due to their modeling flexibility and are applied to the clustering and classification of heterogeneous data. The EM algorithm is largely used for the maximum likelihood estimation of mixture models because the algorithm is stable in convergence and simple in implementation. Despite such advantages, it is pointed out that the EM algorithm is local and has slow convergence as the main drawback. To avoid the local convergence of the EM algorithm, multiple runs from several different initial values are usually used. Then the algorithm may take a large number of iterations and long computation time to find the maximum likelihood estimates. The speedup of computation of the EM algorithm is available for these problems. We give the algorithms to accelerate the convergence of the EM algorithm and apply them to mixture model estimation. Numerical experiments examine the performance of the acceleration algorithms in terms of the number of iterations and computation time.

Evolving Gaussian Mixture Models with Splitting and Merging Mutation Operators

2016 ◽  
Vol 24 (2) ◽  
pp. 293-317 ◽  
Author(s):  
Thiago Ferreira Covões ◽  
Eduardo Raul Hruschka ◽  
Joydeep Ghosh
Keyword(s):  
Em Algorithm ◽  
Mixture Models ◽  
Expectation Maximization ◽  
Complexity Analysis ◽  
A Priori ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Critical Parameters ◽  
Mutation Operators ◽  
Split And Merge

This paper describes the evolutionary split and merge for expectation maximization (ESM-EM) algorithm and eight of its variants, which are based on the use of split and merge operations to evolve Gaussian mixture models. Asymptotic time complexity analysis shows that the proposed algorithms are competitive with the state-of-the-art genetic-based expectation maximization (GA-EM) algorithm. Experiments performed in 35 data sets showed that ESM-EM can be computationally more efficient than the widely used multiple runs of EM (for different numbers of components and initializations). Moreover, a variant of ESM-EM free from critical parameters was shown to be able to provide competitive results with GA-EM, even when GA-EM parameters were fine-tuned a priori.

Statistical Inference in Evolutionary Models of DNA Sequences via the EM Algorithm

2005 ◽  
Vol 4 (1) ◽  
Author(s):  
Asger Hobolth ◽  
Jens Ledet Jensen
Keyword(s):  
Maximum Likelihood ◽  
Em Algorithm ◽  
Statistical Inference ◽  
Dna Sequences ◽  
Expectation Maximization ◽  
Continuous Time ◽  
Information Matrix ◽  
Evolutionary Models ◽  
Likelihood Estimator ◽  
The Em Algorithm

We describe statistical inference in continuous time Markov processes of DNA sequences related by a phylogenetic tree. The maximum likelihood estimator can be found by the expectation maximization (EM) algorithm and an expression for the information matrix is also derived. We provide explicit analytical solutions for the EM algorithm and information matrix.

Unsupervised Learning Through Generalized Mixture Model

2021 ◽  
Author(s):  
Samyajoy Pal ◽  
Christian Heumann
Keyword(s):  
Cluster Analysis ◽  
Maximum Likelihood ◽  
Em Algorithm ◽  
Unsupervised Learning ◽  
Mixture Models ◽  
Mixture Model ◽  
Learning Problems ◽  
Point Estimate ◽  
Model Parameters ◽  
The Em Algorithm

Abstract A generalized way of building mixture models using different distributions is explored in this article. The EM algorithm is used with some modifications to accommodate different distributions within the same model. The model uses any point estimate available for the respective distributions to estimate the mixture components and model parameters. The study is focused on the application of mixture models in unsupervised learning problems, especially cluster analysis. The convenience of building mixture models using the generalized approach is further emphasised by appropriate examples, exploiting the well-known maximum likelihood and Bayesian estimates of the parameters of the parent distributions.

Maximum-likelihood estimation in linkage heterogeneity models including additional information via the EM algorithm

1995 ◽  
Vol 12 (5) ◽  
pp. 515-527 ◽  
Author(s):  
Jeanine J. Houwing-Duistermaat ◽  
Lodewijk A. Sandkuijl ◽  
Arthur A. B. Bergen ◽  
Hans C. van Houwelingen
Keyword(s):  
Maximum Likelihood ◽  
Maximum Likelihood Estimation ◽  
Em Algorithm ◽  
Likelihood Estimation ◽  
Additional Information ◽  
The Em Algorithm

scholarly journals Unsupervised Learning in RSS-Based DFLT Using an EM Algorithm

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5549
Author(s):  
Ossi Kaltiokallio ◽  
Roland Hostettler ◽  
Hüseyin Yiğitler ◽  
Mikko Valkama
Keyword(s):  
Em Algorithm ◽  
Expectation Maximization ◽  
Tracking System ◽  
Model Parameters ◽  
Calibration Data ◽  
Tracking Accuracy ◽  
Time Period ◽  
The Em Algorithm ◽  
Device Free ◽  
Localization And Tracking

Received signal strength (RSS) changes of static wireless nodes can be used for device-free localization and tracking (DFLT). Most RSS-based DFLT systems require access to calibration data, either RSS measurements from a time period when the area was not occupied by people, or measurements while a person stands in known locations. Such calibration periods can be very expensive in terms of time and effort, making system deployment and maintenance challenging. This paper develops an Expectation-Maximization (EM) algorithm based on Gaussian smoothing for estimating the unknown RSS model parameters, liberating the system from supervised training and calibration periods. To fully use the EM algorithm’s potential, a novel localization-and-tracking system is presented to estimate a target’s arbitrary trajectory. To demonstrate the effectiveness of the proposed approach, it is shown that: (i) the system requires no calibration period; (ii) the EM algorithm improves the accuracy of existing DFLT methods; (iii) it is computationally very efficient; and (iv) the system outperforms a state-of-the-art adaptive DFLT system in terms of tracking accuracy.

scholarly journals Models and Algorithms for Tracking Target with Coordinated Turn Motion

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Xianghui Yuan ◽  
Feng Lian ◽  
Chongzhao Han
Keyword(s):  
Em Algorithm ◽  
Expectation Maximization ◽  
Multiple Models ◽  
Interacting Multiple Model ◽  
Multiple Model ◽  
Motion Model ◽  
Single Model ◽  
Kinematic Constraint ◽  
The Em Algorithm ◽  
Turn Rate

Tracking target with coordinated turn (CT) motion is highly dependent on the models and algorithms. First, the widely used models are compared in this paper—coordinated turn (CT) model with known turn rate, augmented coordinated turn (ACT) model with Cartesian velocity, ACT model with polar velocity, CT model using a kinematic constraint, and maneuver centered circular motion model. Then, in the single model tracking framework, the tracking algorithms for the last four models are compared and the suggestions on the choice of models for different practical target tracking problems are given. Finally, in the multiple models (MM) framework, the algorithm based on expectation maximization (EM) algorithm is derived, including both the batch form and the recursive form. Compared with the widely used interacting multiple model (IMM) algorithm, the EM algorithm shows its effectiveness.

Sign in / Sign up

Export Citation Format

Share Document