Nonlinear random effects mixture models: Maximum likelihood estimation via the EM algorithm

Mixture models become increasingly popular due to their modeling flexibility and are applied to the clustering and classification of heterogeneous data. The EM algorithm is largely used for the maximum likelihood estimation of mixture models because the algorithm is stable in convergence and simple in implementation. Despite such advantages, it is pointed out that the EM algorithm is local and has slow convergence as the main drawback. To avoid the local convergence of the EM algorithm, multiple runs from several different initial values are usually used. Then the algorithm may take a large number of iterations and long computation time to find the maximum likelihood estimates. The speedup of computation of the EM algorithm is available for these problems. We give the algorithms to accelerate the convergence of the EM algorithm and apply them to mixture model estimation. Numerical experiments examine the performance of the acceleration algorithms in terms of the number of iterations and computation time.

Download Full-text

Use of the EM algorithm for maximum likelihood estimation in electron microscope autoradiography

Biometrika ◽

10.1093/biomet/81.1.41 ◽

1994 ◽

Vol 81 (1) ◽

pp. 41-52 ◽

Cited By ~ 1

Author(s):

R. G. AYKROYD ◽

C. W. ANDERSON

Keyword(s):

Electron Microscope ◽

Maximum Likelihood ◽

Maximum Likelihood Estimation ◽

Em Algorithm ◽

Likelihood Estimation ◽

Electron Microscope Autoradiography ◽

The Em Algorithm ◽

Microscope Autoradiography

Download Full-text

Maximum likelihood estimation from fuzzy data using the EM algorithm

Fuzzy Sets and Systems ◽

10.1016/j.fss.2011.05.022 ◽

2011 ◽

Vol 183 (1) ◽

pp. 72-91 ◽

Cited By ~ 60

Author(s):

Thierry Denœux

Keyword(s):

Maximum Likelihood ◽

Maximum Likelihood Estimation ◽

Em Algorithm ◽

Likelihood Estimation ◽

Fuzzy Data ◽

The Em Algorithm

Download Full-text

MAXIMUM LIKELIHOOD ESTIMATION FOR SIMPLEX DISTRIBUTION NONLINEAR MIXED MODELS VIA AN EM ALGORITHM

International Journal of Biomathematics ◽

10.1142/s1793524509000455 ◽

2009 ◽

Vol 02 (01) ◽

pp. 9-17

Author(s):

HONGJIE WEI ◽

WENZHUAN ZHANG

Keyword(s):

Maximum Likelihood ◽

Maximum Likelihood Estimation ◽

Em Algorithm ◽

Random Effects ◽

Mixed Models ◽

Likelihood Estimation ◽

Psychological Research ◽

Nonlinear Mixed Models ◽

Simplex Distribution ◽

Intraocular Gas

Longitudinal continuous proportional data is common in many fields such as biomedical research, psychological research and so on. As shown in [16], such data can be fitted with simplex models. Based on the original models of [16] which assumed a fixed effect for every subject, this paper extends the models by adding random effects and proposes simplex distribution nonlinear mixed models which are one kind of nonlinear reproductive dispersion mixed models. By treating the random effects in the models as hypothetical missing data and applying Metropolis–Hastings (M–H) algorithm, this paper develops an EM algorithm with Markov chain Monte–Carlo method for maximum likelihood estimation in the models. The method is illustrated with the same data from an ophthalmology study on the use of intraocular gas in retinal surgeries in [16] for ease of comparison.

Download Full-text

Comparison of Recent Acceleration Techniques for the EM Algorithm in One- and Two-Parameter Logistic IRT Models

Psych ◽

10.3390/psych2040018 ◽

2020 ◽

Vol 2 (4) ◽

pp. 209-252

Author(s):

Marie Beisemann ◽

Ortrud Wartlick ◽

Philipp Doebler

Keyword(s):

Maximum Likelihood ◽

Maximum Likelihood Estimation ◽

Em Algorithm ◽

Binary Data ◽

Likelihood Estimation ◽

Acceleration Techniques ◽

Irt Models ◽

The Em Algorithm ◽

Log Likelihood ◽

Two Parameter

The expectation–maximization (EM) algorithm is an important numerical method for maximum likelihood estimation in incomplete data problems. However, convergence of the EM algorithm can be slow, and for this reason, many EM acceleration techniques have been proposed. After a review of acceleration techniques in a unified notation with illustrations, three recently proposed EM acceleration techniques are compared in detail: quasi-Newton methods (QN), “squared” iterative methods (SQUAREM), and parabolic EM (PEM). These acceleration techniques are applied to marginal maximum likelihood estimation with the EM algorithm in one- and two-parameter logistic item response theory (IRT) models for binary data, and their performance is compared. QN and SQUAREM methods accelerate convergence of the EM algorithm for the two-parameter logistic model significantly in high-dimensional data problems. Compared to the standard EM, all three methods reduce the number of iterations, but increase the number of total marginal log-likelihood evaluations per iteration. Efficient approximations of the marginal log-likelihood are hence an important part of implementation.

Download Full-text