scholarly journals Data Assimilation with Gaussian Mixture Models Using the Dynamically Orthogonal Field Equations. Part II: Applications

2013 ◽  
Vol 141 (6) ◽  
pp. 1761-1785 ◽  
Author(s):  
Thomas Sondergaard ◽  
Pierre F. J. Lermusiaux
Keyword(s):  
Gaussian Mixture Models ◽  
Mean Field ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Sudden Expansion ◽  
Test Case ◽  
Field Equations ◽  
Gaussian Statistics ◽  
Non Gaussian ◽  
Orthogonal Field

Abstract The properties and capabilities of the Gaussian Mixture Model–Dynamically Orthogonal filter (GMM-DO) are assessed and exemplified by applications to two dynamical systems: 1) the double well diffusion and 2) sudden expansion flows; both of which admit far-from-Gaussian statistics. The former test case, or twin experiment, validates the use of the Expectation-Maximization (EM) algorithm and Bayesian Information Criterion with GMMs in a filtering context; the latter further exemplifies its ability to efficiently handle state vectors of nontrivial dimensionality and dynamics with jets and eddies. For each test case, qualitative and quantitative comparisons are made with contemporary filters. The sensitivity to input parameters is illustrated and discussed. Properties of the filter are examined and its estimates are described, including the equation-based and adaptive prediction of the probability densities; the evolution of the mean field, stochastic subspace modes, and stochastic coefficients; the fitting of GMMs; and the efficient and analytical Bayesian updates at assimilation times and the corresponding data impacts. The advantages of respecting nonlinear dynamics and preserving non-Gaussian statistics are brought to light. For realistic test cases admitting complex distributions and with sparse or noisy measurements, the GMM-DO filter is shown to fundamentally improve the filtering skill, outperforming simpler schemes invoking the Gaussian parametric distribution.

scholarly journals Data Assimilation with Gaussian Mixture Models Using the Dynamically Orthogonal Field Equations. Part I: Theory and Scheme

2013 ◽  
Vol 141 (6) ◽  
pp. 1737-1760 ◽  
Author(s):  
Thomas Sondergaard ◽  
Pierre F. J. Lermusiaux
Keyword(s):  
Data Assimilation ◽  
Mixture Models ◽  
Gaussian Mixture Models ◽  
Expectation Maximization Algorithm ◽  
Planck Equation ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Field Equations ◽  
Dynamical Equations ◽  
Prior Probabilities

Abstract This work introduces and derives an efficient, data-driven assimilation scheme, focused on a time-dependent stochastic subspace that respects nonlinear dynamics and captures non-Gaussian statistics as it occurs. The motivation is to obtain a filter that is applicable to realistic geophysical applications, but that also rigorously utilizes the governing dynamical equations with information theory and learning theory for efficient Bayesian data assimilation. Building on the foundations of classical filters, the underlying theory and algorithmic implementation of the new filter are developed and derived. The stochastic Dynamically Orthogonal (DO) field equations and their adaptive stochastic subspace are employed to predict prior probabilities for the full dynamical state, effectively approximating the Fokker–Planck equation. At assimilation times, the DO realizations are fit to semiparametric Gaussian Mixture Models (GMMs) using the Expectation-Maximization algorithm and the Bayesian Information Criterion. Bayes’s law is then efficiently carried out analytically within the evolving stochastic subspace. The resulting GMM-DO filter is illustrated in a very simple example. Variations of the GMM-DO filter are also provided along with comparisons with related schemes.

scholarly journals Nonmonotonic Generalization Bias of Gaussian Mixture Models

2000 ◽  
Vol 12 (6) ◽  
pp. 1411-1427 ◽  
Author(s):  
Shotaro Akaho ◽  
Hilbert J. Kappen
Keyword(s):  
Symmetry Breaking ◽  
Gaussian Mixture Models ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
General Tendency ◽  
Adaptive Parameters ◽  
Neural Information ◽  
The Difference ◽  
Training Error ◽  
Validation Experiments

Theories of learning and generalization hold that the generalization bias, defined as the difference between the training error and the generalization error, increases on average with the number of adaptive parameters. This article, however, shows that this general tendency is violated for a gaussian mixture model. For temperatures just below the first symmetry breaking point, the effective number of adaptive parameters increases and the generalization bias decreases. We compute the dependence of the neural information criterion on temperature around the symmetry breaking. Our results are confirmed by numerical cross-validation experiments.

scholarly journals Understanding and Predicting Nonlinear Turbulent Dynamical Systems with Information Theory

Atmosphere ◽  
2019 ◽  
Vol 10 (5) ◽  
pp. 248
Author(s):  
Nan Chen ◽  
Xiao Hou ◽  
Qin Li ◽  
Yingda Li
Keyword(s):  
Information Theory ◽  
Dynamical Systems ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Model Error ◽  
High Dimensions ◽  
New Approach ◽  
Fluctuation Dissipation ◽  
Spectrum Density ◽  
Non Gaussian

Complex nonlinear turbulent dynamical systems are ubiquitous in many areas. Quantifying the model error and model uncertainty plays an important role in understanding and predicting complex dynamical systems. In the first part of this article, a simple information criterion is developed to assess the model error in imperfect models. This effective information criterion takes into account the information in both the equilibrium statistics and the temporal autocorrelation function, where the latter is written in the form of the spectrum density that permits the quantification via information theory. This information criterion facilitates the study of model reduction, stochastic parameterizations, and intermittent events. In the second part of this article, a new efficient method is developed to improve the computation of the linear response via the Fluctuation Dissipation Theorem (FDT). This new approach makes use of a Gaussian Mixture (GM) to describe the unperturbed probability density function in high dimensions and avoids utilizing Gaussian approximations in computing the statistical response, as is widely used in the quasi-Gaussian (qG) FDT. Testing examples show that this GM FDT outperforms qG FDT in various strong non-Gaussian regimes.

scholarly journals Acoustic Impulsive Noise Based on Non-Gaussian Models: An Experimental Evaluation

Sensors ◽  
2019 ◽  
Vol 19 (12) ◽  
pp. 2827 ◽  
Author(s):  
Danilo Pena ◽  
Carlos Lima ◽  
Matheus Dória ◽  
Luan Pena ◽  
Allan Martins ◽  
...  
Keyword(s):  
Impulsive Noise ◽  
Gaussian Mixture Models ◽  
Audio Signal ◽  
Poor Performance ◽  
Gaussian Mixture ◽  
Stable Model ◽  
Gaussian Models ◽  
Non Gaussian ◽  
Signal Processing Methods ◽  
Best Fit

In general, acoustic channels are not Gaussian distributed neither are second-order stationary. Considering them for signal processing methods designed for Gaussian assumptions is inadequate, consequently yielding in poor performance of such methods. This paper presents an analysis for audio signal corrupted by impulsive noise using non-Gaussian models. Audio samples are compared to the Gaussian, α -stable and Gaussian mixture models, evaluating the fitting by graphical and numerical methods. We discuss fitting properties as the window length and the overlap, finally concluding that the α -stable model has the best fit for all tested scenarios.

scholarly journals Clustering Using Student t Mixture Copulas

2021 ◽  
Vol 2 (2) ◽  
Author(s):  
Till Massing
Keyword(s):  
Gaussian Mixture Models ◽  
Optimization Methods ◽  
Gaussian Mixture ◽  
Copula Models ◽  
Multimodal Distributions ◽  
Non Gaussian ◽  
Clustering Problems ◽  
Cluster Tool ◽  
Natural Way

AbstractTewari et al. (Parametric characterization of multimodal distributions with non-Gaussian modes, pp 286–292, 2011) introduced Gaussian mixture copula models (GMCM) for clustering problems which do not assume normality of the mixture components as Gaussian mixture models (GMM) do. In this paper, we propose Student t mixture copula models (SMCM) as an extension of GMCMs. GMCMs require weak assumptions, yielding a flexible fit and a powerful cluster tool. Our SMCM extension offers, in a natural way, even more flexibility than the GMCM approach. We discuss estimation issues and compare Expectation-Maximization (EM)-based with numerical simplex optimization methods. We illustrate the SMCM as a tool for image segmentation.

scholarly journals Fitting and Analyzing LES Using Multiple Trivariate Gaussians

2015 ◽  
Vol 72 (3) ◽  
pp. 1094-1116 ◽  
Author(s):  
Grant J. Firl ◽  
David A. Randall
Keyword(s):  
Deep Convection ◽  
Gaussian Mixture Models ◽  
Information Criterion ◽  
Gaussian Mixture ◽  
Higher Order ◽  
Higher Order Moments ◽  
Relative Paucity ◽  
Shallow Cumulus ◽  
Minimum Number ◽  
Simplifying Assumptions

Abstract Assumed-PDF methods for the parameterization of subgrid-scale processes in atmospheric models provide many benefits. Many currently used assumed-PDF schemes reconcile the high number of required PDF parameters with the relative paucity of input moments by employing simplifying assumptions that are difficult to test. This paper explores the possibility of constructing a trivariate double-Gaussian PDF from the first three orders of moments without simplifying assumptions and proves that no unique solution exists. In an effort to provide a path for future improvement of current assumed-PDF schemes, the expectation maximization (EM) algorithm for Gaussian mixture models is used with LES output of shallow cumulus, stratocumulus, and deep convection cases to determine “best fit” PDFs using from one through four Gaussian clusters. The EM PDFs are evaluated using PDF-diagnosed higher-order moments, PDF-diagnosed cloud statistics, and the Akaike information criterion. It was found that two Gaussian clusters were almost always adequate to represent both higher-order moments and cloud statistics like cloud fraction, water content, and vertical fluxes of cloud water and buoyancy in layered clouds such as stratocumulus and deep convective anvils. However, higher-order moments and higher-order cloud statistics were only properly represented when three or four Gaussians were used in the upper regions of shallow cumulus layers and throughout the active portion of deep convection. Evidence is also provided that several common assumptions employed to diagnose trivariate double-Gaussian PDFs from a minimum number of input moments are weak.

Sign in / Sign up

Export Citation Format

Share Document