Filtering Turbulent Sparsely Observed Geophysical Flows

Abstract Filtering sparsely turbulent signals from nature is a central problem of contemporary data assimilation. Here, sparsely observed turbulent signals from nature are generated by solutions of two-layer quasigeostrophic models with turbulent cascades from baroclinic instability in two separate regimes with varying Rossby radius mimicking the atmosphere and the ocean. In the “atmospheric” case, large-scale turbulent fluctuations are dominated by barotropic zonal jets with non-Gaussian statistics while the “oceanic” case has large-scale blocking regime transitions with barotropic zonal jets and large-scale Rossby waves. Recently introduced, cheap radical linear stochastic filtering algorithms utilizing mean stochastic models (MSM1, MSM2) that have judicious model errors are developed here as well as a very recent cheap stochastic parameterization extended Kalman filter (SPEKF), which includes stochastic parameterization of additive and multiplicative bias corrections “on the fly.” These cheap stochastic reduced filters as well as a local least squares ensemble adjustment Kalman filter (LLS-EAKF) are compared on the test bed with 36 sparse regularly spaced observations for their skill in recovering turbulent spectra, spatial pattern correlations, and RMS errors. Of these four algorithms, the cheap SPEKF algorithm has the superior overall skill on the stringent test bed, comparable to LLS-EAKF in the atmospheric regime with and without model error and far superior to LLS-EAKF in the ocean regime. LLS-EAKF has special difficulty and high computational cost in the ocean regime with small Rossby radius, which creates stiffness in the perfect dynamics. The even cheaper mean stochastic model, MSM1, has high skill, which is comparable to SPEKF for the oceanic case while MSM2 has significantly worse filtering performance than MSM1 with the same inexpensive computational cost. This is interesting because MSM1 is based on a simple new regression strategy while MSM2 relies on the conventional regression strategy used in stochastic models for shear turbulence.

Download Full-text

The Diffusion Kernel Filter Applied to Lagrangian Data Assimilation

Monthly Weather Review ◽

10.1175/2009mwr2889.1 ◽

2009 ◽

Vol 137 (12) ◽

pp. 4386-4400 ◽

Cited By ~ 6

Author(s):

Paul Krause ◽

Juan M. Restrepo

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Time Series Data ◽

Computational Cost ◽

Series Data ◽

Diffusion Kernel ◽

Test Bed ◽

Error Statistics ◽

Lagrangian Data ◽

Non Gaussian

Abstract The diffusion kernel filter is a sequential particle-method approach to data assimilation of time series data and evolutionary models. The method is applicable to nonlinear/non-Gaussian problems. Within branches of prediction it parameterizes small fluctuations of Brownian-driven paths about deterministic paths. Its implementation is relatively straightforward, provided a tangent linear model is available. A by-product of the parameterization is a bound on the infinity norm of the covariance matrix of such fluctuations (divided by the grid model dimension). As such it can be used to define a notion of “prediction” itself. It can also be used to assess the short time sensitivity of the deterministic history to Brownian noise or Gaussian initial perturbations. In pure oceanic Lagrangian data assimilation, the dynamics and the statistics are nonlinear and non-Gaussian, respectively. Both of these characteristics challenge conventional methods, such as the extended Kalman filter and the popular ensemble Kalman filter. The diffusion kernel filter is proposed as an alternative and is evaluated here on a problem that is often used as a test bed for Lagrangian data assimilation: it consists of tracking point vortices and passive drifters, using a dynamical model and data, both of which have known error statistics. It is found that the diffusion kernel filter captures the first few moments of the random dynamics, with a computational cost that is competitive with a particle filter estimation strategy. The authors also introduce a clustered version of the diffusion kernel filter (cDKF), which is shown to be significantly more efficient with regard to computational cost, at the expense of a slight degradation in the description of the statistics of the dynamical history. Upon parallelizing branches of prediction, cDKF can be computationally competitive with EKF.

Download Full-text

Score matching filters

10.5194/egusphere-egu21-16125 ◽

2021 ◽

Author(s):

Marie Turčičová ◽

Jan Mandel ◽

Kryštof Eben

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Markov Random Fields ◽

Computational Cost ◽

Time Step ◽

Sample Covariance ◽

Monte Carlo Approximation ◽

Gauss Markov Random Fields ◽

Ensemble Filtering ◽

Non Gaussian

A widely popular group of data assimilation methods in meteorological and geophysical sciences is formed by filters based on Monte-Carlo approximation of the traditional Kalman filter, e.g. Ensemble Kalman filter (EnKF), Ensemble square-root filter and others. Due to the computational cost, ensemble size is usually small compared to the dimension of the state vector. Traditional EnKF implicitly uses the sample covariance which is a poor estimate of the background covariance matrix - singular and contaminated by spurious correlations. We focus on modelling the background covariance matrix by means of a linear model for its inverse. This is particularly useful in Gauss-Markov random fields (GMRF), where the inverse covariance matrix has a banded structure. The parameters of the model are estimated by the score matching method which provides estimators in a closed form, cheap to compute. The resulting estimate is a key component of the proposed ensemble filtering algorithms. Under the assumption that the state vector is a GMRF in every time-step, the Score matching filter with Gaussian resampling (SMF-GR) gives in every time-step a consistent (in the large ensemble limit) estimator of mean and covariance matrix of the forecast and analysis distribution. Further, we propose a filtering method called Score matching ensemble filter (SMEF), based on regularization of the EnKF. This filter performs well even for non-Gaussian systems with non-linear dynamics. The performance of both filters is illustrated on a simple linear convection model and Lorenz-96.

Download Full-text

Eddy Amplitudes in Baroclinic Turbulence Driven by Nonzonal Mean Flow: Shear Dispersion of Potential Vorticity

Journal of Physical Oceanography ◽

10.1175/jpo3030.1 ◽

2007 ◽

Vol 37 (4) ◽

pp. 1037-1050 ◽

Cited By ~ 17

Author(s):

K. Shafer Smith

Keyword(s):

Potential Vorticity ◽

Large Scale ◽

Dispersion Model ◽

Baroclinic Instability ◽

Zonal Flow ◽

Mean Flow ◽

Jet Mixing ◽

Zonal Jets ◽

Shear Dispersion ◽

The Cross

Abstract As in the midlatitude atmosphere, midocean eddies are primarily generated by baroclinically unstable mean currents. In contrast to the atmosphere, however, oceanic currents are significantly nonzonal. Even weak nonzonal currents are linearly unstable since β does not suppress growing meridional waves. Theories for the nonlinear equilibration of baroclinic instability, and hence theories for the amplitudes of midocean eddies, must therefore take into account the different dynamics of nonzonal flow. It is shown here that the amplitude of fully developed baroclinic turbulence due to nonzonal shears differs from that due to zonal shears primarily in the nature of the eddy generation. Since β will act to create large-scale zonal jet structures regardless of the generation source, the nature of eddy fluxes of potential vorticity (the source of eddy energy) in the zonal and meridional directions are fundamentally different. The cross-jet mixing has been shown previously to obey a mixing-length scaling, and this corresponds to the generation due to unstable zonal flow. The along-jet mixing, which corresponds to the generation due to the meridional shear, is shown here to be best described by a shear dispersion model. The resulting flux is orders of magnitude higher than in the cross-jet direction, and thus eddy energies driven by baroclinically unstable mean flows with a nonzero meridional component are much larger. This provides an explanation for recently reported results. Moreover, given recent observational and modeling studies showing the ubiquitous presence of zonal jets in the oceans, the results presented here indicate a powerful source of eddy energy.

Download Full-text

Conditional Gaussian Systems for Multiscale Nonlinear Stochastic Systems: Prediction, State Estimation and Uncertainty Quantification

Entropy ◽

10.3390/e20070509 ◽

2018 ◽

Vol 20 (7) ◽

pp. 509 ◽

Cited By ~ 14

Author(s):

Nan Chen ◽

Andrew Majda

Keyword(s):

Large Scale ◽

Stochastic Systems ◽

Planck Equation ◽

Reaction Diffusion ◽

Small Scale ◽

Model Errors ◽

Nonlinear Stochastic Systems ◽

Hybrid Strategy ◽

Highly Nonlinear ◽

Non Gaussian

A conditional Gaussian framework for understanding and predicting complex multiscale nonlinear stochastic systems is developed. Despite the conditional Gaussianity, such systems are nevertheless highly nonlinear and are able to capture the non-Gaussian features of nature. The special structure of the system allows closed analytical formulae for solving the conditional statistics and is thus computationally efficient. A rich gallery of examples of conditional Gaussian systems are illustrated here, which includes data-driven physics-constrained nonlinear stochastic models, stochastically coupled reaction–diffusion models in neuroscience and ecology, and large-scale dynamical models in turbulence, fluids and geophysical flows. Making use of the conditional Gaussian structure, efficient statistically accurate algorithms involving a novel hybrid strategy for different subspaces, a judicious block decomposition and statistical symmetry are developed for solving the Fokker–Planck equation in large dimensions. The conditional Gaussian framework is also applied to develop extremely cheap multiscale data assimilation schemes, such as the stochastic superparameterization, which use particle filters to capture the non-Gaussian statistics on the large-scale part whose dimension is small whereas the statistics of the small-scale part are conditional Gaussian given the large-scale part. Other topics of the conditional Gaussian systems studied here include designing new parameter estimation schemes and understanding model errors.

Download Full-text

Large-scale structure non-Gaussianities with modal methods

Proceedings of the International Astronomical Union ◽

10.1017/s1743921316009649 ◽

2014 ◽

Vol 11 (S308) ◽

pp. 67-68

Author(s):

Marcel Schmittfull

Keyword(s):

Large Scale ◽

Initial Conditions ◽

3D Structure ◽

Computational Cost ◽

Effective Field ◽

Modal Expansion ◽

Large Scale Structures ◽

Bispectrum Estimation ◽

Scale Structures ◽

Non Gaussian

AbstractRelying on a separable modal expansion of the bispectrum, the implementation of a fast estimator for the full bispectrum of a 3d particle distribution is presented. The computational cost of accurate bispectrum estimation is negligible relative to simulation evolution, so the bispectrum can be used as a standard diagnostic whenever the power spectrum is evaluated. As an application, the time evolution of gravitational and primordial dark matter bispectra was measured in a large suite of N-body simulations. The bispectrum shape changes characteristically when the cosmic web becomes dominated by filaments and halos, therefore providing a quantitative probe of 3d structure formation. Our measured bispectra are determined by ∼ 50 coefficients, which can be used as fitting formulae in the nonlinear regime and for non-Gaussian initial conditions. We also compare the measured bispectra with predictions from the Effective Field Theory of Large Scale Structures (EFTofLSS).

Download Full-text

Filtering Partially Observed Multiscale Systems with Heterogeneous Multiscale Methods–Based Reduced Climate Models

Monthly Weather Review ◽

10.1175/mwr-d-10-05067.1 ◽

2012 ◽

Vol 140 (3) ◽

pp. 860-873 ◽

Cited By ~ 10

Author(s):

Emily L. Kang ◽

John Harlim

Keyword(s):

Large Scale ◽

Climate Models ◽

Climate Model ◽

Multiscale Methods ◽

Model Errors ◽

Test Bed ◽

Multiscale Systems ◽

Separation Of Scales ◽

Method Of Moments Estimator ◽

Heterogeneous Multiscale

Abstract This paper presents a fast reduced filtering strategy for assimilating multiscale systems in the presence of observations of only the macroscopic (or large scale) variables. This reduced filtering strategy introduces model errors in estimating the prior forecast statistics through the (heterogeneous multiscale methods) HMM-based reduced climate model as an alternative to the standard expensive (direct numerical simulation) DNS-based fully resolved model. More importantly, this approach is not restricted to any analysis (or Bayesian updating) step from various ensemble-based filters. In a regime where there is a distinctive separation of scales, high filtering skill is obtained through applying the HMM alone with any desirable analysis step from ensemble Kalman filters. When separation of scales is not apparent as typically observed in geophysical turbulent systems, an additional procedure is proposed to reinitialize the microscopic variables to statistically reflect pseudo-observations that are constructed based on the unbiased estimates of the macroscopic variables. Specifically, these pseudo-observations are constructed offline from the conditional distributions of the microscopic forcing to the macroscopic dynamics given the macroscopic variables with the method-of-moments estimator. This HMM-based filter is comparable to the more expensive standard DNS-based filter on a stringent test bed, the two-layer Lorenz’96 model, in various regimes of scale gap, including the not so apparent one. This high filtering skill is robust in the presence of additional model errors through inconsistent pseudo-observations and even when macroscopic observations are spatially incomplete.

Download Full-text

Centered error entropy Kalman filter with application to satellite attitude determination

Transactions of the Institute of Measurement and Control ◽

10.1177/01423312211019867 ◽

2021 ◽

pp. 014233122110198

Author(s):

Baojian Yang ◽

Lu Cao ◽

Dechao Ran ◽

Bing Xiao

Keyword(s):

Kalman Filter ◽

Gaussian Noise ◽

Attitude Determination ◽

Complexity Analysis ◽

Iteration Algorithm ◽

Robust Algorithms ◽

Convergence Conditions ◽

Satellite Attitude ◽

Heavy Tailed ◽

Non Gaussian

Due to unavoidable factors, heavy-tailed noise appears in satellite attitude estimation. Traditional Kalman filter is prone to performance degradation and even filtering divergence when facing non-Gaussian noise. The existing robust algorithms have limited accuracy. To improve the attitude determination accuracy under non-Gaussian noise, we use the centered error entropy (CEE) criterion to derive a new filter named centered error entropy Kalman filter (CEEKF). CEEKF is formed by maximizing the CEE cost function. In the CEEKF algorithm, the prior state values are transmitted the same as the classical Kalman filter, and the posterior states are calculated by the fixed-point iteration method. The CEE EKF (CEE-EKF) algorithm is also derived to improve filtering accuracy in the case of the nonlinear system. We also give the convergence conditions of the iteration algorithm and the computational complexity analysis of CEEKF. The results of the two simulation examples validate the robustness of the algorithm we presented.

Download Full-text

Improved robust Kalman filter for state model errors in GNSS-PPP/MEMS-IMU double state integrated navigation

Advances in Space Research ◽

10.1016/j.asr.2021.02.010 ◽

2021 ◽

Vol 67 (10) ◽

pp. 3156-3168

Author(s):

Zengke Li ◽

Zan Liu ◽

Long Zhao

Keyword(s):

Kalman Filter ◽

State Model ◽

Integrated Navigation ◽

Model Errors ◽

Robust Kalman Filter ◽

Mems Imu

Download Full-text

Renormalization group theory of molecular dynamics

Scientific Reports ◽

10.1038/s41598-021-85286-3 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Daiji Ichishima ◽

Yuya Matsumura

Keyword(s):

Molecular Dynamics ◽

Renormalization Group ◽

Critical Exponent ◽

Computational Efficiency ◽

Large Scale ◽

Dissipative Particle Dynamics ◽

Computational Cost ◽

Coarse Graining ◽

Spatial Dimension ◽

Deborah Number

AbstractLarge scale computation by molecular dynamics (MD) method is often challenging or even impractical due to its computational cost, in spite of its wide applications in a variety of fields. Although the recent advancement in parallel computing and introduction of coarse-graining methods have enabled large scale calculations, macroscopic analyses are still not realizable. Here, we present renormalized molecular dynamics (RMD), a renormalization group of MD in thermal equilibrium derived by using the Migdal–Kadanoff approximation. The RMD method improves the computational efficiency drastically while retaining the advantage of MD. The computational efficiency is improved by a factor of $$2^{n(D+1)}$$ 2 n ( D + 1 ) over conventional MD where D is the spatial dimension and n is the number of applied renormalization transforms. We verify RMD by conducting two simulations; melting of an aluminum slab and collision of aluminum spheres. Both problems show that the expectation values of physical quantities are in good agreement after the renormalization, whereas the consumption time is reduced as expected. To observe behavior of RMD near the critical point, the critical exponent of the Lennard-Jones potential is extracted by calculating specific heat on the mesoscale. The critical exponent is obtained as $$\nu =0.63\pm 0.01$$ ν = 0.63 ± 0.01 . In addition, the renormalization group of dissipative particle dynamics (DPD) is derived. Renormalized DPD is equivalent to RMD in isothermal systems under the condition such that Deborah number $$De\ll 1$$ D e ≪ 1 .

Download Full-text

Experimental Study on Wear Monitoring of Marine Diesel Engine’s Piston Ring by Magnetoresistive Sensor

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.424-425.132 ◽

2012 ◽

Vol 424-425 ◽

pp. 132-136

Author(s):

Guo Jin Chen ◽

Zhang Ming Peng ◽

Jian Guo Yang ◽

Qiao Ying Huang

Keyword(s):

Diesel Engine ◽

Large Scale ◽

Piston Ring ◽

Test Bed ◽

Operating Modes ◽

Magnetoresistive Sensor ◽

Marine Diesel ◽

Wear Monitoring ◽

Oil Tanker ◽

Test Result

On the diesel engine’s test bed, this paper has studied the parameters regarding the diesel engine’s rotational speed, the piston ring’s width and wearing capacity and so on, and their relation with the output signal of the magnetoresistive sensor under the reverse drawing of the diesel engine. The research discovered that the piston ring’s wear and the magnetoresistive sensor’s output have the corresponding relationship. And on the oil tanker with the 6RTA52U diesel engine, the influence of the diesel engine’s operating parameters and the load situations to the magnetoresistive sensor’s output is surveyed under four kinds of different operating modes. The test result and the research conclusion provide the technical foundation for the online Wear monitoring of the large-scale marine diesel engine’s piston ring.

Download Full-text