scholarly journals A Second-Order Exact Ensemble Square Root Filter for Nonlinear Data Assimilation

2015 ◽  
Vol 143 (4) ◽  
pp. 1347-1367 ◽  
Author(s):  
Julian Tödter ◽  
Bodo Ahrens
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Second Order ◽  
Square Root ◽  
Computationally Efficient ◽  
Research Activities ◽  
Wide Range ◽  
Matrix Square Root ◽  
Non Gaussian ◽  
Ensemble Transform

Abstract The ensemble Kalman filter (EnKF) and its deterministic variants, mostly square root filters such as the ensemble transform Kalman filter (ETKF), represent a popular alternative to variational data assimilation schemes and are applied in a wide range of operational and research activities. Their forecast step employs an ensemble integration that fully respects the nonlinear nature of the analyzed system. In the analysis step, they implicitly assume the prior state and observation errors to be Gaussian. Consequently, in nonlinear systems, the analysis mean and covariance are biased, and these filters remain suboptimal. In contrast, the fully nonlinear, non-Gaussian particle filter (PF) only relies on Bayes’s theorem, which guarantees an exact asymptotic behavior, but because of the so-called curse of dimensionality it is exposed to weight collapse. Here, it is shown how to obtain a new analysis ensemble whose mean and covariance exactly match the Bayesian estimates. This is achieved by a deterministic matrix square root transformation of the forecast ensemble, and subsequently a suitable random rotation that significantly contributes to filter stability while preserving the required second-order statistics. The properties and performance of the proposed algorithm are further investigated via a set of experiments. They indicate that such a filter formulation can increase the analysis quality, even for relatively small ensemble sizes, compared to other ensemble filters in nonlinear, non-Gaussian scenarios. Localization enhances the potential applicability of this PF-inspired scheme in larger-dimensional systems. The proposed algorithm, which is fairly easy to implement and computationally efficient, is referred to as the nonlinear ensemble transform filter (NETF).

Implications of the Form of the Ensemble Transformation in the Ensemble Square Root Filters

2008 ◽  
Vol 136 (3) ◽  
pp. 1042-1053 ◽  
Author(s):  
Pavel Sakov ◽  
Peter R. Oke
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Symmetric Solution ◽  
Square Root ◽  
The Mean ◽  
Transform Matrix ◽  
Small Models ◽  
Ensemble Transform

Abstract This paper considers implications of different forms of the ensemble transformation in the ensemble square root filters (ESRFs) for the performance of ESRF-based data assimilation systems. It highlights the importance of using mean-preserving solutions for the ensemble transform matrix (ETM). The paper shows that an arbitrary mean-preserving ETM can be represented as a product of the symmetric solution and an orthonormal mean-preserving matrix. The paper also introduces a new flavor of ESRF, referred to as ESRF with mean-preserving random rotations. To investigate the performance of different solutions for the ETM in ESRFs, experiments with two small models are conducted. In these experiments, the performances of two mean-preserving solutions, two non-mean-preserving solutions, and a traditional ensemble Kalman filter with perturbed observations are compared. The experiments show a significantly better performance of the mean-preserving solutions for the ETM in ESRFs compared to non-mean-preserving solutions. They also show that applying the mean-preserving random rotations prevents the buildup of ensemble outliers in ESRF-based data assimilation systems.

Maximum Correntropy Criterion Constrained Kalman Filter

2017 ◽  
Author(s):  
Seyed Fakoorian ◽  
Mahmoud Moosavi ◽  
Reza Izanloo ◽  
Vahid Azimi ◽  
Dan Simon
Keyword(s):  
Kalman Filter ◽  
Gaussian Noise ◽  
State Constraints ◽  
Statistical Information ◽  
Second Order ◽  
System Model ◽  
Error Covariance ◽  
Inequality State ◽  
Maximum Correntropy Criterion ◽  
Non Gaussian

Non-Gaussian noise may degrade the performance of the Kalman filter because the Kalman filter uses only second-order statistical information, so it is not optimal in non-Gaussian noise environments. Also, many systems include equality or inequality state constraints that are not directly included in the system model, and thus are not incorporated in the Kalman filter. To address these combined issues, we propose a robust Kalman-type filter in the presence of non-Gaussian noise that uses information from state constraints. The proposed filter, called the maximum correntropy criterion constrained Kalman filter (MCC-CKF), uses a correntropy metric to quantify not only second-order information but also higher-order moments of the non-Gaussian process and measurement noise, and also enforces constraints on the state estimates. We analytically prove that our newly derived MCC-CKF is an unbiased estimator and has a smaller error covariance than the standard Kalman filter under certain conditions. Simulation results show the superiority of the MCC-CKF compared with other estimators when the system measurement is disturbed by non-Gaussian noise and when the states are constrained.

scholarly journals Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization

2017 ◽  
Vol 145 (11) ◽  
pp. 4575-4592 ◽  
Author(s):  
Craig H. Bishop ◽  
Jeffrey S. Whitaker ◽  
Lili Lei
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Satellite Data ◽  
Localization Length ◽  
Model Space ◽  
Ensemble Size ◽  
Length Scales ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Expanded Ensemble

To ameliorate suboptimality in ensemble data assimilation, methods have been introduced that involve expanding the ensemble size. Such expansions can incorporate model space covariance localization and/or estimates of climatological or model error covariances. Model space covariance localization in the vertical overcomes problematic aspects of ensemble-based satellite data assimilation. In the case of the ensemble transform Kalman filter (ETKF), the expanded ensemble size associated with vertical covariance localization would also enable the simultaneous update of entire vertical columns of model variables from hyperspectral and multispectral satellite sounders. However, if the original formulation of the ETKF were applied to an expanded ensemble, it would produce an analysis ensemble that was the same size as the expanded forecast ensemble. This article describes a variation on the ETKF called the gain ETKF (GETKF) that takes advantage of covariances from the expanded ensemble, while producing an analysis ensemble that has the required size of the unexpanded forecast ensemble. The approach also yields an inflation factor that depends on the localization length scale that causes the GETKF to perform differently to an ensemble square root filter (EnSRF) using the same expanded ensemble. Experimentation described herein shows that the GETKF outperforms a range of alternative ETKF-based solutions to the aforementioned problems. In cycling data assimilation experiments with a newly developed storm-track version of the Lorenz-96 model, the GETKF analysis root-mean-square error (RMSE) matches the EnSRF RMSE at shorter than optimal localization length scales but is superior in that it yields smaller RMSEs for longer localization length scales.

scholarly journals Estimating Surface Carbon Fluxes Based on a Local Ensemble Transform Kalman Filter with a Short Assimilation Window and a Long Observation Window

2017 ◽  
Author(s):  
Yun Liu ◽  
Eugenia Kalnay ◽  
Ning Zeng ◽  
Ghassem Asrar ◽  
Zhaohui Chen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Grid Point ◽  
Carbon Fluxes ◽  
Transfer Model ◽  
Observation Window ◽  
Surface Carbon ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Assimilation System

Abstract. We developed a Carbon data assimilation system to estimate the surface carbon fluxes using the Local Ensemble Transform Kalman Filter and atmospheric transfer model of GEOS-Chem driven by the MERRA-1 reanalysis of the meteorological fields based on the Goddard Earth Observing System Model, Version 5 (GEOS-5). This assimilation system is inspired by the method of Kang et al. [2011, 2012], who estimated the surface carbon fluxes in an Observing System Simulation Experiment (OSSE) mode, as evolving parameters in the assimilation of the atmospheric CO2, using a short assimilation window of 6 hours. They included the assimilation of the standard meteorological variables, so that the ensemble provided a measure of the uncertainty in the CO2 transport. After introducing new techniques such as variable localization, and increased observation weights near the surface, they obtained accurate carbon fluxes at grid point resolution. We developed a new version of the LETKF related to the Running-in-Place (RIP) method used to accelerate the spin-up of EnKF data assimilation [Kalnay and Yang, 2010; Wang et al., 2013, Yang et al., 2014]. Like RIP, the new assimilation system uses the no-cost smoothing algorithm for the LETKF [Kalnay et al., 2007b], which allows shifting at no cost the Kalman Filter solution forward or backward within an assimilation window. In the new scheme a long observation window (e.g., 7-days or longer) is used to create an LETKF ensemble at 7-days. Then, the RIP smoother is used to obtain an accurate final analysis at 1-day. This analysis has the advantage of being based on a short assimilation window, which makes it more accurate, and of having been exposed to the future 7-days observations, which accelerates the spin up. The assimilation and observation windows are then shifted forward by one day, and the process is repeated. This reduces significantly the analysis error, suggesting that this method could be used in other data assimilation problems.

scholarly journals Application of ensemble transform data assimilation methods for parameter estimation in reservoir modeling

2018 ◽  
Vol 25 (4) ◽  
pp. 731-746 ◽  
Author(s):  
Sangeetika Ruchi ◽  
Svetlana Dubinkina
Keyword(s):  
Parameter Estimation ◽  
Kalman Filter ◽  
Data Assimilation ◽  
Particle Filter ◽  
Reservoir Modeling ◽  
Model Parameters ◽  
Uncertain Parameters ◽  
Initial Ensemble ◽  
Leading Mode ◽  
Ensemble Transform

Abstract. Over the years data assimilation methods have been developed to obtain estimations of uncertain model parameters by taking into account a few observations of a model state. The most reliable Markov chain Monte Carlo (MCMC) methods are computationally expensive. Sequential ensemble methods such as ensemble Kalman filters and particle filters provide a favorable alternative. However, ensemble Kalman filter has an assumption of Gaussianity. Ensemble transform particle filter does not have this assumption and has proven to be highly beneficial for an initial condition estimation and a small number of parameter estimations in chaotic dynamical systems with non-Gaussian distributions. In this paper we employ ensemble transform particle filter (ETPF) and ensemble transform Kalman filter (ETKF) for parameter estimation in nonlinear problems with 1, 5, and 2500 uncertain parameters and compare them to importance sampling (IS). The large number of uncertain parameters is of particular interest for subsurface reservoir modeling as it allows us to parameterize permeability on the grid. We prove that the updated parameters obtained by ETPF lie within the range of an initial ensemble, which is not the case for ETKF. We examine the performance of ETPF and ETKF in a twin experiment setup, where observations of pressure are synthetically created based on the known values of parameters. For a small number of uncertain parameters (one and five) ETPF performs comparably to ETKF in terms of the mean estimation. For a large number of uncertain parameters (2500) ETKF is robust with respect to the initial ensemble, while ETPF is sensitive due to sampling error. Moreover, for the high-dimensional test problem ETPF gives an increase in the root mean square error after data assimilation is performed. This is resolved by applying distance-based localization, which however deteriorates a posterior estimation of the leading mode by largely increasing the variance due to a combination of less varying localized weights, not keeping the imposed bounds on the modes via the Karhunen–Loeve expansion, and the main variability explained by the leading mode. A possible remedy is instead of applying localization to use only leading modes that are well estimated by ETPF, which demands knowledge of which mode to truncate.

Sign in / Sign up

Export Citation Format

Share Document