A Multiscale Local Gain Form Ensemble Transform Kalman Filter (MLGETKF)

2020 ◽  
Author(s):  
Xuguang Wang ◽  
Hristo G. Chipilski ◽  
Craig H. Bishop ◽  
Elizabeth Satterfield ◽  
Nancy Baker ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Lead Time ◽  
Potential Temperature ◽  
Scale Model ◽  
Small Scale ◽  
Space Localization ◽  
Ensemble Mean ◽  
The Mean ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

AbstractA new multiscale, ensemble-based data assimilation (DA) method, MLGETKF (Multiscale Local Gain Form Ensemble Transform Kalman Filter), is introduced. MLGETKF allows simultaneous update of multiple scales for both the mean and ensemble perturbations through assimilating all observations at once. MLGETKF performs DA in independent local volumes, which lends the algorithm a high degree of computational scalability. The multiscale analysis is enabled through the rapid creation of many pseudo ensemble perturbations via a multiscale ensemble modulation procedure. The Kalman gain that is used to update the raw background ensemble mean and perturbations is based on this modulated ensemble, which intrinsically includes multi-scale model space localization.Experiments with a non-cycled statistical model show that the full background covariance estimated by MLGETKF more accurately resembles the shape of the true covariance than a scale-unaware localization. The mean analysis from the best-performing MLGETKF is statistically significantly more accurate than the best performing scale unaware LGETKF. The accuracy of the MLGETKF analysis is more sensitive to small-scale band localization radius than large-scale band. MLGETKF is further examined in a cycling DA context with a Surface Quasi-Geostrophic model. The root-mean-square potential temperature analysis error of the best performing MLGETKF is 17.2% lower than that of the best-performing LGETKF. MLGETKF reduces analysis errors measured in kinetic energy spectra space by 30-80% relative to LGETKF with the largest improvement at large scales. MLGETKF deterministic and ensemble mean forecasts are more accurate than LGETKF for full and large scales up to 5-6 day lead-time and for small scales up to 3-4 day lead-time, gaining 12-hour ~ 1-day of predictability.

Local Ensemble Transform Kalman Filter with Cross Validation

2020 ◽  
Vol 148 (6) ◽  
pp. 2265-2282 ◽  
Author(s):  
Mark Buehner
Keyword(s):  
Kalman Filter ◽  
Cross Validation ◽  
Ensemble Spread ◽  
Stochastic Version ◽  
Highly Nonlinear ◽  
Ensemble Mean ◽  
Forecast Models ◽  
Ensemble Transform Kalman Filter ◽  
The Individual ◽  
Ensemble Transform

Abstract Many ensemble data assimilation (DA) approaches suffer from the so-called inbreeding problem. As a consequence, there is an excessive reduction in ensemble spread by the DA procedure, causing the analysis ensemble spread to systematically underestimate the uncertainty of the ensemble mean analysis. The stochastic EnKF used for operational NWP in Canada largely avoids this problem by applying cross validation, that is, using an independent subset of ensemble members for updating each member. The goal of the present study is to evaluate two new variations of the local ensemble transform Kalman filter (LETKF) that also incorporate cross validation. In idealized numerical experiments with Gaussian-distributed background ensembles, the two new LETKF approaches are shown to produce reliable analysis ensembles such that the ensemble spread closely matches the uncertainty of the ensemble mean, without any ensemble inflation. In ensemble DA experiments with highly nonlinear idealized forecast models, the deterministic version of the LETKF with cross validation quickly diverges, but the stochastic version produces better results, nearly identical to the stochastic EnKF with cross validation. In the context of a regional NWP system, ensemble DA experiments are performed with the two new LETKF-based approaches with cross validation, the standard LETKF, and the stochastic EnKF. All approaches with cross validation produce similar ensemble spread at the first analysis time, though the amplitude of the changes to the individual members is larger with the stochastic approaches. Over the 10-day period of the experiments, the fit of the ensemble mean background state to radiosonde observations is statistically indistinguishable for all approaches evaluated.

scholarly journals Handling Nonlinearity in an Ensemble Kalman Filter: Experiments with the Three-Variable Lorenz Model

2012 ◽  
Vol 140 (8) ◽  
pp. 2628-2646 ◽  
Author(s):  
Shu-Chih Yang ◽  
Eugenia Kalnay ◽  
Brian Hunt
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
Linear Models ◽  
Low Frequency ◽  
Lorenz Model ◽  
Outer Loop ◽  
Nonlinear Growth ◽  
Ensemble Mean ◽  
Ensemble Transform Kalman Filter ◽  
The One

Abstract An ensemble Kalman filter (EnKF) is optimal only for linear models because it assumes Gaussian distributions. A new type of outer loop, different from the one used in 3D and 4D variational data assimilation (Var), is proposed for EnKF to improve its ability to handle nonlinear dynamics, especially for long assimilation windows. The idea of the “running in place” (RIP) algorithm is to increase the observation influence by reusing observations when there is strong nonlinear error growth, and thus improve the ensemble mean and perturbations within the local ensemble transform Kalman filter (LETKF) framework. The “quasi-outer-loop” (QOL) algorithm, proposed here as a simplified version of RIP, aims to improve the ensemble mean so that ensemble perturbations are centered at a more accurate state. The performances of LETKF–RIP and LETKF–QOL in the presence of nonlinearities are tested with the three-variable Lorenz model. Results show that RIP and QOL allow LETKF to use longer assimilation windows with significant improvement of the analysis accuracy during periods of high nonlinear growth. For low-frequency observations (every 25 time steps, leading to long assimilation windows), and using the optimal inflation, the standard LETKF RMS error is 0.68, whereas for QOL and RIP the RMS errors are 0.47 and 0.35, respectively. This can be compared to the best 4D-Var analysis error of 0.53, obtained by using both the optimal long assimilation windows (75 time steps) and quasi-static variational analysis.

scholarly journals Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization

2017 ◽  
Vol 145 (11) ◽  
pp. 4575-4592 ◽  
Author(s):  
Craig H. Bishop ◽  
Jeffrey S. Whitaker ◽  
Lili Lei
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Satellite Data ◽  
Localization Length ◽  
Model Space ◽  
Ensemble Size ◽  
Length Scales ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Expanded Ensemble

To ameliorate suboptimality in ensemble data assimilation, methods have been introduced that involve expanding the ensemble size. Such expansions can incorporate model space covariance localization and/or estimates of climatological or model error covariances. Model space covariance localization in the vertical overcomes problematic aspects of ensemble-based satellite data assimilation. In the case of the ensemble transform Kalman filter (ETKF), the expanded ensemble size associated with vertical covariance localization would also enable the simultaneous update of entire vertical columns of model variables from hyperspectral and multispectral satellite sounders. However, if the original formulation of the ETKF were applied to an expanded ensemble, it would produce an analysis ensemble that was the same size as the expanded forecast ensemble. This article describes a variation on the ETKF called the gain ETKF (GETKF) that takes advantage of covariances from the expanded ensemble, while producing an analysis ensemble that has the required size of the unexpanded forecast ensemble. The approach also yields an inflation factor that depends on the localization length scale that causes the GETKF to perform differently to an ensemble square root filter (EnSRF) using the same expanded ensemble. Experimentation described herein shows that the GETKF outperforms a range of alternative ETKF-based solutions to the aforementioned problems. In cycling data assimilation experiments with a newly developed storm-track version of the Lorenz-96 model, the GETKF analysis root-mean-square error (RMSE) matches the EnSRF RMSE at shorter than optimal localization length scales but is superior in that it yields smaller RMSEs for longer localization length scales.

scholarly journals Estimating surface carbon fluxes based on a local ensemble transform Kalman filter with a short assimilation window and a long observation window: an observing system simulation experiment test in GEOS-Chem 10.1

2019 ◽  
Vol 12 (7) ◽  
pp. 2899-2914
Author(s):  
Yun Liu ◽  
Eugenia Kalnay ◽  
Ning Zeng ◽  
Ghassem Asrar ◽  
Zhaohui Chen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Simulation Experiment ◽  
System Simulation ◽  
Carbon Fluxes ◽  
Observation Window ◽  
Surface Carbon ◽  
Ensemble Transform Kalman Filter ◽  
Observing System Simulation Experiment ◽  
Ensemble Transform ◽  
Assimilation System

Abstract. We developed a carbon data assimilation system to estimate surface carbon fluxes using the local ensemble transform Kalman filter (LETKF) and atmospheric transport model GEOS-Chem driven by the MERRA-1 reanalysis of the meteorological field based on the Goddard Earth Observing System model, version 5 (GEOS-5). This assimilation system is inspired by the method of Kang et al. (2011, 2012), who estimated the surface carbon fluxes in an observing system simulation experiment (OSSE) as evolving parameters in the assimilation of the atmospheric CO2, using a short assimilation window of 6 h. They included the assimilation of the standard meteorological variables, so that the ensemble provided a measure of the uncertainty in the CO2 transport. After introducing new techniques such as “variable localization”, and increased observation weights near the surface, they obtained accurate surface carbon fluxes at grid-point resolution. We developed a new version of the local ensemble transform Kalman filter related to the “running-in-place” (RIP) method used to accelerate the spin-up of ensemble Kalman filter (EnKF) data assimilation (Kalnay and Yang, 2010; Wang et al., 2013; Yang et al., 2012). Like RIP, the new assimilation system uses the “no cost smoothing” algorithm for the LETKF (Kalnay et al., 2007b), which allows shifting the Kalman filter solution forward or backward within an assimilation window at no cost. In the new scheme a long “observation window” (e.g., 7 d or longer) is used to create a LETKF ensemble at 7 d. Then, the RIP smoother is used to obtain an accurate final analysis at 1 d. This new approach has the advantage of being based on a short assimilation window, which makes it more accurate, and of having been exposed to the future 7 d observations, which improves the analysis and accelerates the spin-up. The assimilation and observation windows are then shifted forward by 1 d, and the process is repeated. This reduces significantly the analysis error, suggesting that the newly developed assimilation method can be used with other Earth system models, especially in order to make greater use of observations in conjunction with models.

scholarly journals Estimating Surface Carbon Fluxes Based on a Local Ensemble Transform Kalman Filter with a Short Assimilation Window and a Long Observation Window

2017 ◽  
Author(s):  
Yun Liu ◽  
Eugenia Kalnay ◽  
Ning Zeng ◽  
Ghassem Asrar ◽  
Zhaohui Chen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Grid Point ◽  
Carbon Fluxes ◽  
Transfer Model ◽  
Observation Window ◽  
Surface Carbon ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Assimilation System

Abstract. We developed a Carbon data assimilation system to estimate the surface carbon fluxes using the Local Ensemble Transform Kalman Filter and atmospheric transfer model of GEOS-Chem driven by the MERRA-1 reanalysis of the meteorological fields based on the Goddard Earth Observing System Model, Version 5 (GEOS-5). This assimilation system is inspired by the method of Kang et al. [2011, 2012], who estimated the surface carbon fluxes in an Observing System Simulation Experiment (OSSE) mode, as evolving parameters in the assimilation of the atmospheric CO2, using a short assimilation window of 6 hours. They included the assimilation of the standard meteorological variables, so that the ensemble provided a measure of the uncertainty in the CO2 transport. After introducing new techniques such as variable localization, and increased observation weights near the surface, they obtained accurate carbon fluxes at grid point resolution. We developed a new version of the LETKF related to the Running-in-Place (RIP) method used to accelerate the spin-up of EnKF data assimilation [Kalnay and Yang, 2010; Wang et al., 2013, Yang et al., 2014]. Like RIP, the new assimilation system uses the no-cost smoothing algorithm for the LETKF [Kalnay et al., 2007b], which allows shifting at no cost the Kalman Filter solution forward or backward within an assimilation window. In the new scheme a long observation window (e.g., 7-days or longer) is used to create an LETKF ensemble at 7-days. Then, the RIP smoother is used to obtain an accurate final analysis at 1-day. This analysis has the advantage of being based on a short assimilation window, which makes it more accurate, and of having been exposed to the future 7-days observations, which accelerates the spin up. The assimilation and observation windows are then shifted forward by one day, and the process is repeated. This reduces significantly the analysis error, suggesting that this method could be used in other data assimilation problems.

Sign in / Sign up

Export Citation Format

Share Document