The Hybrid Local Ensemble Transform Kalman Filter

2014 ◽  
Vol 142 (6) ◽  
pp. 2139-2149 ◽  
Author(s):  
Stephen G. Penny
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Variational Methods ◽  
Hybrid Methods ◽  
Dynamic Error ◽  
Background Error ◽  
Error Covariance ◽  
Small Ensemble ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract Hybrid data assimilation methods combine elements of ensemble Kalman filters (EnKF) and variational methods. While most approaches have focused on augmenting an operational variational system with dynamic error covariance information from an ensemble, this study takes the opposite perspective of augmenting an operational EnKF with information from a simple 3D variational data assimilation (3D-Var) method. A class of hybrid methods is introduced that combines the gain matrices of the ensemble and variational methods, rather than linearly combining the respective background error covariances. A hybrid local ensemble transform Kalman filter (Hybrid-LETKF) is presented in two forms: 1) a traditionally motivated Hybrid/Covariance-LETKF that combines the background error covariance matrices of LETKF and 3D-Var, and 2) a simple-to-implement algorithm called the Hybrid/Mean-LETKF that falls into the new class of hybrid gain methods. Both forms improve analysis errors when using small ensemble sizes and low observation coverage versus either LETKF or 3D-Var used alone. The results imply that for small ensemble sizes, allowing a solution to be found outside of the space spanned by ensemble members provides robustness in both hybrid methods compared to LETKF alone. Finally, the simplicity of the Hybrid/Mean-LETKF design implies that this algorithm can be applied operationally while requiring only minor modifications to an existing operational 3D-Var system.

scholarly journals The High-Rank Ensemble Transform Kalman Filter

2019 ◽  
Vol 147 (8) ◽  
pp. 3025-3043 ◽  
Author(s):  
Bo Huang ◽  
Xuguang Wang ◽  
Craig H. Bishop
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
High Rank ◽  
Background Error ◽  
Localization Method ◽  
Error Covariance ◽  
Higher Rank ◽  
Small Ensemble ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract The ensemble Kalman filter is typically implemented either by applying the localization on the background error covariance matrix (B-localization) or by inflating the observation error variances (R-localization). A mathematical demonstration suggests that for the same effective localization function, the background error covariance matrix from the B-localization method shows a higher rank than the R-localization method. The B-localization method is realized in the ensemble transform Kalman filter (ETKF) by extending the background ensemble perturbations through modulation (MP-localization). Specifically, the modulation functions are constructed from the leading eigenvalues and eigenvectors of the original B-localization matrix. Because of its higher rank than the classic R-localized ETKF, the B-/MP-localized ETKF is termed as the high-rank ETKF (HETKF). The performances of the HETKF and R-localized ETKF were compared through cycled data assimilation experiments using the Lorenz model II. The results show that the HETKF outperforms the R-localized ETKF especially for a small ensemble. The improved analysis in the HETKF is likely associated with the higher rank from the B-/MP-localization method, since its higher rank is expected to contribute more positively to alleviating the rank deficiency issue and thus improve the analysis for a small ensemble. The HETKF is less sensitive to the localization length scales and inflation factors. Furthermore, the experiments suggest that the above conclusion comparing the HETKF and R-localized ETKF does not depend on how the analyzed ensemble perturbations are subselected in the HETKF.

scholarly journals Univariate and Multivariate Assimilation of AIRS Humidity Retrievals with the Local Ensemble Transform Kalman Filter

2009 ◽  
Vol 137 (11) ◽  
pp. 3918-3932 ◽  
Author(s):  
Junjie Liu ◽  
Hong Li ◽  
Eugenia Kalnay ◽  
Eric J. Kostelich ◽  
Istvan Szunyogh
Keyword(s):  
Kalman Filter ◽  
Precipitable Water ◽  
The Other ◽  
Specific Humidity ◽  
Atmospheric Infrared Sounder ◽  
Background Error ◽  
Error Covariance ◽  
Ensemble Transform Kalman Filter ◽  
Forecast Time ◽  
Ensemble Transform

Abstract This study uses the local ensemble transform Kalman filter to assimilate Atmospheric Infrared Sounder (AIRS) specific humidity retrievals with pseudo relative humidity (pseudo-RH) as the observation variable. Three approaches are tested: (i) updating specific humidity with observations other than specific humidity (“passive q”), (ii) updating specific humidity only with humidity observations (“univariate q”), and (iii) assimilating the humidity and the other observations together (“multivariate q”). This is the first time that the performance of the univariate and multivariate assimilation of q is compared within an ensemble Kalman filter framework. The results show that updating the humidity analyses by either AIRS specific humidity retrievals or nonhumidity observations improves both the humidity and wind analyses. The improvement with the multivariate-q experiment is by far the largest for all dynamical variables at both analysis and forecast time, indicating that the interaction between the specific humidity and the other dynamical variables through the background error covariance during data assimilation process yields more balanced analysis fields. In the univariate assimilation of q, the humidity interacts with the other dynamical variables only through the forecast process. The univariate assimilation produces more accurate humidity analyses than those obtained when no humidity observations are assimilated, but it does not improve the accuracy of the zonal wind analyses. The 6-h total column precipitable water forecast also benefits from the improved humidity analyses, with the multivariate q experiment having the largest improvement.

scholarly journals Comparison of Local Ensemble Transform Kalman Filter, 3DVAR, and 4DVAR in a Quasigeostrophic Model

2009 ◽  
Vol 137 (2) ◽  
pp. 693-709 ◽  
Author(s):  
Shu-Chih Yang ◽  
Matteo Corazza ◽  
Alberto Carrassi ◽  
Eugenia Kalnay ◽  
Takemasa Miyoshi
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Channel Model ◽  
Variational Data Assimilation ◽  
Ensemble Averaging ◽  
Background Error ◽  
Advantages And Disadvantages ◽  
Bred Vectors ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract Local ensemble transform Kalman filter (LETKF) data assimilation, three-dimensional variational data assimilation (3DVAR), and four-dimensional variational data assimilation (4DVAR) schemes are implemented in a quasigeostrophic channel model. Their advantages and disadvantages are compared to assess their use in practical applications. LETKF and 4DVAR, which take into account the flow-dependent errors, outperform 3DVAR under a perfect model scenario. Given the same observations, LETKF produces more accurate analyses than 4DVAR with a 12-h window by effectively correcting the fast-growing errors with the flow-dependent background error covariance. Even though 4DVAR performance benefits substantially from using a longer assimilation window, LETKF is also able to achieve a satisfactory accuracy compared to the 24-h 4DVAR analyses. It is shown that the advantage of the LETKF over 3DVAR is a result of both the ensemble averaging and the information about the “errors of the day” provided by the ensemble. The analysis corrections at the end of the 12-h assimilation window are similar for LETKF and the 12-h window 4DVAR, and they both resemble bred vectors. At the beginning of the assimilation window, LETKF analysis corrections obtained using a no-cost smoother also resemble the corresponding bred vectors, whereas the 4DVAR corrections are significantly different with much larger horizontal scales.

scholarly journals The local ensemble transform Kalman filter and the running-in-place algorithm applied to a global ocean general circulation model

2013 ◽  
Vol 20 (6) ◽  
pp. 1031-1046 ◽  
Author(s):  
S. G. Penny ◽  
E. Kalnay ◽  
J. A. Carton ◽  
B. R. Hunt ◽  
K. Ide ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
General Circulation ◽  
Initial Conditions ◽  
Circulation Model ◽  
Global Ocean ◽  
Observation Data ◽  
Background Error ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract. The most widely used methods of data assimilation in large-scale oceanography, such as the Simple Ocean Data Assimilation (SODA) algorithm, specify the background error covariances and thus are unable to refine the weights in the assimilation as the circulation changes. In contrast, the more computationally expensive Ensemble Kalman Filters (EnKF) such as the Local Ensemble Transform Kalman Filter (LETKF) use an ensemble of model forecasts to predict changes in the background error covariances and thus should produce more accurate analyses. The EnKFs are based on the approximation that ensemble members reflect a Gaussian probability distribution that is transformed linearly during the forecast and analysis cycle. In the presence of nonlinearity, EnKFs can gain from replacing each analysis increment by a sequence of smaller increments obtained by recursively applying the forecast model and data assimilation procedure over a single analysis cycle. This has led to the development of the "running in place" (RIP) algorithm by Kalnay and Yang (2010) and Yang et al. (2012a,b) in which the weights computed at the end of each analysis cycle are used recursively to refine the ensemble at the beginning of the analysis cycle. To date, no studies have been carried out with RIP in a global domain with real observations. This paper provides a comparison of the aforementioned assimilation methods in a set of experiments spanning seven years (1997–2003) using identical forecast models, initial conditions, and observation data. While the emphasis is on understanding the similarities and differences between the assimilation methods, comparisons are also made to independent ocean station temperature, salinity, and velocity time series, as well as ocean transports, providing information about the absolute error of each. Comparisons to independent observations are similar for the assimilation methods but the observation-minus-background temperature differences are distinctly lower for LETKF and RIP. The results support the potential for LETKF to improve the quality of ocean analyses on the space and timescales of interest for seasonal prediction and for RIP to accelerate the spin up of the system.

scholarly journals Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization

2017 ◽  
Vol 145 (11) ◽  
pp. 4575-4592 ◽  
Author(s):  
Craig H. Bishop ◽  
Jeffrey S. Whitaker ◽  
Lili Lei
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Satellite Data ◽  
Localization Length ◽  
Model Space ◽  
Ensemble Size ◽  
Length Scales ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Expanded Ensemble

To ameliorate suboptimality in ensemble data assimilation, methods have been introduced that involve expanding the ensemble size. Such expansions can incorporate model space covariance localization and/or estimates of climatological or model error covariances. Model space covariance localization in the vertical overcomes problematic aspects of ensemble-based satellite data assimilation. In the case of the ensemble transform Kalman filter (ETKF), the expanded ensemble size associated with vertical covariance localization would also enable the simultaneous update of entire vertical columns of model variables from hyperspectral and multispectral satellite sounders. However, if the original formulation of the ETKF were applied to an expanded ensemble, it would produce an analysis ensemble that was the same size as the expanded forecast ensemble. This article describes a variation on the ETKF called the gain ETKF (GETKF) that takes advantage of covariances from the expanded ensemble, while producing an analysis ensemble that has the required size of the unexpanded forecast ensemble. The approach also yields an inflation factor that depends on the localization length scale that causes the GETKF to perform differently to an ensemble square root filter (EnSRF) using the same expanded ensemble. Experimentation described herein shows that the GETKF outperforms a range of alternative ETKF-based solutions to the aforementioned problems. In cycling data assimilation experiments with a newly developed storm-track version of the Lorenz-96 model, the GETKF analysis root-mean-square error (RMSE) matches the EnSRF RMSE at shorter than optimal localization length scales but is superior in that it yields smaller RMSEs for longer localization length scales.

scholarly journals Estimating Surface Carbon Fluxes Based on a Local Ensemble Transform Kalman Filter with a Short Assimilation Window and a Long Observation Window

2017 ◽  
Author(s):  
Yun Liu ◽  
Eugenia Kalnay ◽  
Ning Zeng ◽  
Ghassem Asrar ◽  
Zhaohui Chen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Grid Point ◽  
Carbon Fluxes ◽  
Transfer Model ◽  
Observation Window ◽  
Surface Carbon ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Assimilation System

Abstract. We developed a Carbon data assimilation system to estimate the surface carbon fluxes using the Local Ensemble Transform Kalman Filter and atmospheric transfer model of GEOS-Chem driven by the MERRA-1 reanalysis of the meteorological fields based on the Goddard Earth Observing System Model, Version 5 (GEOS-5). This assimilation system is inspired by the method of Kang et al. [2011, 2012], who estimated the surface carbon fluxes in an Observing System Simulation Experiment (OSSE) mode, as evolving parameters in the assimilation of the atmospheric CO2, using a short assimilation window of 6 hours. They included the assimilation of the standard meteorological variables, so that the ensemble provided a measure of the uncertainty in the CO2 transport. After introducing new techniques such as variable localization, and increased observation weights near the surface, they obtained accurate carbon fluxes at grid point resolution. We developed a new version of the LETKF related to the Running-in-Place (RIP) method used to accelerate the spin-up of EnKF data assimilation [Kalnay and Yang, 2010; Wang et al., 2013, Yang et al., 2014]. Like RIP, the new assimilation system uses the no-cost smoothing algorithm for the LETKF [Kalnay et al., 2007b], which allows shifting at no cost the Kalman Filter solution forward or backward within an assimilation window. In the new scheme a long observation window (e.g., 7-days or longer) is used to create an LETKF ensemble at 7-days. Then, the RIP smoother is used to obtain an accurate final analysis at 1-day. This analysis has the advantage of being based on a short assimilation window, which makes it more accurate, and of having been exposed to the future 7-days observations, which accelerates the spin up. The assimilation and observation windows are then shifted forward by one day, and the process is repeated. This reduces significantly the analysis error, suggesting that this method could be used in other data assimilation problems.

Sign in / Sign up

Export Citation Format

Share Document