A New Reduced Rank Square Root Kalman Filter for Data Assimilation in Mathematical Models

Abstract The ensemble Kalman filter (EnKF) and its deterministic variants, mostly square root filters such as the ensemble transform Kalman filter (ETKF), represent a popular alternative to variational data assimilation schemes and are applied in a wide range of operational and research activities. Their forecast step employs an ensemble integration that fully respects the nonlinear nature of the analyzed system. In the analysis step, they implicitly assume the prior state and observation errors to be Gaussian. Consequently, in nonlinear systems, the analysis mean and covariance are biased, and these filters remain suboptimal. In contrast, the fully nonlinear, non-Gaussian particle filter (PF) only relies on Bayes’s theorem, which guarantees an exact asymptotic behavior, but because of the so-called curse of dimensionality it is exposed to weight collapse. Here, it is shown how to obtain a new analysis ensemble whose mean and covariance exactly match the Bayesian estimates. This is achieved by a deterministic matrix square root transformation of the forecast ensemble, and subsequently a suitable random rotation that significantly contributes to filter stability while preserving the required second-order statistics. The properties and performance of the proposed algorithm are further investigated via a set of experiments. They indicate that such a filter formulation can increase the analysis quality, even for relatively small ensemble sizes, compared to other ensemble filters in nonlinear, non-Gaussian scenarios. Localization enhances the potential applicability of this PF-inspired scheme in larger-dimensional systems. The proposed algorithm, which is fairly easy to implement and computationally efficient, is referred to as the nonlinear ensemble transform filter (NETF).

Download Full-text

Reduced-rank sigma-point Kalman filter for geophysical data assimilation

10.24124/2015/bpgub1095 ◽

2015 ◽

Author(s):

Manoj K. Kizhakkeniyil

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Geophysical Data ◽

Reduced Rank ◽

Sigma Point

Download Full-text

Singular-Vector-Based Covariance Propagation in a Quasigeostrophic Assimilation System

Monthly Weather Review ◽

10.1175/mwr2909.1 ◽

2005 ◽

Vol 133 (5) ◽

pp. 1295-1310 ◽

Cited By ~ 13

Author(s):

Alexander Beck ◽

Martin Ehrendorfer

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Variational Data Assimilation ◽

Dynamic Information ◽

Dynamic Background ◽

Background Error ◽

Singular Vectors ◽

Reduced Rank ◽

The Impact ◽

Static Background

Abstract Variational data assimilation systems require the specification of the covariances of background and observation errors. Although the specification of the background-error covariances has been the subject of intense research, current operational data assimilation systems still rely on essentially static and thus flow-independent background-error covariances. At least theoretically, it is possible to use flow-dependent background-error covariances in four-dimensional variational data assimilation (4DVAR) through exploiting the connection between variational data assimilation and estimation theory. This paper reports on investigations concerning the impact of flow-dependent background-error covariances in an idealized 4DVAR system that, based on quasigeostrophic dynamics, assimilates artificial observations. The main emphasis is placed on quantifying the improvement in analysis quality that is achievable in 4DVAR through the use of flow-dependent background-error covariances. Flow dependence is achieved through dynamical error-covariance evolution based on singular vectors in a reduced-rank approach, referred to as reduced-rank Kalman filter (RRKF). The RRKF yields partly dynamic background-error covariances through blending static and dynamic information, where the dynamic information is obtained from error evolution in a subspace of dimension k (defined here through the singular vectors) that may be small compared to the dimension of the model’s phase space n, which is equal to 1449 in the system investigated here. The results show that the use of flow-dependent background-error covariances based on the RRKF leads to improved analyses compared to a system using static background-error statistics. That latter system uses static background-error covariances that are carefully tuned given the model dynamics and the observational information available. It is also shown that the performance of the RRKF approaches the performance of the extended Kalman filter, as k approaches n. Results therefore support the hypothesis that significant analysis improvement is possible through the use of flow-dependent background-error covariances given that a sufficiently large number (here on the order of n/10) of singular vectors is used.

Download Full-text

How can Thermospheric Neutral Density (TND) estimates from CHAMP and GRACE accelerometer observations be used to improve empirical models?

10.5194/egusphere-egu2020-369 ◽

2020 ◽

Author(s):

Ehsan Forootan ◽

Saeed Farzaneh ◽

Mona Kosary ◽

Maike Schumacher

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Orbit Determination ◽

Ensemble Kalman Filter ◽

Precise Orbit Determination ◽

Accurate Estimation ◽

Neutral Density ◽

Square Root ◽

Precise Orbit ◽

Leo Satellites

An accurate estimation of the Thermospheric Neutral Density (TND) is important to compute drag forces acting on Low-Earth-Orbit (LEO) satellites and debris. Empirical thermospheric models are often used to compute TNDs (along-track of LEO satellites) for the Precise Orbit Determination (POD) experiments. However, recent studies indicate that the TNDs of available models do not perfectly reproduce TNDs derived from accelerometer observations. In this study, we use TND estimates from the Challenging Minisatellite Payload (CHAMP) and Gravity Recovery and Climate Experiment (GRACE) missions and merge them with the NRLMSISE00 from the Mass Spectrometer and Incoherent Scatter family. The integration is implemented by applying a simultaneous Calibration and Data Assimilation (C/DA) technique. The application of C/DA is advantageous since it uses model equation to interpolate and extrapolate TNDs that are not covered by CHAMP and GRACE. It also modifies the model's selected parameters to simulate TNDs that are closer to those of CHAMP and GRACE. The C/DA of this study is implemented daily using CHAMP- and/or GRACE-TNDs, while using the Ensemble Kalman Filter (EnKF) and Ensemble Square-Root Kalman Filter (EnSRF) as merger. Compared to the original model, on average, we found 27% (in the range of 2% to 56%) improvements in the estimation of TNDs. In addition, the results of the C/DA are compared with the TND outputs of the JB2008 model along the CHAMP and GRACE orbits, whose results indicate that the daily C/DA outputs are 60% closer to the observed TNDs (that are not used for the C/DA). Overall, our assessment indicates that EnSRF results in more realistic TND simulation and prediction compared to those derived from EnKF. We show that the improved TND estimates of this study will be beneficial for Precise Orbit Determination (POD) studies. &#160;Keywords: Thermosphere, Calibration and Data Assimilation (C/DA), NRLMSISE00, Ensemble Kalman Filter (EnKF), Ensemble Square-Root Kalman Filter (EnSRF)

Download Full-text

On the construction of a reduced rank square-root Kalman filter for efficient uncertainty propagation

Future Generation Computer Systems ◽

10.1016/j.future.2004.03.005 ◽

2005 ◽

Vol 21 (7) ◽

pp. 1047-1055 ◽

Cited By ~ 4

Author(s):

Dimitri Treebushny ◽

Henrik Madsen

Keyword(s):

Kalman Filter ◽

Uncertainty Propagation ◽

Square Root ◽

Reduced Rank

Download Full-text

THE REDUCED RANK TRANSFORM SQUARE ROOT FILTER FOR DATA ASSIMILATION

IFAC Proceedings Volumes ◽

10.3182/20060329-3-au-2901.00202 ◽

2006 ◽

Vol 39 (1) ◽

pp. 1252-1257 ◽

Cited By ~ 1

Author(s):

S. Gillijns ◽

D.S. Bernstein ◽

B. De Moor

Keyword(s):

Data Assimilation ◽

Square Root ◽

Reduced Rank ◽

Rank Transform ◽

Square Root Filter

Download Full-text

Reduced-Rank Sigma-Point Kalman Filter and Its Application in ENSO Model

Journal of Atmospheric and Oceanic Technology ◽

10.1175/jtech-d-13-00172.1 ◽

2014 ◽

Vol 31 (10) ◽

pp. 2350-2366 ◽

Cited By ~ 3

Author(s):

K. K. Manoj ◽

Youmin Tang ◽

Ziwang Deng ◽

Dake Chen ◽

Yanjie Cheng

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Unscented Kalman Filter ◽

Southern Oscillation ◽

Estimation Accuracy ◽

Dimensional System ◽

Square Root ◽

Reduced Rank ◽

Sigma Point ◽

Enso Events

Abstract The huge computational expense has been a main challenge while applying the sigma-point unscented Kalman filter (SPUKF) to a high-dimensional system. This study focuses on this issue and presents two methods to construct a reduced-rank sigma-point unscented Kalman filter (RRSPUKF). Both techniques employ the truncated singular value decomposition (TSVD) to factorize the covariance matrix and reduce its rank through truncation. The reduced-rank square root matrix is used to select the most important sigma points that can retain the main statistical features of the original sigma points. In the first technique, TSVD is applied on the covariance matrix constructed in the data space [RRSPUKF(D)], whereas in the second technique TSVD is applied on the covariance matrix constructed in the ensemble space [RRSPUKF(E)]. The two methods are applied to a realistic El Niño–Southern Oscillation (ENSO) prediction model [Lamont-Doherty Earth Observatory model, version 5 (LDEO5)] to assimilate the sea surface temperature (SST) anomalies. The results show that both the methods are more computationally efficient than the full-rank SPUKF, in spite of losing some estimation accuracy. When the truncation reaches a trade-off between cost expense and estimation accuracy, both methods are able to analyze the phase and intensity of all major ENSO events from 1971 to 2001 with comparable estimation accuracy. Furthermore, the RRSPUKF is compared against ensemble square root filter (EnSRF), showing that the overall analysis skill of RRSPUKF and EnSRF are comparable to each other, but the former is more robust than the latter.

Download Full-text

Implications of the Form of the Ensemble Transformation in the Ensemble Square Root Filters

Monthly Weather Review ◽

10.1175/2007mwr2021.1 ◽

2008 ◽

Vol 136 (3) ◽

pp. 1042-1053 ◽

Cited By ~ 109

Author(s):

Pavel Sakov ◽

Peter R. Oke

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Symmetric Solution ◽

Square Root ◽

The Mean ◽

Transform Matrix ◽

Small Models ◽

Ensemble Transform

Abstract This paper considers implications of different forms of the ensemble transformation in the ensemble square root filters (ESRFs) for the performance of ESRF-based data assimilation systems. It highlights the importance of using mean-preserving solutions for the ensemble transform matrix (ETM). The paper shows that an arbitrary mean-preserving ETM can be represented as a product of the symmetric solution and an orthonormal mean-preserving matrix. The paper also introduces a new flavor of ESRF, referred to as ESRF with mean-preserving random rotations. To investigate the performance of different solutions for the ETM in ESRFs, experiments with two small models are conducted. In these experiments, the performances of two mean-preserving solutions, two non-mean-preserving solutions, and a traditional ensemble Kalman filter with perturbed observations are compared. The experiments show a significantly better performance of the mean-preserving solutions for the ETM in ESRFs compared to non-mean-preserving solutions. They also show that applying the mean-preserving random rotations prevents the buildup of ensemble outliers in ESRF-based data assimilation systems.

Download Full-text

A Hybrid Kalman Filter Algorithm for Large-Scale Atmospheric Chemistry Data Assimilation

Monthly Weather Review ◽

10.1175/mwr3269.1 ◽

2007 ◽

Vol 135 (1) ◽

pp. 140-151 ◽

Cited By ~ 13

Author(s):

R. G. Hanea ◽

G. J. M. Velders ◽

A. J. Segers ◽

M. Verlaan ◽

A. W. Heemink

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Atmospheric Chemistry ◽

Large Scale ◽

Transport Model ◽

Reduced Rank ◽

Nonlinear Properties ◽

Rank Approximation ◽

Kalman Filter Algorithm ◽

Better Than

Abstract In the past, a number of algorithms have been introduced to solve data assimilation problems for large-scale applications. Here, several Kalman filters, coupled to the European Operational Smog (EUROS) atmospheric chemistry transport model, are used to estimate the ozone concentrations in the boundary layer above Europe. Two Kalman filter algorithms, the reduced-rank square root (RRSQRT) and the ensemble Kalman filter (EnKF), were implemented in a prior study. To combine the best features of these two filters, a hybrid filter was constructed by making use of the reduced-rank approximation of the covariance matrix as a variance reducer for the EnKF. This hybrid algorithm, complementary orthogonal subspace filter for efficient ensembles (COFFEE), is coupled to the EUROS model. The performance of all algorithms is compared in terms of residual errors and number of EUROS model evaluations. The COFFEE results score somewhere between the EnKF and RRSQRT results for less than approximately 30 model evaluations. For more than approximately 30 model evaluations, the COFFEE results are, in all cases, better than the EnKF and RRSQRT results. The results of the COFFEE simulations with more than about 60 model evaluations proved to be significantly better than all the EnKF and RRSQRT simulations (even better than those with 100 and 200 modes or ensemble members). The performance of both the EnKF- and RRSQRT-type filters is affected by the nonlinear properties of the model and observation operator, because both rely on linearization to some extent. To further study this aspect, several measures of nonlinearity were calculated and linked with the performance of these algorithms.

Download Full-text