scholarly journals Hybrid Gain Data Assimilation Using Variational Corrections in the Subspace Orthogonal to the Ensemble

2020 ◽  
Vol 148 (6) ◽  
pp. 2331-2350 ◽  
Author(s):  
Chih-Chien Chang ◽  
Stephen G. Penny ◽  
Shu-Chih Yang
Keyword(s):  
Data Assimilation ◽  
Large Scale ◽  
Hybrid Methods ◽  
Spatial Scales ◽  
Observation Error ◽  
Online Data ◽  
Background Error ◽  
Hybrid Data Assimilation ◽  
Error Covariance ◽  
Parameter Dependent

Abstract The viability of a parameterless hybrid data assimilation algorithm is investigated. As an alternative to the traditional hybrid covariance scheme, hybrid gain data assimilation (HGDA) was proposed to blend the gain matrix derived from the variational method and the ensemble-based Kalman filter (EnKF). A previously proposed HGDA algorithm uses a two-step process applying the EnKF with a variational update. The algorithm is modified here to limit the variational correction to the subspace orthogonal to the ensemble perturbation subspace without the use of a hybrid weighting parameter, as the optimization of such a parameter is nontrivial. The modified HGDA algorithm is investigated with a quasigeostrophic (QG) model. Results indicate that when the climatological background error covariance matrix B and the observation error covariance R are well estimated, state estimates from the parameterless HGDA are more accurate than the parameter-dependent HGDA. The parameterless HGDA not only has potential advantages over the standard HGDA as an online data assimilation algorithm but can also serve as a valuable diagnostic tool for tuning the B and R matrices. It is also found that in this QG model, the empirically best static B matrix for the stand-alone 3DVAR has high variance at larger spatial scales, which degrades the accuracy of the HGDA systems and may not be the best choice for hybrid methods in general. A comparison of defining the orthogonal subspace globally or locally demonstrates that global orthogonality is more advantageous for stabilizing the hybrid system and maintains large-scale balances.

scholarly journals Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

2011 ◽  
Vol 139 (11) ◽  
pp. 3389-3404 ◽  
Author(s):  
Thomas Milewski ◽  
Michel S. Bourqui
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Climate Model ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

Scale-Dependent Background Error Covariance Localization: Evaluation in a Global Deterministic Weather Forecasting System

2018 ◽  
Vol 146 (5) ◽  
pp. 1367-1381 ◽  
Author(s):  
Jean-François Caron ◽  
Mark Buehner
Keyword(s):  
Data Assimilation ◽  
Weather Forecasting ◽  
Spatial Scales ◽  
Winter Period ◽  
Wind Fields ◽  
Horizontal Scale ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Forecasting System

Abstract Scale-dependent localization (SDL) consists of applying the appropriate (i.e., different) amount of localization to different ranges of background error covariance spatial scales while simultaneously assimilating all of the available observations. The SDL method proposed by Buehner and Shlyaeva for ensemble–variational (EnVar) data assimilation was tested in a 3D-EnVar version of the Canadian operational global data assimilation system. It is shown that a horizontal-scale-dependent horizontal localization leads to implicit vertical-level-dependent, variable-dependent, and location-dependent horizontal localization. The results from data assimilation cycles show that horizontal-scale-dependent horizontal covariance localization is able to improve the forecasts up to day 5 in the Northern Hemisphere extratropical summer period and up to day 7 in the Southern Hemisphere extratropical winter period. In the tropics, use of SDL results in improvements similar to what can be obtained by increasing the uniform amount of spatial localization. An investigation of the dynamical balance in the resulting analysis increments demonstrates that SDL does not further harm the balance between the mass and the rotational wind fields, as compared to the traditional localization approach. Potential future applications for the SDL method are also discussed.

An Ensemble-Based Four-Dimensional Variational Data Assimilation Scheme. Part I: Technical Formulation and Preliminary Test

2008 ◽  
Vol 136 (9) ◽  
pp. 3363-3373 ◽  
Author(s):  
Chengsi Liu ◽  
Qingnong Xiao ◽  
Bin Wang
Keyword(s):  
Data Assimilation ◽  
Preliminary Test ◽  
Background Field ◽  
Variational Data Assimilation ◽  
Water Model ◽  
Observation Error ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance

Abstract Applying a flow-dependent background error covariance (𝗕 matrix) in variational data assimilation has been a topic of interest among researchers in recent years. In this paper, an ensemble-based four-dimensional variational (En4DVAR) algorithm, designed by the authors, is presented that uses a flow-dependent background error covariance matrix constructed by ensemble forecasts and performs 4DVAR optimization to produce a balanced analysis. A great advantage of this En4DVAR design over standard 4DVAR methods is that the tangent linear and adjoint models can be avoided in its formulation and implementation. In addition, it can be easily incorporated into variational data assimilation systems that are already in use at operational centers and among the research community. A one-dimensional shallow water model was used for preliminary tests of the En4DVAR scheme. Compared with standard 4DVAR, the En4DVAR converges well and can produce results that are as good as those with 4DVAR but with far less computation cost in its minimization. In addition, a comparison of the results from En4DVAR with those from other data assimilation schemes [e.g., 3DVAR and ensemble Kalman filter (EnKF)] is made. The results show that the En4DVAR yields an analysis that is comparable to the widely used variational or ensemble data assimilation schemes and can be a promising approach for real-time applications. In addition, experiments were carried out to test the sensitivities of EnKF and En4DVAR, whose background error covariance is estimated from the same ensemble forecasts. The experiments indicated that En4DVAR obtained reasonably sound analysis even with larger observation error, higher observation frequency, and more unbalanced background field.

scholarly journals A New Data Assimilation Scheme: The Space-Expanded Ensemble Localization Kalman Filter

2013 ◽  
Vol 2013 ◽  
pp. 1-6 ◽  
Author(s):  
Hongze Leng ◽  
Junqiang Song ◽  
Fengshun Lu ◽  
Xiaoqun Cao
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Forecast Error ◽  
Three Dimensional ◽  
Full Rank ◽  
Observation Error ◽  
Model Framework ◽  
Background Error ◽  
Error Covariance ◽  
Expanded Ensemble

This study considers a new hybrid three-dimensional variational (3D-Var) and ensemble Kalman filter (EnKF) data assimilation (DA) method in a non-perfect-model framework, named space-expanded ensemble localization Kalman filter (SELKF). In this method, the localization operation is directly applied to the ensemble anomalies with a Schur Product, rather than to the full error covariance of the state in the EnKF. Meanwhile, the correction space of analysis increment is expanded to a space with larger dimension, and the rank of the forecast error covariance is significantly increased. This scheme can reduce the spurious correlations in the covariance and approximate the full-rank background error covariance well. Furthermore, a deterministic scheme is used to generate the analysis anomalies. The results show that the SELKF outperforms the perturbed EnKF given a relatively small ensemble size, especially when the length scale is relatively long or the observation error covariance is relatively small.

scholarly journals A Multiscale Variational Data Assimilation Scheme: Formulation and Illustration

2015 ◽  
Vol 143 (9) ◽  
pp. 3804-3822 ◽  
Author(s):  
Zhijin Li ◽  
James C. McWilliams ◽  
Kayo Ide ◽  
John D. Farrara
Keyword(s):  
High Resolution ◽  
Data Assimilation ◽  
Cost Function ◽  
Spatial Scales ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
The Cost ◽  
Fine Resolution

Abstract A multiscale data assimilation (MS-DA) scheme is formulated for fine-resolution models. A decomposition of the cost function is derived for a set of distinct spatial scales. The decomposed cost function allows for the background error covariance to be estimated separately for the distinct spatial scales, and multi-decorrelation scales to be explicitly incorporated in the background error covariance. MS-DA minimizes the partitioned cost functions sequentially from large to small scales. The multi-decorrelation length scale background error covariance enhances the spreading of sparse observations and prevents fine structures in high-resolution observations from being overly smoothed. The decomposition of the cost function also provides an avenue for mitigating the effects of scale aliasing and representativeness errors that inherently exist in a multiscale system, thus further improving the effectiveness of the assimilation of high-resolution observations. A set of one-dimensional experiments is performed to examine the properties of the MS-DA scheme. Emphasis is placed on the assimilation of patchy high-resolution observations representing radar and satellite measurements, alongside sparse observations representing those from conventional in situ platforms. The results illustrate how MS-DA improves the effectiveness of the assimilation of both these types of observations simultaneously.

scholarly journals Multivariate localization functions for strongly coupled data assimilation in the bivariate Lorenz ’96 system

2021 ◽  
Author(s):  
Zofia Stanley ◽  
Ian Grooms ◽  
William Kleiber
Keyword(s):  
Data Assimilation ◽  
Spatial Scales ◽  
Background Error ◽  
Strongly Coupled ◽  
Localization Function ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Coupled Data Assimilation ◽  
Lorenz 96 ◽  
The Impact

Abstract. Localization is widely used in data assimilation schemes to mitigate the impact of sampling errors on ensemble-derived background error covariance matrices. Strongly coupled data assimilation allows observations in one component of a coupled model to directly impact another component through inclusion of cross-domain terms in the background error covariance matrix. When different components have disparate dominant spatial scales, localization between model domains must properly account for the multiple length scales at play. In this work we develop two new multivariate localization functions, one of which is a multivariate extension of the fifth-order piecewise rational Gaspari-Cohn localization function; the within-component localization functions are standard Gaspari-Cohn with different localization radii while the cross-localization function is newly constructed. The functions produce non-negative definite localization matrices, which are suitable for use in variational data assimilation schemes. We compare the performance of our two new multivariate localization functions to two other multivariate localization functions and to the univariate analogs of all four functions in a simple experiment with the bivariate Lorenz '96 system. In our experiment the multivariate Gaspari-Cohn function leads to better performance than any of the other localization functions.

The Hybrid Local Ensemble Transform Kalman Filter

2014 ◽  
Vol 142 (6) ◽  
pp. 2139-2149 ◽  
Author(s):  
Stephen G. Penny
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Variational Methods ◽  
Hybrid Methods ◽  
Dynamic Error ◽  
Background Error ◽  
Error Covariance ◽  
Small Ensemble ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract Hybrid data assimilation methods combine elements of ensemble Kalman filters (EnKF) and variational methods. While most approaches have focused on augmenting an operational variational system with dynamic error covariance information from an ensemble, this study takes the opposite perspective of augmenting an operational EnKF with information from a simple 3D variational data assimilation (3D-Var) method. A class of hybrid methods is introduced that combines the gain matrices of the ensemble and variational methods, rather than linearly combining the respective background error covariances. A hybrid local ensemble transform Kalman filter (Hybrid-LETKF) is presented in two forms: 1) a traditionally motivated Hybrid/Covariance-LETKF that combines the background error covariance matrices of LETKF and 3D-Var, and 2) a simple-to-implement algorithm called the Hybrid/Mean-LETKF that falls into the new class of hybrid gain methods. Both forms improve analysis errors when using small ensemble sizes and low observation coverage versus either LETKF or 3D-Var used alone. The results imply that for small ensemble sizes, allowing a solution to be found outside of the space spanned by ensemble members provides robustness in both hybrid methods compared to LETKF alone. Finally, the simplicity of the Hybrid/Mean-LETKF design implies that this algorithm can be applied operationally while requiring only minor modifications to an existing operational 3D-Var system.

scholarly journals Comparison of NMC and Ensemble-Based Climatological Background-Error Covariances in an Operational Limited-Area Data Assimilation System

Atmosphere ◽  
2019 ◽  
Vol 10 (10) ◽  
pp. 570
Author(s):  
Antonio Stanesic ◽  
Kristian Horvath ◽  
Endi Keresturi
Keyword(s):  
Data Assimilation ◽  
Spatial Scales ◽  
Specific Humidity ◽  
Estimation Methods ◽  
Lateral Boundary Condition ◽  
Limited Area ◽  
Lateral Boundary ◽  
Background Error ◽  
Error Covariance ◽  
Lateral Boundary Conditions

The evaluation of several climatological background-error covariance matrix (defined as the B matrix) estimation methods was performed using the ALADIN limited-area modeling data-assimilation system at a 4 km horizontal grid spacing. The B matrices compared were derived using the standard National Meteorological Center (NMC) and ensemble-based estimation methods. To test the influence of lateral boundary condition (LBC) perturbations on the characteristics of ensemble-based B matrix, two ensemble prediction systems were established: one used unperturbed lateral boundary conditions (ENS) and another used perturbed lateral boundary conditions (ENSLBC). The characteristics of the three B matrices were compared through a diagnostic comparison, while the influence of the different B matrices on the analysis and quality of the forecast were evaluated for the ENSLBC and NMC matrices. The results showed that the lateral boundary condition perturbations affected all the control variables, while the smallest influence was found for the specific humidity. The diagnostic comparison showed that the ensemble-based estimation method shifted the correlations toward the smaller spatial scales, while the LBC perturbations gave rise to larger spatial scales. The influence on the analysis showed a smaller spatial correlation for the ensemble B matrix compared to that of the NMC, with the most pronounced differences for the specific humidity. The verification of the forecast showed modest improvement for the experiment with the ensemble B matrix. Among the methods tested, the results suggest that the ensemble-based data-assimilation method is the favorable approach for background-error covariance calculation in high-resolution limited-area data assimilation systems.

scholarly journals Multivariate localization functions for strongly coupled data assimilation in the bivariate Lorenz 96 system

2021 ◽  
Vol 28 (4) ◽  
pp. 565-583
Author(s):  
Zofia Stanley ◽  
Ian Grooms ◽  
William Kleiber
Keyword(s):  
Data Assimilation ◽  
Spatial Scales ◽  
Background Error ◽  
Strongly Coupled ◽  
Localization Function ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Coupled Data Assimilation ◽  
Lorenz 96 ◽  
The Impact

Abstract. Localization is widely used in data assimilation schemes to mitigate the impact of sampling errors on ensemble-derived background error covariance matrices. Strongly coupled data assimilation allows observations in one component of a coupled model to directly impact another component through the inclusion of cross-domain terms in the background error covariance matrix. When different components have disparate dominant spatial scales, localization between model domains must properly account for the multiple length scales at play. In this work, we develop two new multivariate localization functions, one of which is a multivariate extension of the fifth-order piecewise rational Gaspari–Cohn localization function; the within-component localization functions are standard Gaspari–Cohn with different localization radii, while the cross-localization function is newly constructed. The functions produce positive semidefinite localization matrices which are suitable for use in both Kalman filters and variational data assimilation schemes. We compare the performance of our two new multivariate localization functions to two other multivariate localization functions and to the univariate and weakly coupled analogs of all four functions in a simple experiment with the bivariate Lorenz 96 system. In our experiments, the multivariate Gaspari–Cohn function leads to better performance than any of the other multivariate localization functions.

Improving Background Error Covariances in a 3D Ensemble–Variational Data Assimilation System for Regional NWP

2019 ◽  
Vol 147 (1) ◽  
pp. 135-151 ◽  
Author(s):  
Jean-François Caron ◽  
Yann Michel ◽  
Thibaut Montmerle ◽  
Étienne Arbogast
Keyword(s):  
Data Assimilation ◽  
Spatial Scales ◽  
Three Dimensional ◽  
Winter Period ◽  
Ensemble Size ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Error Covariance ◽  
Time Lagged ◽  
Recent Formulation

Following the recent development of a three-dimensional ensemble–variational (3DEnVar) data assimilation algorithm for the AROME-France NWP system, this paper examines different approaches to reduce the sampling noise in the ensemble-derived background error covariances in this new scheme without modifying the background ensemble generation strategy. We first examine two variants of scale-dependent localization: one method consists of applying different amounts of localization to different ranges of background error covariance spatial scales, while simultaneously assimilating all of the available observations. Another separate approach uses time-lagged forecasts in order to increase the effective ensemble size, up to a factor of 3 here. This approach of time-lagged forecasts is considered both on its own and together with scale-dependent localization. When the background error covariances are derived from the most recent 25-member ensemble forecasts, the results from data assimilation cycles over a 33-day winter period show that avoiding cross covariances between scales in the scale-dependent localization formulation first proposed by Buehner performs better than the more recent formulation of Buehner and Shlyaeva. However, when increasing the effective ensemble size to 75 members with time-lagged forecasts, the two scale-dependent formulations provide similar forecast improvements overall. It is also found that the lagged-members approach outperforms scale-dependent localization on its own. The largest forecast improvements are obtained when combining the two approaches.

Sign in / Sign up

Export Citation Format

Share Document