scholarly journals Singular-Vector-Based Covariance Propagation in a Quasigeostrophic Assimilation System

2005 ◽  
Vol 133 (5) ◽  
pp. 1295-1310 ◽  
Author(s):  
Alexander Beck ◽  
Martin Ehrendorfer
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Variational Data Assimilation ◽  
Dynamic Information ◽  
Dynamic Background ◽  
Background Error ◽  
Singular Vectors ◽  
Reduced Rank ◽  
The Impact ◽  
Static Background

Abstract Variational data assimilation systems require the specification of the covariances of background and observation errors. Although the specification of the background-error covariances has been the subject of intense research, current operational data assimilation systems still rely on essentially static and thus flow-independent background-error covariances. At least theoretically, it is possible to use flow-dependent background-error covariances in four-dimensional variational data assimilation (4DVAR) through exploiting the connection between variational data assimilation and estimation theory. This paper reports on investigations concerning the impact of flow-dependent background-error covariances in an idealized 4DVAR system that, based on quasigeostrophic dynamics, assimilates artificial observations. The main emphasis is placed on quantifying the improvement in analysis quality that is achievable in 4DVAR through the use of flow-dependent background-error covariances. Flow dependence is achieved through dynamical error-covariance evolution based on singular vectors in a reduced-rank approach, referred to as reduced-rank Kalman filter (RRKF). The RRKF yields partly dynamic background-error covariances through blending static and dynamic information, where the dynamic information is obtained from error evolution in a subspace of dimension k (defined here through the singular vectors) that may be small compared to the dimension of the model’s phase space n, which is equal to 1449 in the system investigated here. The results show that the use of flow-dependent background-error covariances based on the RRKF leads to improved analyses compared to a system using static background-error statistics. That latter system uses static background-error covariances that are carefully tuned given the model dynamics and the observational information available. It is also shown that the performance of the RRKF approaches the performance of the extended Kalman filter, as k approaches n. Results therefore support the hypothesis that significant analysis improvement is possible through the use of flow-dependent background-error covariances given that a sufficiently large number (here on the order of n/10) of singular vectors is used.

scholarly journals Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part I: Description and Single-Observation Experiments

2010 ◽  
Vol 138 (5) ◽  
pp. 1550-1566 ◽  
Author(s):  
Mark Buehner ◽  
P. L. Houtekamer ◽  
Cecilien Charette ◽  
Herschel L. Mitchell ◽  
Bin He
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Spatial Covariance ◽  
4D Flow ◽  
Background Error ◽  
Single Observation ◽  
The Impact

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of global deterministic NWP. In an EnKF experiment having the same spatial resolution as the inner loop in the four-dimensional variational data assimilation system (4D-Var), the mean of each analysis ensemble is used to initialize the higher-resolution deterministic forecasts. Five different variational data assimilation experiments are also conducted. These include both 4D-Var and 3D-Var (with first guess at appropriate time) experiments using either (i) prescribed background-error covariances similar to those used operationally, which are static in time and include horizontally homogeneous and isotropic correlations; or (ii) flow-dependent covariances computed from the EnKF background ensembles with spatial covariance localization applied. The fifth variational data assimilation experiment is a new approach called the Ensemble-4D-Var (En-4D-Var). This approach uses 4D flow-dependent background-error covariances estimated from EnKF ensembles to produce a 4D analysis without the need for tangent-linear or adjoint versions of the forecast model. In this first part of a two-part paper, results from a series of idealized assimilation experiments are presented. In these experiments, only a single observation or vertical profile of observations is assimilated to explore the impact of various fundamental differences among the EnKF and the various variational data assimilation approaches considered. In particular, differences in the application of covariance localization in the EnKF and variational approaches are shown to have a significant impact on the assimilation of satellite radiance observations. The results also demonstrate that 4D-Var and the EnKF can both produce similar 4D background-error covariances within a 6-h assimilation window. In the second part, results from medium-range deterministic forecasts for the study period of February 2007 are presented for the EnKF and the five variational data assimilation approaches considered.

scholarly journals An Examination of the Incremental Balance in a Global Ensemble-Based 3D-Var Data Assimilation System

2010 ◽  
Vol 138 (10) ◽  
pp. 3946-3966 ◽  
Author(s):  
Jean-François Caron ◽  
Luc Fillion
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Three Dimensional ◽  
Vertical Motion ◽  
Variational Data Assimilation ◽  
Spatial Covariance ◽  
Background Error ◽  
Localization Technique ◽  
Vertical Localization

Abstract This study examines the modification to the balance properties of the analysis increments in a global three-dimensional variational data assimilation scheme when using flow-dependent background-error covariances derived from an operational ensemble Kalman filter instead of static homogenous and isotropic background-error covariances based on lagged forecast differences. It is shown that the degree of balance in the analysis increments is degraded when the former method is used. This change can be attributed in part to the reduced degree of rotational balance found in short-term ensemble Kalman filter perturbations as compared to lagged forecast differences based on longer-range forecasts. However, the use of a horizontal and vertical localization technique to increase the rank of the ensemble-based covariances are found to have a significant deleterious effect on the rotational balance with the largest detrimental impact coming from the vertical localization and affecting particularly the upper levels. The examination of the vertical motion part of the analysis increments revealed that the spatial covariance localization technique also produces unrealistic vertical structure of vertical motion increments with abnormally large increments near the surface. A comparison between the analysis increments from the ensemble Kalman filter and from the ensemble-based three-dimensional variational data assimilation (3D-Var) scheme showed that the balance characteristics of the analysis increments resulting from the two systems are very similar.

scholarly journals Generalized background error covariance matrix model (GEN_BE v2.0)

2015 ◽  
Vol 8 (3) ◽  
pp. 669-696 ◽  
Author(s):  
G. Descombes ◽  
T. Auligné ◽  
F. Vandenberghe ◽  
D. M. Barker ◽  
J. Barré
Keyword(s):  
Data Assimilation ◽  
Covariance Matrix ◽  
Chemical Species ◽  
Variational Data Assimilation ◽  
Control Variables ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Background Error Covariance ◽  
Error Covariance ◽  
The Impact

Abstract. The specification of state background error statistics is a key component of data assimilation since it affects the impact observations will have on the analysis. In the variational data assimilation approach, applied in geophysical sciences, the dimensions of the background error covariance matrix (B) are usually too large to be explicitly determined and B needs to be modeled. Recent efforts to include new variables in the analysis such as cloud parameters and chemical species have required the development of the code to GENerate the Background Errors (GEN_BE) version 2.0 for the Weather Research and Forecasting (WRF) community model. GEN_BE allows for a simpler, flexible, robust, and community-oriented framework that gathers methods used by some meteorological operational centers and researchers. We present the advantages of this new design for the data assimilation community by performing benchmarks of different modeling of B and showing some of the new features in data assimilation test cases. As data assimilation for clouds remains a challenge, we present a multivariate approach that includes hydrometeors in the control variables and new correlated errors. In addition, the GEN_BE v2.0 code is employed to diagnose error parameter statistics for chemical species, which shows that it is a tool flexible enough to implement new control variables. While the generation of the background errors statistics code was first developed for atmospheric research, the new version (GEN_BE v2.0) can be easily applied to other domains of science and chosen to diagnose and model B. Initially developed for variational data assimilation, the model of the B matrix may be useful for variational ensemble hybrid methods as well.

scholarly journals Generalized Background Error covariance matrix model (GEN_BE v2.0)

2014 ◽  
Vol 7 (4) ◽  
pp. 4291-4352
Author(s):  
G. Descombes ◽  
T. Auligné ◽  
F. Vandenberghe ◽  
D. M. Barker
Keyword(s):  
Data Assimilation ◽  
Covariance Matrix ◽  
Chemical Species ◽  
Variational Data Assimilation ◽  
Control Variables ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Background Error Covariance ◽  
Error Covariance ◽  
The Impact

Abstract. The specification of state background error statistics is a key component of data assimilation since it affects the impact observations will have on the analysis. In the variational data assimilation approach, applied in geophysical sciences, the dimensions of the background error covariance matrix (B) are usually too large to be explicitly determined and B needs to be modeled. Recent efforts to include new variables in the analysis such as cloud parameters and chemical species have required the development of the code to GENerate the Background Errors (GEN_BE) version 2.0 for the Weather Research and Forecasting (WRF) community model to allow for a simpler, flexible, robust, and community-oriented framework that gathers methods used by meteorological operational centers and researchers. We present the advantages of this new design for the data assimilation community by performing benchmarks and showing some of the new features on data assimilation test cases. As data assimilation for clouds remains a challenge, we present a multivariate approach that includes hydrometeors in the control variables and new correlated errors. In addition, the GEN_BE v2.0 code is employed to diagnose error parameter statistics for chemical species, which shows that it is a tool flexible enough to involve new control variables. While the generation of the background errors statistics code has been first developed for atmospheric research, the new version (GEN_BE v2.0) can be easily extended to other domains of science and be chosen as a testbed for diagnostic and new modeling of B. Initially developed for variational data assimilation, the model of the B matrix may be useful for variational ensemble hybrid methods as well.

scholarly journals Comparison of Local Ensemble Transform Kalman Filter, 3DVAR, and 4DVAR in a Quasigeostrophic Model

2009 ◽  
Vol 137 (2) ◽  
pp. 693-709 ◽  
Author(s):  
Shu-Chih Yang ◽  
Matteo Corazza ◽  
Alberto Carrassi ◽  
Eugenia Kalnay ◽  
Takemasa Miyoshi
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Channel Model ◽  
Variational Data Assimilation ◽  
Ensemble Averaging ◽  
Background Error ◽  
Advantages And Disadvantages ◽  
Bred Vectors ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract Local ensemble transform Kalman filter (LETKF) data assimilation, three-dimensional variational data assimilation (3DVAR), and four-dimensional variational data assimilation (4DVAR) schemes are implemented in a quasigeostrophic channel model. Their advantages and disadvantages are compared to assess their use in practical applications. LETKF and 4DVAR, which take into account the flow-dependent errors, outperform 3DVAR under a perfect model scenario. Given the same observations, LETKF produces more accurate analyses than 4DVAR with a 12-h window by effectively correcting the fast-growing errors with the flow-dependent background error covariance. Even though 4DVAR performance benefits substantially from using a longer assimilation window, LETKF is also able to achieve a satisfactory accuracy compared to the 24-h 4DVAR analyses. It is shown that the advantage of the LETKF over 3DVAR is a result of both the ensemble averaging and the information about the “errors of the day” provided by the ensemble. The analysis corrections at the end of the 12-h assimilation window are similar for LETKF and the 12-h window 4DVAR, and they both resemble bred vectors. At the beginning of the assimilation window, LETKF analysis corrections obtained using a no-cost smoother also resemble the corresponding bred vectors, whereas the 4DVAR corrections are significantly different with much larger horizontal scales.

scholarly journals Strongly Coupled Data Assimilation Experiments with Linearized Ocean–Atmosphere Balance Relationships

2018 ◽  
Vol 146 (4) ◽  
pp. 1233-1257 ◽  
Author(s):  
Andrea Storto ◽  
Matthew J. Martin ◽  
Bruno Deremble ◽  
Simona Masina
Keyword(s):  
Data Assimilation ◽  
General Circulation ◽  
Circulation Model ◽  
Variational Data Assimilation ◽  
Global Ocean ◽  
Background Error ◽  
Near Surface ◽  
Strongly Coupled ◽  
Coupled Data Assimilation ◽  
The Impact

Coupled data assimilation is emerging as a target approach for Earth system prediction and reanalysis systems. Coupled data assimilation may be indeed able to minimize unbalanced air–sea initialization and maximize the intermedium propagation of observations. Here, we use a simplified framework where a global ocean general circulation model (NEMO) is coupled to an atmospheric boundary layer model [Cheap Atmospheric Mixed Layer (CheapAML)], which includes prognostic prediction of near-surface air temperature and moisture and allows for thermodynamic but not dynamic air–sea coupling. The control vector of an ocean variational data assimilation system is augmented to include 2-m atmospheric parameters. Cross-medium balances are formulated either through statistical cross covariances from monthly anomalies or through the application of linearized air–sea flux relationships derived from the tangent linear approximation of bulk formulas, which represents a novel solution to the coupled assimilation problem. As a proof of concept, the methodology is first applied to study the impact of in situ ocean observing networks on the near-surface atmospheric analyses and later to the complementary study of the impact of 2-m air observations on sea surface parameters, to assess benefits of strongly versus weakly coupled data assimilation. Several forecast experiments have been conducted for the period from June to December 2011. We find that especially after day 2 of the forecasts, strongly coupled data assimilation provides a beneficial impact, particularly in the tropical oceans. In most areas, the use of linearized air–sea balances outperforms the statistical relationships used, providing a motivation for implementing coupled tangent linear trajectories in four-dimensional variational data assimilation systems. Further impacts of strongly coupled data assimilation might be found by retuning the background error covariances.

scholarly journals An Ensemble Kalman Filter for Numerical Weather Prediction Based on Variational Data Assimilation: VarEnKF

2017 ◽  
Vol 145 (2) ◽  
pp. 617-635 ◽  
Author(s):  
Mark Buehner ◽  
Ron McTaggart-Cowan ◽  
Sylvain Heilliette
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Variational Approach ◽  
Variational Data Assimilation ◽  
Ensemble Prediction ◽  
Deterministic System ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Ensemble Mean

Several NWP centers currently employ a variational data assimilation approach for initializing deterministic forecasts and a separate ensemble Kalman filter (EnKF) system both for initializing ensemble forecasts and for providing ensemble background error covariances for the deterministic system. This study describes a new approach for performing the data assimilation step within a perturbed-observation EnKF. In this approach, called VarEnKF, the analysis increment is computed with a variational data assimilation approach both for the ensemble mean and for all of the ensemble perturbations. To obtain a computationally efficient algorithm, a much simpler configuration is used for the ensemble perturbations, whereas the configuration used for the ensemble mean is similar to that used for the deterministic system. Numerous practical benefits may be realized by using a variational approach for both deterministic and ensemble prediction, including improved efficiency for the development and maintenance of the computer code. Also, the use of essentially the same data assimilation algorithm would likely reduce the amount of numerical experimentation required when making system changes, since their impacts in the two systems would be very similar. The variational approach enables the use of hybrid background error covariances and may also allow the assimilation of a larger volume of observations. Preliminary tests with the Canadian global 256-member system produced significantly improved ensemble forecasts with VarEnKF as compared with the current EnKF and at a comparable computational cost. These improvements resulted entirely from changes to the ensemble mean analysis increment calculation. Moreover, because each ensemble perturbation is updated independently, VarEnKF scales perfectly up to a very large number of processors.

scholarly journals Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part II: One-Month Experiments with Real Observations

2010 ◽  
Vol 138 (5) ◽  
pp. 1567-1586 ◽  
Author(s):  
Mark Buehner ◽  
P. L. Houtekamer ◽  
Cecilien Charette ◽  
Herschel L. Mitchell ◽  
Bin He
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Background Error ◽  
Single Observation ◽  
Operational System ◽  
Medium Range ◽  
Forecast Quality

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of producing global deterministic numerical weather forecasts. Five different variational data assimilation approaches are considered including four-dimensional variational data assimilation (4D-Var) and three-dimensional variational data assimilation (3D-Var) with first guess at the appropriate time (3D-FGAT). Also included among these is a new approach, called Ensemble-4D-Var (En-4D-Var), that uses 4D ensemble background-error covariances from the EnKF. A description of the experimental configurations and results from single-observation experiments are presented in the first part of this two-part paper. The present paper focuses on results from medium-range deterministic forecasts initialized with analyses from the EnKF and the five variational data assimilation approaches for the period of February 2007. All experiments assimilate exactly the same full set of meteorological observations and use the same configuration of the forecast model to produce global deterministic medium-range forecasts. The quality of forecasts in the short (medium) range obtained by using the EnKF ensemble mean analysis is slightly degraded (improved) in the extratropics relative to using the 4D-Var analysis with background-error covariances similar to those used operationally. The use of the EnKF flow-dependent error covariances in the variational system (4D-Var or 3D-FGAT) leads to large (modest) forecast improvements in the southern extratropics (tropics) as compared with using covariances similar to the operational system (a gain of up to 9 h at day 5). The En-4D-Var approach leads to (i) either improved or similar forecast quality when compared with the 4D-Var experiment similar to the currently operational system, (ii) slightly worse forecast quality when compared with the 4D-Var experiment with EnKF error covariances, and (iii) generally similar forecast quality when compared with the EnKF experiment.

A New Approach for Estimating the Observation Impact in Ensemble–Variational Data Assimilation

2018 ◽  
Vol 146 (2) ◽  
pp. 447-465 ◽  
Author(s):  
Mark Buehner ◽  
Ping Du ◽  
Joël Bédard
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Observation Error ◽  
New Approach ◽  
Regional Prediction ◽  
Observation Impact ◽  
Adjoint Approach ◽  
The Impact

Abstract Two types of approaches are commonly used for estimating the impact of arbitrary subsets of observations on short-range forecast error. The first was developed for variational data assimilation systems and requires the adjoint of the forecast model. Comparable approaches were developed for use with the ensemble Kalman filter and rely on ensembles of forecasts. In this study, a new approach for computing observation impact is proposed for ensemble–variational data assimilation (EnVar). Like standard adjoint approaches, the adjoint of the data assimilation procedure is implemented through the iterative minimization of a modified cost function. However, like ensemble approaches, the adjoint of the forecast step is obtained by using an ensemble of forecasts. Numerical experiments were performed to compare the new approach with the standard adjoint approach in the context of operational deterministic NWP. Generally similar results are obtained with both approaches, especially when the new approach uses covariance localization that is horizontally advected between analysis and forecast times. However, large differences in estimated impacts are obtained for some surface observations. Vertical propagation of the observation impact is noticeably restricted with the new approach because of vertical covariance localization. The new approach is used to evaluate changes in observation impact as a result of the use of interchannel observation error correlations for radiance observations. The estimated observation impact in similarly configured global and regional prediction systems is also compared. Overall, the new approach should provide useful estimates of observation impact for data assimilation systems based on EnVar when an adjoint model is not available.

Sign in / Sign up

Export Citation Format

Share Document