On the Theoretical Equivalence of Differently Proposed Ensemble–3DVAR Hybrid Analysis Schemes

2007 ◽  
Vol 135 (1) ◽  
pp. 222-227 ◽  
Author(s):  
Xuguang Wang ◽  
Chris Snyder ◽  
Thomas M. Hamill
Keyword(s):  
Three Dimensional ◽  
Hybrid Scheme ◽  
Hybrid Analysis ◽  
Background Error ◽  
Hybrid Schemes ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Background Term ◽  
Covariance Term ◽  
The Cost

Abstract Hybrid ensemble–three-dimensional variational analysis schemes incorporate flow-dependent, ensemble-estimated background-error covariances into the three-dimensional variational data assimilation (3DVAR) framework. Typically the 3DVAR background-error covariance estimate is assumed to be stationary, nearly homogeneous, and isotropic. A hybrid scheme can be achieved by 1) directly replacing the background-error covariance term in the cost function by a linear combination of the original background-error covariance with the ensemble covariance or 2) through augmenting the state vector with another set of control variables preconditioned upon the square root of the ensemble covariance. These differently proposed hybrid schemes are proven to be equivalent. The latter framework may be a simpler way to incorporate ensemble information into operational 3DVAR schemes, where the preconditioning is performed with respect to the background term.

scholarly journals Relationships among Four-Dimensional Hybrid Ensemble–Variational Data Assimilation Algorithms with Full and Approximate Ensemble Covariance Localization

2016 ◽  
Vol 144 (2) ◽  
pp. 591-606 ◽  
Author(s):  
Chengsi Liu ◽  
Ming Xue
Keyword(s):  
Data Assimilation ◽  
Linear Model ◽  
Control Variable ◽  
Adjoint Model ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Covariance Term ◽  
Observation Space ◽  
Static Background

Abstract Ensemble–variational data assimilation algorithms that can incorporate the time dimension (four-dimensional or 4D) and combine static and ensemble-derived background error covariances (hybrid) are formulated in general forms based on the extended control variable and the observation-space-perturbation approaches. The properties and relationships of these algorithms and their approximated formulations are discussed. The main algorithms discussed include the following: 1) the standard ensemble 4DVar (En4DVar) algorithm incorporating ensemble-derived background error covariance through the extended control variable approach, 2) the 4DEnVar neglecting the time propagation of the extended control variable (4DEnVar-NPC), 3) the 4D ensemble–variational algorithm based on observation space perturbation (4DEnVar), and 4) the 4DEnVar with no propagation of covariance localization (4DEnVar-NPL). Without the static background error covariance term, none of the algorithms requires the adjoint model except for En4DVar. Costly applications of the tangent linear model to localized ensemble perturbations can be avoided by making the NPC and NPL approximations. It is proven that En4DVar and 4DEnVar are mathematically equivalent, while 4DEnVar-NPC and 4DEnVar-NPL are mathematically equivalent. Such equivalences are also demonstrated by single-observation assimilation experiments with a 1D linear advection model. The effects of the non-flow-following or stationary localization approximations are also examined through the experiments. All of the above algorithms can include the static background error covariance term to establish a hybrid formulation. When the static term is included, all algorithms will require a tangent linear model and an adjoint model. The first guess at appropriate time (FGAT) approximation is proposed to avoid the tangent linear and adjoint models. Computational costs of the algorithms are also discussed.

scholarly journals A Comparison of Hybrid Ensemble Transform Kalman Filter–Optimum Interpolation and Ensemble Square Root Filter Analysis Schemes

2007 ◽  
Vol 135 (3) ◽  
pp. 1055-1076 ◽  
Author(s):  
Xuguang Wang ◽  
Thomas M. Hamill ◽  
Jeffrey S. Whitaker ◽  
Craig H. Bishop
Keyword(s):  
Ensemble Size ◽  
Hybrid Scheme ◽  
Background Error ◽  
Background Error Covariance ◽  
Optimum Interpolation ◽  
Error Covariance ◽  
Analysis Scheme ◽  
Ensemble Transform Kalman Filter ◽  
Square Root Filter ◽  
Static Background

Abstract A hybrid ensemble transform Kalman filter (ETKF)–optimum interpolation (OI) analysis scheme is described and compared with an ensemble square root filter (EnSRF) analysis scheme. A two-layer primitive equation model was used under perfect-model assumptions. A simplified observation network was used, and the OI method utilized a static background error covariance constructed from a large inventory of historical forecast errors. The hybrid scheme updated the ensemble mean using a hybridized ensemble and static background-error covariance. The ensemble perturbations in the hybrid scheme were updated by the ETKF scheme. The EnSRF ran parallel data assimilation cycles for each member and serially assimilated the observations. The EnSRF background-error covariance was estimated fully from the ensemble. For 50-member ensembles, the analyses from the hybrid scheme were as accurate or nearly as accurate as those from the EnSRF, depending on the norm. For 20-member ensembles, the analyses from the hybrid scheme were more accurate than analyses from the EnSRF under certain norms. Both hybrid and EnSRF analyses were more accurate than the analyses from the OI. Further reducing the ensemble size to five members, the EnSRF exhibited filter divergence, whereas the analyses from the hybrid scheme were still better than those updated by the OI. Additionally, the hybrid scheme was less prone to spurious gravity wave activity than the EnSRF, especially when the ensemble size was small. Maximal growth in the ETKF ensemble perturbation space exceeded that in the EnSRF ensemble perturbation space. The relationship of the ETKF ensemble variance to the analysis error variance, a measure of a spread–skill relationship, was similar to that of the EnSRF ensemble. The hybrid scheme can be implemented in a reasonably straightforward manner in the operational variational frameworks, and the computational cost of the hybrid is expected to be much less than the EnSRF in the operational settings.

scholarly journals An Ensemble-Based Four-Dimensional Variational Data Assimilation Scheme. Part III: Antarctic Applications with Advanced Research WRF Using Real Data

2013 ◽  
Vol 141 (8) ◽  
pp. 2721-2739 ◽  
Author(s):  
Chengsi Liu ◽  
Qingnong Xiao
Keyword(s):  
Data Assimilation ◽  
Three Dimensional ◽  
Real Data ◽  
Variational Data Assimilation ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Short Period ◽  
Inflation Factor

Abstract A four-dimensional ensemble-based variational data assimilation (4DEnVar) algorithm proposed in Part I of the 4DEnVar series (denoted En4DVar in Part I, but here we refer to it as 4DEnVar according to WMO conference recommendation to differentiate it from En4DVar algorithm using adjoint model) uses a flow-dependent background error covariance calculated from ensemble forecasts and performs 4DVar optimization based on an incremental approach and a preconditioning algorithm. In Part II, the authors evaluated 4DEnVar with observing system simulation experiments (OSSEs) using the Advanced Research Weather Research and Forecasting Model (ARW-WRF, hereafter WRF). The current study extends the 4DEnVar to assimilate real observations for a cyclone in the Antarctic and the Southern Ocean in October 2007. The authors performed an intercomparison of four different WRF variational approaches for the case, including three-dimensional variational data assimilation (3DVar), first guess at the appropriate time (FGAT), and ensemble-based three-dimensional (En3DVar) and four-dimensional (4DEnVar) variational data assimilations. It is found that all data assimilation approaches produce positive impacts in this case. Applying the flow-dependent background error covariance in En3DVar and 4DEnVar yields forecast skills superior to those with the homogeneous and isotropic background error covariance in 3DVar and FGAT. In addition, the authors carried out FGAT and 4DEnVar 3-day cycling and 72-h forecasts. The results show that 4DEnVar produces a better performance in the cyclone prediction. The inflation factor on 4DEnVar can effectively improve the 4DEnVar analysis. The authors also conducted a short period (10-day lifetime of the cyclone in the domain) of analysis/forecast intercomparison experiments using 4DEnVar, FGAT, and 3DVar. The 4DEnVar scheme demonstrates overall superior and robust performance.

scholarly journals A Multiscale Variational Data Assimilation Scheme: Formulation and Illustration

2015 ◽  
Vol 143 (9) ◽  
pp. 3804-3822 ◽  
Author(s):  
Zhijin Li ◽  
James C. McWilliams ◽  
Kayo Ide ◽  
John D. Farrara
Keyword(s):  
High Resolution ◽  
Data Assimilation ◽  
Cost Function ◽  
Spatial Scales ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
The Cost ◽  
Fine Resolution

Abstract A multiscale data assimilation (MS-DA) scheme is formulated for fine-resolution models. A decomposition of the cost function is derived for a set of distinct spatial scales. The decomposed cost function allows for the background error covariance to be estimated separately for the distinct spatial scales, and multi-decorrelation scales to be explicitly incorporated in the background error covariance. MS-DA minimizes the partitioned cost functions sequentially from large to small scales. The multi-decorrelation length scale background error covariance enhances the spreading of sparse observations and prevents fine structures in high-resolution observations from being overly smoothed. The decomposition of the cost function also provides an avenue for mitigating the effects of scale aliasing and representativeness errors that inherently exist in a multiscale system, thus further improving the effectiveness of the assimilation of high-resolution observations. A set of one-dimensional experiments is performed to examine the properties of the MS-DA scheme. Emphasis is placed on the assimilation of patchy high-resolution observations representing radar and satellite measurements, alongside sparse observations representing those from conventional in situ platforms. The results illustrate how MS-DA improves the effectiveness of the assimilation of both these types of observations simultaneously.

scholarly journals A Hybrid Variational-Ensemble data assimilation scheme with systematic error correction for limited area ocean model

2016 ◽  
Author(s):  
Paolo Oddo ◽  
Andrea Storto ◽  
Srdjan Dobricic ◽  
Aniello Russo ◽  
Craig Lewis ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Systematic Error ◽  
Error Correction ◽  
Covariance Matrix ◽  
Limited Area ◽  
Hybrid Scheme ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Background Error Covariance ◽  
Error Covariance

Abstract. A hybrid variational-ensemble data assimilation scheme to estimate the vertical and horizontal parts of the background-error covariance matrix for an ocean variational data assimilation system is presented and tested in a limited area ocean model implemented in the western Mediterranean Sea. An extensive dataset collected during the Recognized Environmental Picture Experiments conducted in June 2014 by the Centre for Maritime Research and Experimentation has been used for assimilation and validation. The hybrid scheme is used to both correct the systematic error introduced in the system from the external forcing (initial, lateral and surface open boundary conditions) and model parameterization and improve the representation of small scale errors in the Background Error Covariance matrix. An ensemble system is run off-line for further use in the hybrid scheme, generated through perturbation of assimilated observations. Results of four different experiments have been compared. The reference experiment uses the classical static formulation of the background error covariance matrix and has no systematic error correction. The other three experiments account, or not, for systematic error correction and hybrid daily estimates of the background error covariance matrix combining the static and the ensemble derived errors statistics. Results show that the hybrid scheme when used in conjunction with the systematic error correction reduce the mean absolute error of temperature and salinity misfit by 55 % and 42 % respectively versus statistics arising from standard climatological covariances without systematic error correction.

Retrieval of Moisture from Simulated GPS Slant-Path Water Vapor Observations Using 3DVAR with Anisotropic Recursive Filters

2007 ◽  
Vol 135 (4) ◽  
pp. 1506-1521 ◽  
Author(s):  
Haixia Liu ◽  
Ming Xue ◽  
R. James Purser ◽  
David F. Parrish
Keyword(s):  
Water Vapor ◽  
Weather Prediction ◽  
Three Dimensional ◽  
Background Error ◽  
Special Configuration ◽  
Background Error Covariance ◽  
Recursive Filters ◽  
Error Covariance ◽  
Slant Path ◽  
The Impact

Abstract Anisotropic recursive filters are implemented within a three-dimensional variational data assimilation (3DVAR) framework to efficiently model the effect of flow-dependent background error covariance. The background error covariance is based on an estimated error field and on the idea of Riishøjgaard. In the anisotropic case, the background error pattern can be stretched or flattened in directions oblique to the alignment of the grid coordinates and is constructed by applying, at each point, six recursive filters along six directions corresponding, in general, to a special configuration of oblique lines of the grid. The recursive filters are much more efficient than corresponding explicit filters used in an earlier study and are therefore more suitable for real-time numerical weather prediction. A set of analysis experiments are conducted at a mesoscale resolution to examine the effectiveness of the 3DVAR system in analyzing simulated global positioning system (GPS) slant-path water vapor observations from ground-based GPS receivers and observations from collocated surface stations. It is shown that the analyses produced with recursive filters are at least as good as those with corresponding explicit filters. In some cases, the recursive filters actually perform better. The impact of flow-dependent background errors modeled using the anisotropic recursive filters is also examined. The use of anisotropic filters improves the analysis, especially in terms of finescale structures. The analysis system is found to be effective in the presence of typical observational errors. The sensitivity of isotropic and anisotropic recursive-filter analyses to the decorrelation scales is also examined systematically.

Direct Assimilation of Radar Reflectivity Data Using 3DVAR: Treatment of Hydrometeor Background Errors and OSSE Tests

2019 ◽  
Vol 147 (1) ◽  
pp. 17-29 ◽  
Author(s):  
Chengsi Liu ◽  
Ming Xue ◽  
Rong Kong
Keyword(s):  
Three Dimensional ◽  
Error Variance ◽  
Radar Reflectivity ◽  
Temperature Dependent ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Homogeneous Background ◽  
Reflectivity Data ◽  
Error Profiles

Despite the well-known importance of background error covariance in data assimilation, not much study has been focused on its impact on the assimilation of radar reflectivity within a three-dimensional variational (3DVar) framework. In this study, it is shown that unphysical analysis increments of hydrometeors are produced when using vertically homogeneous background error variance. This issue cannot be fully solved by using the so-called hydrometeor classification in the reflectivity observation operator. Alternatively, temperature-dependent background error profiles for hydrometeor control variables are proposed. With such a treatment, the vertical background error profiles are specified to be temperature dependent, allowing for more physical partitioning of radar-observed precipitation information among the liquid and ice hydrometeors. The 3DVar analyses using our treatment are compared with those using constant background error or “hydrometeor classification” through observing system simulation experiments with a simulated supercell storm. Results show that 1) 3DVar with constant hydrometeor background errors produces unphysical rainwater at the high levels and unphysical snow at the low levels; 2) the hydrometeor classification approach reduces unphysical rainwater and snow at those levels, but the analysis increments are still unphysically spread in the vertical by the background error covariance when the vertically invariant background errors are used; and 3) the temperature-dependent background error profiles enable physically more reasonable analyses of liquid and ice hydrometeors from reflectivity assimilation.

scholarly journals A hybrid variational-ensemble data assimilation scheme with systematic error correction for limited-area ocean models

Ocean Science ◽  
2016 ◽  
Vol 12 (5) ◽  
pp. 1137-1153 ◽  
Author(s):  
Paolo Oddo ◽  
Andrea Storto ◽  
Srdjan Dobricic ◽  
Aniello Russo ◽  
Craig Lewis ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Systematic Error ◽  
Error Correction ◽  
Covariance Matrix ◽  
Limited Area ◽  
Hybrid Scheme ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Background Error Covariance ◽  
Error Covariance

Abstract. A hybrid variational-ensemble data assimilation scheme to estimate the vertical and horizontal parts of the background error covariance matrix for an ocean variational data assimilation system is presented and tested in a limited-area ocean model implemented in the western Mediterranean Sea. An extensive data set collected during the Recognized Environmental Picture Experiments conducted in June 2014 by the Centre for Maritime Research and Experimentation has been used for assimilation and validation. The hybrid scheme is used to both correct the systematic error introduced in the system from the external forcing (initialisation, lateral and surface open boundary conditions) and model parameterisation, and improve the representation of small-scale errors in the background error covariance matrix. An ensemble system is run offline for further use in the hybrid scheme, generated through perturbation of assimilated observations. Results of four different experiments have been compared. The reference experiment uses the classical stationary formulation of the background error covariance matrix and has no systematic error correction. The other three experiments account for, or not, systematic error correction and hybrid background error covariance matrix combining the static and the ensemble-derived errors of the day. Results show that the hybrid scheme when used in conjunction with the systematic error correction reduces the mean absolute error of temperature and salinity misfit by 55 and 42 % respectively, versus statistics arising from standard climatological covariances without systematic error correction.

scholarly journals THE INFLUENCE OF BACKGROUND ERROR COVARIANCE OF GSI THREE-DIMENSIONAL VARIATIONAL DATA ASSIMILATION ON METEOROLOGY AND AEROSOL PREDICTION

2018 ◽  
Vol XLII-3/W5 ◽  
pp. 17-23
Author(s):  
Y. Hu ◽  
M. Zhang ◽  
Y. Liang ◽  
L. Ye ◽  
D. Zhao ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Three Dimensional ◽  
Variational Data Assimilation ◽  
Small Scale ◽  
Length Scale ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Horizontal Length ◽  
Mean Square Errors

<p><strong>Abstract.</strong> Background error covariance (BEC) plays a key role in a variational data assimilation system. It determines variable analysis increments by spreading information from observation points. In order to test the influence of BEC on the GSI data assimilation and prediction of aerosol in Beijing-Tianjin-Hebei, a regional BEC is calculated using one month series of numerical forecast fields of November 2017 based on the National Meteorological Center (NMC) method, and compared with the global BEC.The results show that the standard deviation of stream function of the regional BEC is larger than that of the global BEC. And the horizontal length-scale of the regional BEC is smaller than that of the global BEC, white the vertical length-scale of the regional BEC is similar with that of the global BEC. The increments of the assimilation experiment with the regional BEC present more small scale information than that with the global BEC. The forecast skill of the experiment with the regional BEC is higher than that with the global BEC in the stations of Beijing, Tianjin, Chengde and Taiyuan, and the average root-mean-square errors (RMSE) reduces by over 13.4%.</p>

scholarly journals Background error covariance with balance constraints for aerosol species and applications in variational data assimilation

2016 ◽  
Vol 9 (8) ◽  
pp. 2623-2638 ◽  
Author(s):  
Zengliang Zang ◽  
Zilong Hao ◽  
Yi Li ◽  
Xiaobin Pan ◽  
Wei You ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Three Dimensional ◽  
Variational Data Assimilation ◽  
Statistical Regression ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Balance Analysis ◽  
Cross Correlations ◽  
Data Assimilation System

Abstract. Balance constraints are important for background error covariance (BEC) in data assimilation to spread information between different variables and produce balance analysis fields. Using statistical regression, we develop a balance constraint for the BEC of aerosol variables and apply it to a three-dimensional variational data assimilation system in the WRF/Chem model; 1-month forecasts from the WRF/Chem model are employed for BEC statistics. The cross-correlations between the different species are generally high. The largest correlation occurs between elemental carbon and organic carbon with as large as 0.9. After using the balance constraints, the correlations between the unbalanced variables reduce to less than 0.2. A set of data assimilation and forecasting experiments is performed. In these experiments, surface PM2.5 concentrations and speciated concentrations along aircraft flight tracks are assimilated. The analysis increments with the balance constraints show spatial distributions more complex than those without the balance constraints, which is a consequence of the spreading of observation information across variables due to the balance constraints. The forecast skills with the balance constraints show substantial and durable improvements from the 2nd hour to the 16th hour compared with the forecast skills without the balance constraints. The results suggest that the developed balance constraints are important for the aerosol assimilation and forecasting.

Sign in / Sign up

Export Citation Format

Share Document