An Examination of the Incremental Balance in a Global Ensemble-Based 3D-Var Data Assimilation System

Abstract This study examines the modification to the balance properties of the analysis increments in a global three-dimensional variational data assimilation scheme when using flow-dependent background-error covariances derived from an operational ensemble Kalman filter instead of static homogenous and isotropic background-error covariances based on lagged forecast differences. It is shown that the degree of balance in the analysis increments is degraded when the former method is used. This change can be attributed in part to the reduced degree of rotational balance found in short-term ensemble Kalman filter perturbations as compared to lagged forecast differences based on longer-range forecasts. However, the use of a horizontal and vertical localization technique to increase the rank of the ensemble-based covariances are found to have a significant deleterious effect on the rotational balance with the largest detrimental impact coming from the vertical localization and affecting particularly the upper levels. The examination of the vertical motion part of the analysis increments revealed that the spatial covariance localization technique also produces unrealistic vertical structure of vertical motion increments with abnormally large increments near the surface. A comparison between the analysis increments from the ensemble Kalman filter and from the ensemble-based three-dimensional variational data assimilation (3D-Var) scheme showed that the balance characteristics of the analysis increments resulting from the two systems are very similar.

Download Full-text

Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part I: Description and Single-Observation Experiments

Monthly Weather Review ◽

10.1175/2009mwr3157.1 ◽

2010 ◽

Vol 138 (5) ◽

pp. 1550-1566 ◽

Cited By ~ 155

Author(s):

Mark Buehner ◽

P. L. Houtekamer ◽

Cecilien Charette ◽

Herschel L. Mitchell ◽

Bin He

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Forecast Model ◽

Variational Data Assimilation ◽

Spatial Covariance ◽

4D Flow ◽

Background Error ◽

Single Observation ◽

The Impact

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of global deterministic NWP. In an EnKF experiment having the same spatial resolution as the inner loop in the four-dimensional variational data assimilation system (4D-Var), the mean of each analysis ensemble is used to initialize the higher-resolution deterministic forecasts. Five different variational data assimilation experiments are also conducted. These include both 4D-Var and 3D-Var (with first guess at appropriate time) experiments using either (i) prescribed background-error covariances similar to those used operationally, which are static in time and include horizontally homogeneous and isotropic correlations; or (ii) flow-dependent covariances computed from the EnKF background ensembles with spatial covariance localization applied. The fifth variational data assimilation experiment is a new approach called the Ensemble-4D-Var (En-4D-Var). This approach uses 4D flow-dependent background-error covariances estimated from EnKF ensembles to produce a 4D analysis without the need for tangent-linear or adjoint versions of the forecast model. In this first part of a two-part paper, results from a series of idealized assimilation experiments are presented. In these experiments, only a single observation or vertical profile of observations is assimilated to explore the impact of various fundamental differences among the EnKF and the various variational data assimilation approaches considered. In particular, differences in the application of covariance localization in the EnKF and variational approaches are shown to have a significant impact on the assimilation of satellite radiance observations. The results also demonstrate that 4D-Var and the EnKF can both produce similar 4D background-error covariances within a 6-h assimilation window. In the second part, results from medium-range deterministic forecasts for the study period of February 2007 are presented for the EnKF and the five variational data assimilation approaches considered.

Download Full-text

E3DVar: Coupling an Ensemble Kalman Filter with Three-Dimensional Variational Data Assimilation in a Limited-Area Weather Prediction Model and Comparison to E4DVar

Monthly Weather Review ◽

10.1175/mwr-d-12-00075.1 ◽

2013 ◽

Vol 141 (3) ◽

pp. 900-917 ◽

Cited By ~ 51

Author(s):

Fuqing Zhang ◽

Meng Zhang ◽

Jonathan Poterjoy

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Weather Prediction ◽

Three Dimensional ◽

Variational Data Assimilation ◽

Lead Times ◽

Limited Area ◽

Model Errors ◽

Mean Square Errors

Abstract This study examines the performance of a hybrid ensemble-variational data assimilation system (E3DVar) that couples an ensemble Kalman filter (EnKF) with the three-dimensional variational data assimilation (3DVar) system for the Weather Research and Forecasting (WRF) Model. The performance of E3DVar and the component EnKF and 3DVar systems are compared over the eastern United States for June 2003. Conventional sounding and surface observations as well as data from wind profilers, aircraft and ships, and cloud-tracked winds from satellites, are assimilated every 6 h during the experiments, and forecasts are verified using standard sounding observations. Forecasts with 12- to 72-h lead times are found to have noticeably smaller root-mean-square errors when initialized with the E3DVar system, as opposed to the EnKF, especially for the 12-h wind and moisture fields. The E3DVar system demonstrates similar performance as an EnKF, while using less than half the number of ensemble members, and is less sensitive to the use of a multiphysics ensemble to account for model errors. The E3DVar system is also compared with a similar hybrid method that replaces the 3DVar component with the WRF four-dimensional variational data assimilation (4DVar) method (denoted E4DVar). The E4DVar method demonstrated considerable improvements over E3DVar for nearly all model levels and variables at the shorter forecast lead times (12–48 h), but the forecast accuracies of all three ensemble-based methods (EnKF, E3DVar, and E4DVar) converge to similar results at longer lead times (60–72 h). Nevertheless, all methods that used ensemble information produced considerably better forecasts than the two methods that relied solely on static background error covariance (i.e., 3DVar and 4DVar).

Download Full-text

An Ensemble Kalman Filter for Numerical Weather Prediction Based on Variational Data Assimilation: VarEnKF

Monthly Weather Review ◽

10.1175/mwr-d-16-0106.1 ◽

2017 ◽

Vol 145 (2) ◽

pp. 617-635 ◽

Cited By ~ 8

Author(s):

Mark Buehner ◽

Ron McTaggart-Cowan ◽

Sylvain Heilliette

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Variational Approach ◽

Variational Data Assimilation ◽

Ensemble Prediction ◽

Deterministic System ◽

Ensemble Forecasts ◽

Background Error ◽

Ensemble Mean

Several NWP centers currently employ a variational data assimilation approach for initializing deterministic forecasts and a separate ensemble Kalman filter (EnKF) system both for initializing ensemble forecasts and for providing ensemble background error covariances for the deterministic system. This study describes a new approach for performing the data assimilation step within a perturbed-observation EnKF. In this approach, called VarEnKF, the analysis increment is computed with a variational data assimilation approach both for the ensemble mean and for all of the ensemble perturbations. To obtain a computationally efficient algorithm, a much simpler configuration is used for the ensemble perturbations, whereas the configuration used for the ensemble mean is similar to that used for the deterministic system. Numerous practical benefits may be realized by using a variational approach for both deterministic and ensemble prediction, including improved efficiency for the development and maintenance of the computer code. Also, the use of essentially the same data assimilation algorithm would likely reduce the amount of numerical experimentation required when making system changes, since their impacts in the two systems would be very similar. The variational approach enables the use of hybrid background error covariances and may also allow the assimilation of a larger volume of observations. Preliminary tests with the Canadian global 256-member system produced significantly improved ensemble forecasts with VarEnKF as compared with the current EnKF and at a comparable computational cost. These improvements resulted entirely from changes to the ensemble mean analysis increment calculation. Moreover, because each ensemble perturbation is updated independently, VarEnKF scales perfectly up to a very large number of processors.

Download Full-text

Preemptive Forecasts Using an Ensemble Kalman Filter

Monthly Weather Review ◽

10.1175/mwr3480.1 ◽

2007 ◽

Vol 135 (10) ◽

pp. 3484-3495 ◽

Cited By ~ 9

Author(s):

Brian J. Etherton

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Three Dimensional ◽

Forecast Model ◽

Observation Time ◽

Model Integration ◽

Model Forecast ◽

Error Statistics ◽

Background Error

Abstract An ensemble Kalman filter (EnKF) estimates the error statistics of a model forecast using an ensemble of model forecasts. One use of an EnKF is data assimilation, resulting in the creation of an increment to the first-guess field at the observation time. Another use of an EnKF is to propagate error statistics of a model forecast forward in time, such as is done for optimizing the location of adaptive observations. Combining these two uses of an ensemble Kalman filter, a “preemptive forecast” can be generated. In a preemptive forecast, the increment to the first-guess field is, using ensembles, propagated to some future time and added to the future control forecast, resulting in a new forecast. This new forecast requires no more time to produce than the time needed to run a data assimilation scheme, as no model integration is necessary. In an observing system simulation experiment (OSSE), a barotropic vorticity model was run to produce a 300-day “nature run.” The same model, run with a different vorticity forcing scheme, served as the forecast model. The model produced 24- and 48-h forecasts for each of the 300 days. The model was initialized every 24 h by assimilating observations of the nature run using a hybrid ensemble Kalman filter–three-dimensional variational data assimilation (3DVAR) scheme. In addition to the control forecast, a 64-member forecast ensemble was generated for each of the 300 days. Every 24 h, given a set of observations, the 64-member ensemble, and the control run, an EnKF was used to create 24-h preemptive forecasts. The preemptive forecasts were more accurate than the unmodified, original 48-h forecasts, though not quite as accurate as the 24-h forecast obtained from a new model integration initialized by assimilating the same observations as were used in the preemptive forecasts. The accuracy of the preemptive forecasts improved significantly when 1) the ensemble-based error statistics used by the EnKF were localized using a Schur product and 2) a model error term was included in the background error covariance matrices.

Download Full-text

An Adjoint-Based Adaptive Ensemble Kalman Filter

Monthly Weather Review ◽

10.1175/mwr-d-12-00244.1 ◽

2013 ◽

Vol 141 (10) ◽

pp. 3343-3359 ◽

Cited By ~ 8

Author(s):

Hajoon Song ◽

Ibrahim Hoteit ◽

Bruce D. Cornuelle ◽

Xiaodong Luo ◽

Aneesh C. Subramanian

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Three Dimensional ◽

Variational Data Assimilation ◽

Optimal Interpolation ◽

Adjoint Model ◽

Estimation Errors ◽

New Members

Abstract A new hybrid ensemble Kalman filter/four-dimensional variational data assimilation (EnKF/4D-VAR) approach is introduced to mitigate background covariance limitations in the EnKF. The work is based on the adaptive EnKF (AEnKF) method, which bears a strong resemblance to the hybrid EnKF/three-dimensional variational data assimilation (3D-VAR) method. In the AEnKF, the representativeness of the EnKF ensemble is regularly enhanced with new members generated after back projection of the EnKF analysis residuals to state space using a 3D-VAR [or optimal interpolation (OI)] scheme with a preselected background covariance matrix. The idea here is to reformulate the transformation of the residuals as a 4D-VAR problem, constraining the new member with model dynamics and the previous observations. This should provide more information for the estimation of the new member and reduce dependence of the AEnKF on the assumed stationary background covariance matrix. This is done by integrating the analysis residuals backward in time with the adjoint model. Numerical experiments are performed with the Lorenz-96 model under different scenarios to test the new approach and to evaluate its performance with respect to the EnKF and the hybrid EnKF/3D-VAR. The new method leads to the least root-mean-square estimation errors as long as the linear assumption guaranteeing the stability of the adjoint model holds. It is also found to be less sensitive to choices of the assimilation system inputs and parameters.

Download Full-text

Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part II: One-Month Experiments with Real Observations

Monthly Weather Review ◽

10.1175/2009mwr3158.1 ◽

2010 ◽

Vol 138 (5) ◽

pp. 1567-1586 ◽

Cited By ~ 146

Author(s):

Mark Buehner ◽

P. L. Houtekamer ◽

Cecilien Charette ◽

Herschel L. Mitchell ◽

Bin He

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Forecast Model ◽

Variational Data Assimilation ◽

Background Error ◽

Single Observation ◽

Operational System ◽

Medium Range ◽

Forecast Quality

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of producing global deterministic numerical weather forecasts. Five different variational data assimilation approaches are considered including four-dimensional variational data assimilation (4D-Var) and three-dimensional variational data assimilation (3D-Var) with first guess at the appropriate time (3D-FGAT). Also included among these is a new approach, called Ensemble-4D-Var (En-4D-Var), that uses 4D ensemble background-error covariances from the EnKF. A description of the experimental configurations and results from single-observation experiments are presented in the first part of this two-part paper. The present paper focuses on results from medium-range deterministic forecasts initialized with analyses from the EnKF and the five variational data assimilation approaches for the period of February 2007. All experiments assimilate exactly the same full set of meteorological observations and use the same configuration of the forecast model to produce global deterministic medium-range forecasts. The quality of forecasts in the short (medium) range obtained by using the EnKF ensemble mean analysis is slightly degraded (improved) in the extratropics relative to using the 4D-Var analysis with background-error covariances similar to those used operationally. The use of the EnKF flow-dependent error covariances in the variational system (4D-Var or 3D-FGAT) leads to large (modest) forecast improvements in the southern extratropics (tropics) as compared with using covariances similar to the operational system (a gain of up to 9 h at day 5). The En-4D-Var approach leads to (i) either improved or similar forecast quality when compared with the 4D-Var experiment similar to the currently operational system, (ii) slightly worse forecast quality when compared with the 4D-Var experiment with EnKF error covariances, and (iii) generally similar forecast quality when compared with the EnKF experiment.

Download Full-text

Evaluation of Surface Analyses and Forecasts with a Multiscale Ensemble Kalman Filter in Regions of Complex Terrain

Monthly Weather Review ◽

10.1175/2010mwr3612.1 ◽

2011 ◽

Vol 139 (6) ◽

pp. 2008-2024 ◽

Cited By ~ 24

Author(s):

Brian C. Ancell ◽

Clifford F. Mass ◽

Gregory J. Hakim

Keyword(s):

Kalman Filter ◽

High Resolution ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Complex Terrain ◽

Error Variance ◽

Small Scale ◽

Observation Error ◽

Grid Spacing ◽

Background Error

Abstract Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30–50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%–30%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested.

Download Full-text

A radar reflectivity operator with ice-phase hydrometeors for variational data assimilation (version 1.0) and its evaluation with real radar data

Geoscientific Model Development ◽

10.5194/gmd-12-4031-2019 ◽

2019 ◽

Vol 12 (9) ◽

pp. 4031-4051 ◽

Cited By ~ 1

Author(s):

Shizhang Wang ◽

Zhiquan Liu

Keyword(s):

Data Assimilation ◽

Three Dimensional ◽

Wrf Model ◽

Radar Data ◽

Poor Quality ◽

Variational Data Assimilation ◽

Radar Reflectivity ◽

Background Error ◽

Background Fields ◽

Ice Phase

Abstract. A reflectivity forward operator and its associated tangent linear and adjoint operators (together named RadarVar) were developed for variational data assimilation (DA). RadarVar can analyze both rainwater and ice-phase species (snow and graupel) by directly assimilating radar reflectivity observations. The results of three-dimensional variational (3D-Var) DA experiments with a 3 km grid mesh setting of the Weather Research and Forecasting (WRF) model showed that RadarVar was effective at producing an analysis of reflectivity pattern and intensity similar to the observed data. Two to three outer loops with 50–100 iterations in each loop were needed to obtain a converged 3-D analysis of reflectivity, rainwater, snow, and graupel, including the melting layers with mixed-phase hydrometeors. It is shown that the deficiencies in the analysis using this operator, caused by the poor quality of the background fields and the use of the static background error covariance, can be partially resolved by using radar-retrieved hydrometeors in a preprocessing step and tuning the spatial correlation length scales of the background errors. The direct radar reflectivity assimilation using RadarVar also improved the short-term (2–5 h) precipitation forecasts compared to those of the experiment without DA.

Download Full-text

E4DVar: Coupling an Ensemble Kalman Filter with Four-Dimensional Variational Data Assimilation in a Limited-Area Weather Prediction Model

Monthly Weather Review ◽

10.1175/mwr-d-11-00023.1 ◽

2012 ◽

Vol 140 (2) ◽

pp. 587-600 ◽

Cited By ~ 69

Author(s):

Meng Zhang ◽

Fuqing Zhang

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Prediction Model ◽

Ensemble Kalman Filter ◽

Weather Prediction ◽

Coupled System ◽

Limited Area ◽

Ensemble Forecasts ◽

Background Error ◽

Hybrid Data Assimilation

A hybrid data assimilation approach that couples the ensemble Kalman filter (EnKF) and four-dimensional variational (4DVar) methods is implemented for the first time in a limited-area weather prediction model. In this coupled system, denoted E4DVar, the EnKF and 4DVar systems run in parallel while feeding into each other. The multivariate, flow-dependent background error covariance estimated from the EnKF ensemble is used in the 4DVar minimization and the ensemble mean in the EnKF analysis is replaced by the 4DVar analysis, while updating the analysis perturbations for the next cycle of ensemble forecasts with the EnKF. Therefore, the E4DVar can obtain flow-dependent information from both the explicit covariance matrix derived from ensemble forecasts, as well as implicitly from the 4DVar trajectory. The performance of an E4DVar system is compared with the uncoupled 4DVar and EnKF for a limited-area model by assimilating various conventional observations over the contiguous United States for June 2003. After verifying the forecasts from each analysis against standard sounding observations, it is found that the E4DVar substantially outperforms both the EnKF and 4DVar during this active summer month, which featured several episodes of severe convective weather. On average, the forecasts produced from E4DVar analyses have considerably smaller errors than both of the stand-alone EnKF and 4DVar systems for forecast lead times up to 60 h.

Download Full-text

Intercomparison of an Ensemble Kalman Filter with Three- and Four-Dimensional Variational Data Assimilation Methods in a Limited-Area Model over the Month of June 2003

Monthly Weather Review ◽

10.1175/2010mwr3610.1 ◽

2011 ◽

Vol 139 (2) ◽

pp. 566-572 ◽

Cited By ~ 48

Author(s):

Meng Zhang ◽

Fuqing Zhang ◽

Xiang-Yu Huang ◽

Xin Zhang

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Forecast Error ◽

Wrf Model ◽

Warm Season ◽

Variational Data Assimilation ◽

Lead Times ◽

Limited Area ◽

Forecast Errors

Abstract This study compares the performance of an ensemble Kalman filter (EnKF) with both the three-dimensional and four-dimensional variational data assimilation (3DVar and 4DVar) methods of the Weather Research and Forecasting (WRF) model over the contiguous United States in a warm-season month (June) of 2003. The data assimilated every 6 h include conventional sounding and surface observations as well as data from wind profilers, ships and aircraft, and the cloud-tracked winds from satellites. The performances of these methods are evaluated through verifying the 12- to 72-h forecasts initialized twice daily from the analysis of each method against the standard sounding observations. It is found that 4DVar has consistently smaller error than that of 3DVar for winds and temperature at all forecast lead times except at 60 and 72 h when their forecast errors become comparable in amplitude, while the two schemes have similar performance in moisture at all lead times. The forecast error of the EnKF is comparable to that of the 4DVar at 12–36-h lead times, both of which are substantially smaller than that of the 3DVar, despite the fact that 3DVar fits the sounding observations much more closely at the analysis time. The advantage of the EnKF becomes even more evident at 48–72-h lead times; the 72-h forecast error of the EnKF is comparable in magnitude to the 48-h error of 3DVar/4DVar.

Download Full-text