scholarly journals Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part II: One-Month Experiments with Real Observations

2010 ◽  
Vol 138 (5) ◽  
pp. 1567-1586 ◽  
Author(s):  
Mark Buehner ◽  
P. L. Houtekamer ◽  
Cecilien Charette ◽  
Herschel L. Mitchell ◽  
Bin He
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Background Error ◽  
Single Observation ◽  
Operational System ◽  
Medium Range ◽  
Forecast Quality

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of producing global deterministic numerical weather forecasts. Five different variational data assimilation approaches are considered including four-dimensional variational data assimilation (4D-Var) and three-dimensional variational data assimilation (3D-Var) with first guess at the appropriate time (3D-FGAT). Also included among these is a new approach, called Ensemble-4D-Var (En-4D-Var), that uses 4D ensemble background-error covariances from the EnKF. A description of the experimental configurations and results from single-observation experiments are presented in the first part of this two-part paper. The present paper focuses on results from medium-range deterministic forecasts initialized with analyses from the EnKF and the five variational data assimilation approaches for the period of February 2007. All experiments assimilate exactly the same full set of meteorological observations and use the same configuration of the forecast model to produce global deterministic medium-range forecasts. The quality of forecasts in the short (medium) range obtained by using the EnKF ensemble mean analysis is slightly degraded (improved) in the extratropics relative to using the 4D-Var analysis with background-error covariances similar to those used operationally. The use of the EnKF flow-dependent error covariances in the variational system (4D-Var or 3D-FGAT) leads to large (modest) forecast improvements in the southern extratropics (tropics) as compared with using covariances similar to the operational system (a gain of up to 9 h at day 5). The En-4D-Var approach leads to (i) either improved or similar forecast quality when compared with the 4D-Var experiment similar to the currently operational system, (ii) slightly worse forecast quality when compared with the 4D-Var experiment with EnKF error covariances, and (iii) generally similar forecast quality when compared with the EnKF experiment.

scholarly journals Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part I: Description and Single-Observation Experiments

2010 ◽  
Vol 138 (5) ◽  
pp. 1550-1566 ◽  
Author(s):  
Mark Buehner ◽  
P. L. Houtekamer ◽  
Cecilien Charette ◽  
Herschel L. Mitchell ◽  
Bin He
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Spatial Covariance ◽  
4D Flow ◽  
Background Error ◽  
Single Observation ◽  
The Impact

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of global deterministic NWP. In an EnKF experiment having the same spatial resolution as the inner loop in the four-dimensional variational data assimilation system (4D-Var), the mean of each analysis ensemble is used to initialize the higher-resolution deterministic forecasts. Five different variational data assimilation experiments are also conducted. These include both 4D-Var and 3D-Var (with first guess at appropriate time) experiments using either (i) prescribed background-error covariances similar to those used operationally, which are static in time and include horizontally homogeneous and isotropic correlations; or (ii) flow-dependent covariances computed from the EnKF background ensembles with spatial covariance localization applied. The fifth variational data assimilation experiment is a new approach called the Ensemble-4D-Var (En-4D-Var). This approach uses 4D flow-dependent background-error covariances estimated from EnKF ensembles to produce a 4D analysis without the need for tangent-linear or adjoint versions of the forecast model. In this first part of a two-part paper, results from a series of idealized assimilation experiments are presented. In these experiments, only a single observation or vertical profile of observations is assimilated to explore the impact of various fundamental differences among the EnKF and the various variational data assimilation approaches considered. In particular, differences in the application of covariance localization in the EnKF and variational approaches are shown to have a significant impact on the assimilation of satellite radiance observations. The results also demonstrate that 4D-Var and the EnKF can both produce similar 4D background-error covariances within a 6-h assimilation window. In the second part, results from medium-range deterministic forecasts for the study period of February 2007 are presented for the EnKF and the five variational data assimilation approaches considered.

scholarly journals An Examination of the Incremental Balance in a Global Ensemble-Based 3D-Var Data Assimilation System

2010 ◽  
Vol 138 (10) ◽  
pp. 3946-3966 ◽  
Author(s):  
Jean-François Caron ◽  
Luc Fillion
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Three Dimensional ◽  
Vertical Motion ◽  
Variational Data Assimilation ◽  
Spatial Covariance ◽  
Background Error ◽  
Localization Technique ◽  
Vertical Localization

Abstract This study examines the modification to the balance properties of the analysis increments in a global three-dimensional variational data assimilation scheme when using flow-dependent background-error covariances derived from an operational ensemble Kalman filter instead of static homogenous and isotropic background-error covariances based on lagged forecast differences. It is shown that the degree of balance in the analysis increments is degraded when the former method is used. This change can be attributed in part to the reduced degree of rotational balance found in short-term ensemble Kalman filter perturbations as compared to lagged forecast differences based on longer-range forecasts. However, the use of a horizontal and vertical localization technique to increase the rank of the ensemble-based covariances are found to have a significant deleterious effect on the rotational balance with the largest detrimental impact coming from the vertical localization and affecting particularly the upper levels. The examination of the vertical motion part of the analysis increments revealed that the spatial covariance localization technique also produces unrealistic vertical structure of vertical motion increments with abnormally large increments near the surface. A comparison between the analysis increments from the ensemble Kalman filter and from the ensemble-based three-dimensional variational data assimilation (3D-Var) scheme showed that the balance characteristics of the analysis increments resulting from the two systems are very similar.

scholarly journals An Ensemble Kalman Filter for Numerical Weather Prediction Based on Variational Data Assimilation: VarEnKF

2017 ◽  
Vol 145 (2) ◽  
pp. 617-635 ◽  
Author(s):  
Mark Buehner ◽  
Ron McTaggart-Cowan ◽  
Sylvain Heilliette
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Variational Approach ◽  
Variational Data Assimilation ◽  
Ensemble Prediction ◽  
Deterministic System ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Ensemble Mean

Several NWP centers currently employ a variational data assimilation approach for initializing deterministic forecasts and a separate ensemble Kalman filter (EnKF) system both for initializing ensemble forecasts and for providing ensemble background error covariances for the deterministic system. This study describes a new approach for performing the data assimilation step within a perturbed-observation EnKF. In this approach, called VarEnKF, the analysis increment is computed with a variational data assimilation approach both for the ensemble mean and for all of the ensemble perturbations. To obtain a computationally efficient algorithm, a much simpler configuration is used for the ensemble perturbations, whereas the configuration used for the ensemble mean is similar to that used for the deterministic system. Numerous practical benefits may be realized by using a variational approach for both deterministic and ensemble prediction, including improved efficiency for the development and maintenance of the computer code. Also, the use of essentially the same data assimilation algorithm would likely reduce the amount of numerical experimentation required when making system changes, since their impacts in the two systems would be very similar. The variational approach enables the use of hybrid background error covariances and may also allow the assimilation of a larger volume of observations. Preliminary tests with the Canadian global 256-member system produced significantly improved ensemble forecasts with VarEnKF as compared with the current EnKF and at a comparable computational cost. These improvements resulted entirely from changes to the ensemble mean analysis increment calculation. Moreover, because each ensemble perturbation is updated independently, VarEnKF scales perfectly up to a very large number of processors.

scholarly journals Preemptive Forecasts Using an Ensemble Kalman Filter

2007 ◽  
Vol 135 (10) ◽  
pp. 3484-3495 ◽  
Author(s):  
Brian J. Etherton
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Three Dimensional ◽  
Forecast Model ◽  
Observation Time ◽  
Model Integration ◽  
Model Forecast ◽  
Error Statistics ◽  
Background Error

Abstract An ensemble Kalman filter (EnKF) estimates the error statistics of a model forecast using an ensemble of model forecasts. One use of an EnKF is data assimilation, resulting in the creation of an increment to the first-guess field at the observation time. Another use of an EnKF is to propagate error statistics of a model forecast forward in time, such as is done for optimizing the location of adaptive observations. Combining these two uses of an ensemble Kalman filter, a “preemptive forecast” can be generated. In a preemptive forecast, the increment to the first-guess field is, using ensembles, propagated to some future time and added to the future control forecast, resulting in a new forecast. This new forecast requires no more time to produce than the time needed to run a data assimilation scheme, as no model integration is necessary. In an observing system simulation experiment (OSSE), a barotropic vorticity model was run to produce a 300-day “nature run.” The same model, run with a different vorticity forcing scheme, served as the forecast model. The model produced 24- and 48-h forecasts for each of the 300 days. The model was initialized every 24 h by assimilating observations of the nature run using a hybrid ensemble Kalman filter–three-dimensional variational data assimilation (3DVAR) scheme. In addition to the control forecast, a 64-member forecast ensemble was generated for each of the 300 days. Every 24 h, given a set of observations, the 64-member ensemble, and the control run, an EnKF was used to create 24-h preemptive forecasts. The preemptive forecasts were more accurate than the unmodified, original 48-h forecasts, though not quite as accurate as the 24-h forecast obtained from a new model integration initialized by assimilating the same observations as were used in the preemptive forecasts. The accuracy of the preemptive forecasts improved significantly when 1) the ensemble-based error statistics used by the EnKF were localized using a Schur product and 2) a model error term was included in the background error covariance matrices.

Evaluation of Surface Analyses and Forecasts with a Multiscale Ensemble Kalman Filter in Regions of Complex Terrain

2011 ◽  
Vol 139 (6) ◽  
pp. 2008-2024 ◽  
Author(s):  
Brian C. Ancell ◽  
Clifford F. Mass ◽  
Gregory J. Hakim
Keyword(s):  
Kalman Filter ◽  
High Resolution ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Complex Terrain ◽  
Error Variance ◽  
Small Scale ◽  
Observation Error ◽  
Grid Spacing ◽  
Background Error

Abstract Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30–50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%–30%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested.

scholarly journals Simulation of Monsoon Depressions Using WRF-VAR: Impact of Different Background Error Statistics and Lateral Boundary Conditions

2014 ◽  
Vol 142 (10) ◽  
pp. 3586-3613 ◽  
Author(s):  
A. Routray ◽  
S. C. Kar ◽  
P. Mali ◽  
K. Sowjanya
Keyword(s):  
Data Assimilation ◽  
Boundary Conditions ◽  
Initial Conditions ◽  
Variational Data Assimilation ◽  
Error Statistics ◽  
Background Error ◽  
Single Observation ◽  
Monsoon Depressions ◽  
Data Assimilation System ◽  
Assimilation System

Abstract In a variational data assimilation system, background error statistics (BES) spread the influence of the observations in space and filter analysis increments through dynamic balance or statistical relationships. In a data-sparse region such as the Bay of Bengal, BES play an important role in defining the location and structure of monsoon depressions (MDs). In this study, the Indian-region-specific BES have been computed for the Weather Research and Forecasting (WRF) three-dimensional variational data assimilation system. A comparative study using single observation tests is carried out using the computed BES and global BES within the WRF system. Both sets of BES are used in the assimilation cycles and forecast runs for simulating the meteorological features associated with the MDs. Numerical experiments have been conducted to assess the relative impact of various BES in the analysis and simulations of the MDs. The results show that use of regional BES in the assimilation cycle has a positive impact on the prediction of the location, propagation, and development of rainbands associated with the MDs. The track errors of MDs are smaller when domain-specific BES are used in the assimilation cycle. Additional experiments have been conducted using data from the Interim European Centre for Medium-Range Weather Forecasts Re-Analysis (ERA-Interim) as initial and boundary conditions (IBCs) in the assimilation cycle. The results indicate that the use of domain-dependent BES and high-resolution ERA-I data as IBCs further improved the initial conditions for the model leading to better forecasts of the MDs.

E4DVar: Coupling an Ensemble Kalman Filter with Four-Dimensional Variational Data Assimilation in a Limited-Area Weather Prediction Model

2012 ◽  
Vol 140 (2) ◽  
pp. 587-600 ◽  
Author(s):  
Meng Zhang ◽  
Fuqing Zhang
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Prediction Model ◽  
Ensemble Kalman Filter ◽  
Weather Prediction ◽  
Coupled System ◽  
Limited Area ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Hybrid Data Assimilation

A hybrid data assimilation approach that couples the ensemble Kalman filter (EnKF) and four-dimensional variational (4DVar) methods is implemented for the first time in a limited-area weather prediction model. In this coupled system, denoted E4DVar, the EnKF and 4DVar systems run in parallel while feeding into each other. The multivariate, flow-dependent background error covariance estimated from the EnKF ensemble is used in the 4DVar minimization and the ensemble mean in the EnKF analysis is replaced by the 4DVar analysis, while updating the analysis perturbations for the next cycle of ensemble forecasts with the EnKF. Therefore, the E4DVar can obtain flow-dependent information from both the explicit covariance matrix derived from ensemble forecasts, as well as implicitly from the 4DVar trajectory. The performance of an E4DVar system is compared with the uncoupled 4DVar and EnKF for a limited-area model by assimilating various conventional observations over the contiguous United States for June 2003. After verifying the forecasts from each analysis against standard sounding observations, it is found that the E4DVar substantially outperforms both the EnKF and 4DVar during this active summer month, which featured several episodes of severe convective weather. On average, the forecasts produced from E4DVar analyses have considerably smaller errors than both of the stand-alone EnKF and 4DVar systems for forecast lead times up to 60 h.

Intercomparison of an Ensemble Kalman Filter with Three- and Four-Dimensional Variational Data Assimilation Methods in a Limited-Area Model over the Month of June 2003

2011 ◽  
Vol 139 (2) ◽  
pp. 566-572 ◽  
Author(s):  
Meng Zhang ◽  
Fuqing Zhang ◽  
Xiang-Yu Huang ◽  
Xin Zhang
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Error ◽  
Wrf Model ◽  
Warm Season ◽  
Variational Data Assimilation ◽  
Lead Times ◽  
Limited Area ◽  
Forecast Errors

Abstract This study compares the performance of an ensemble Kalman filter (EnKF) with both the three-dimensional and four-dimensional variational data assimilation (3DVar and 4DVar) methods of the Weather Research and Forecasting (WRF) model over the contiguous United States in a warm-season month (June) of 2003. The data assimilated every 6 h include conventional sounding and surface observations as well as data from wind profilers, ships and aircraft, and the cloud-tracked winds from satellites. The performances of these methods are evaluated through verifying the 12- to 72-h forecasts initialized twice daily from the analysis of each method against the standard sounding observations. It is found that 4DVar has consistently smaller error than that of 3DVar for winds and temperature at all forecast lead times except at 60 and 72 h when their forecast errors become comparable in amplitude, while the two schemes have similar performance in moisture at all lead times. The forecast error of the EnKF is comparable to that of the 4DVar at 12–36-h lead times, both of which are substantially smaller than that of the 3DVar, despite the fact that 3DVar fits the sounding observations much more closely at the analysis time. The advantage of the EnKF becomes even more evident at 48–72-h lead times; the 72-h forecast error of the EnKF is comparable in magnitude to the 48-h error of 3DVar/4DVar.

scholarly journals Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

2011 ◽  
Vol 139 (11) ◽  
pp. 3389-3404 ◽  
Author(s):  
Thomas Milewski ◽  
Michel S. Bourqui
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Climate Model ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

scholarly journals Singular-Vector-Based Covariance Propagation in a Quasigeostrophic Assimilation System

2005 ◽  
Vol 133 (5) ◽  
pp. 1295-1310 ◽  
Author(s):  
Alexander Beck ◽  
Martin Ehrendorfer
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Variational Data Assimilation ◽  
Dynamic Information ◽  
Dynamic Background ◽  
Background Error ◽  
Singular Vectors ◽  
Reduced Rank ◽  
The Impact ◽  
Static Background

Abstract Variational data assimilation systems require the specification of the covariances of background and observation errors. Although the specification of the background-error covariances has been the subject of intense research, current operational data assimilation systems still rely on essentially static and thus flow-independent background-error covariances. At least theoretically, it is possible to use flow-dependent background-error covariances in four-dimensional variational data assimilation (4DVAR) through exploiting the connection between variational data assimilation and estimation theory. This paper reports on investigations concerning the impact of flow-dependent background-error covariances in an idealized 4DVAR system that, based on quasigeostrophic dynamics, assimilates artificial observations. The main emphasis is placed on quantifying the improvement in analysis quality that is achievable in 4DVAR through the use of flow-dependent background-error covariances. Flow dependence is achieved through dynamical error-covariance evolution based on singular vectors in a reduced-rank approach, referred to as reduced-rank Kalman filter (RRKF). The RRKF yields partly dynamic background-error covariances through blending static and dynamic information, where the dynamic information is obtained from error evolution in a subspace of dimension k (defined here through the singular vectors) that may be small compared to the dimension of the model’s phase space n, which is equal to 1449 in the system investigated here. The results show that the use of flow-dependent background-error covariances based on the RRKF leads to improved analyses compared to a system using static background-error statistics. That latter system uses static background-error covariances that are carefully tuned given the model dynamics and the observational information available. It is also shown that the performance of the RRKF approaches the performance of the extended Kalman filter, as k approaches n. Results therefore support the hypothesis that significant analysis improvement is possible through the use of flow-dependent background-error covariances given that a sufficiently large number (here on the order of n/10) of singular vectors is used.

Sign in / Sign up

Export Citation Format

Share Document