Adaptive Ensemble Covariance Localization in Ensemble 4D-VAR State Estimation

2011 ◽  
Vol 139 (4) ◽  
pp. 1241-1255 ◽  
Author(s):  
Craig H. Bishop ◽  
Daniel Hodyss
Keyword(s):  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Large Scale ◽  
Far Field ◽  
Dimensional Error ◽  
Error Covariance ◽  
Separable Variable ◽  
Significant Difference ◽  
Forecasting System

Abstract An adaptive ensemble covariance localization technique, previously used in “local” forms of the ensemble Kalman filter, is extended to a global ensemble four-dimensional variational data assimilation (4D-VAR) scheme. The purely adaptive part of the localization matrix considered is given by the element-wise square of the correlation matrix of a smoothed ensemble of streamfunction perturbations. It is found that these purely adaptive localization functions have spurious far-field correlations as large as 0.1 with a 128-member ensemble. To attenuate the spurious features of the purely adaptive localization functions, the authors multiply the adaptive localization functions with very broadscale nonadaptive localization functions. Using the Navy’s operational ensemble forecasting system, it is shown that the covariance localization functions obtained by this approach adapt to spatially anisotropic aspects of the flow, move with the flow, and are free of far-field spurious correlations. The scheme is made computationally feasible by (i) a method for inexpensively generating the square root of an adaptively localized global four-dimensional error covariance model in terms of products or modulations of smoothed ensemble perturbations with themselves and with raw ensemble perturbations, and (ii) utilizing algorithms that speed ensemble covariance localization when localization functions are separable, variable-type independent, and/or large scale. In spite of the apparently useful characteristics of adaptive localization, single analysis/forecast experiments assimilating 583 200 observations over both 6- and 12-h data assimilation windows failed to identify any significant difference in the quality of the analyses and forecasts obtained using nonadaptive localization from that obtained using adaptive localization.

TREATMENT OF THE ERROR DUE TO UNRESOLVED SCALES IN SEQUENTIAL DATA ASSIMILATION

2011 ◽  
Vol 21 (12) ◽  
pp. 3619-3626 ◽  
Author(s):  
ALBERTO CARRASSI ◽  
STÉPHANE VANNITSEM
Keyword(s):  
Dynamical System ◽  
Kalman Filter ◽  
Data Assimilation ◽  
Large Scale ◽  
Model Error ◽  
Environmental Modeling ◽  
Sequential Data ◽  
Chaotic Dynamical System ◽  
Error Covariance ◽  
Sequential Data Assimilation

In this paper, a method to account for model error due to unresolved scales in sequential data assimilation, is proposed. An equation for the model error covariance required in the extended Kalman filter update is derived along with an approximation suitable for application with large scale dynamics typical in environmental modeling. This approach is tested in the context of a low order chaotic dynamical system. The results show that the filter skill is significantly improved by implementing the proposed scheme for the treatment of the unresolved scales.

Theoretical Application of Ensemble Kalman Filter to Adsorptive Solute Cr(VI) Transfer from Soil into Surface Runoff

2014 ◽  
Vol 919-921 ◽  
pp. 1257-1261
Author(s):  
Chao Qun Tan ◽  
Ju Xiu Tong ◽  
Bill X. Hu ◽  
Jin Zhong Yang
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Surface Runoff ◽  
Amplification Factor ◽  
Forecast Error ◽  
Assimilation Effect ◽  
Solute Transfer ◽  
Error Covariance ◽  
Theoretical Application

This paper mainly discusses some details when applying data assimilation method via an ensemble Kalman filter (EnKF) to improve prediction of adsorptive solute Cr(VI) transfer from soil into runoff. Based on this work, we could make better use of our theoretical model to predict adsorptive solute transfer from soil into surface runoff in practice. The results show that the ensemble number of 100 is reasonable, considering assimilation effect and efficiency after selecting its number from 25 to 225 at an interval of 25. While the initial ensemble value makes little difference to data assimilation (DA) results. Besides, DA results could be improved by multiplying an amplification factor to forecast error covariance matrix due to underestimation of forecast error.

scholarly journals Scale-Dependent Representation of the Information Content of Observations in the Global Ensemble Kalman Filter Data Assimilation

2016 ◽  
Vol 144 (8) ◽  
pp. 2927-2945
Author(s):  
Nedjeljka Žagar ◽  
Jeffrey Anderson ◽  
Nancy Collins ◽  
Timothy Hoar ◽  
Kevin Raeder ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Information Content ◽  
Ensemble Kalman Filter ◽  
Large Scale ◽  
Variance Reduction ◽  
Weather Prediction ◽  
Current Data ◽  
Model Framework ◽  
The Tropics

Abstract Global data assimilation systems for numerical weather prediction (NWP) are characterized by significant uncertainties in tropical analysis fields. Furthermore, the largest spread of global ensemble forecasts in the short range on all scales is in the tropics. The presented results suggest that these properties hold even in the perfect-model framework and the ensemble Kalman filter data assimilation with a globally homogeneous network of wind and temperature profiles. The reasons for this are discussed by using the modal analysis, which provides information about the scale dependency of analysis and forecast uncertainties and information about the efficiency of data assimilation to reduce the prior uncertainties in the balanced and inertio-gravity dynamics. The scale-dependent representation of variance reduction of the prior ensemble by the data assimilation shows that the peak efficiency of data assimilation is on the synoptic scales in the midlatitudes that are associated with quasigeostrophic dynamics. In contrast, the variance associated with the inertia–gravity modes is less successfully reduced on all scales. A smaller information content of observations on planetary scales with respect to the synoptic scales is discussed in relation to the large-scale tropical uncertainties that current data assimilation methodologies do not address successfully. In addition, it is shown that a smaller reduction of the large-scale uncertainties in the prior state for NWP in the tropics than in the midlatitudes is influenced by the applied radius for the covariance localization.

scholarly journals Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

2011 ◽  
Vol 139 (11) ◽  
pp. 3389-3404 ◽  
Author(s):  
Thomas Milewski ◽  
Michel S. Bourqui
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Climate Model ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

scholarly journals On Serial Observation Processing in Localized Ensemble Kalman Filters

2015 ◽  
Vol 143 (5) ◽  
pp. 1554-1567 ◽  
Author(s):  
Lars Nerger
Keyword(s):  
Ensemble Kalman Filter ◽  
Kalman Filters ◽  
Large Scale ◽  
The State ◽  
Ensemble Kalman Filters ◽  
Filter Performance ◽  
Error Covariance ◽  
Analysis Process ◽  
Scale Models ◽  
Serial Observation

Abstract Ensemble square root filters can either assimilate all observations that are available at a given time at once, or assimilate the observations in batches or one at a time. For large-scale models, the filters are typically applied with a localized analysis step. This study demonstrates that the interaction of serial observation processing and localization can destabilize the analysis process, and it examines under which conditions the instability becomes significant. The instability results from a repeated inconsistent update of the state error covariance matrix that is caused by the localization. The inconsistency is present in all ensemble Kalman filters, except for the classical ensemble Kalman filter with perturbed observations. With serial observation processing, its effect is small in cases when the assimilation changes the ensemble of model states only slightly. However, when the assimilation has a strong effect on the state estimates, the interaction of localization and serial observation processing can significantly deteriorate the filter performance. In realistic large-scale applications, when the assimilation changes the states only slightly and when the distribution of the observations is irregular and changing over time, the instability is likely not significant.

Scale-Dependent Background Error Covariance Localization: Evaluation in a Global Deterministic Weather Forecasting System

2018 ◽  
Vol 146 (5) ◽  
pp. 1367-1381 ◽  
Author(s):  
Jean-François Caron ◽  
Mark Buehner
Keyword(s):  
Data Assimilation ◽  
Weather Forecasting ◽  
Spatial Scales ◽  
Winter Period ◽  
Wind Fields ◽  
Horizontal Scale ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Forecasting System

Abstract Scale-dependent localization (SDL) consists of applying the appropriate (i.e., different) amount of localization to different ranges of background error covariance spatial scales while simultaneously assimilating all of the available observations. The SDL method proposed by Buehner and Shlyaeva for ensemble–variational (EnVar) data assimilation was tested in a 3D-EnVar version of the Canadian operational global data assimilation system. It is shown that a horizontal-scale-dependent horizontal localization leads to implicit vertical-level-dependent, variable-dependent, and location-dependent horizontal localization. The results from data assimilation cycles show that horizontal-scale-dependent horizontal covariance localization is able to improve the forecasts up to day 5 in the Northern Hemisphere extratropical summer period and up to day 7 in the Southern Hemisphere extratropical winter period. In the tropics, use of SDL results in improvements similar to what can be obtained by increasing the uniform amount of spatial localization. An investigation of the dynamical balance in the resulting analysis increments demonstrates that SDL does not further harm the balance between the mass and the rotational wind fields, as compared to the traditional localization approach. Potential future applications for the SDL method are also discussed.

scholarly journals A Fuzzy-Logic-Based Covariance Localization Method in Data Assimilation

Atmosphere ◽  
2020 ◽  
Vol 11 (10) ◽  
pp. 1055
Author(s):  
Yulong Bai ◽  
Xiaoyan Ma ◽  
Lin Ding
Keyword(s):  
Fuzzy Logic ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Atmospheric Model ◽  
Localization Method ◽  
Error Covariance ◽  
Logic Control ◽  
Power Spectral ◽  
Performance Evaluation Index ◽  
Lorenz 96

In ensemble data assimilation systems, the impracticalities of full sampling and systematic error often lead to spurious correlations between two variables with low actual correlations. To solve these problems, researchers have previously proposed a covariance localization (CL) method, which mainly involves the Schur product between a state error covariance matrix and a distance-based correlation matrix. Although this CL method can reduce spurious correlations to a certain extent, observational data remain difficult to be used effectively, which results in unreasonable assimilation. In this study, we develop a new CL method coupled with a fuzzy logic control algorithm, which we call the covariance fuzzy (CF) method. The proposed CF method is a distance-based localization method with “fuzzy” vanishing correlations in data assimilation (DA) systems. To verify the effectiveness of the new algorithm, we conducted a set of experiments using an ensemble Kalman filter (EnKF) that combines the nonlinear Lorenz-96 model or the quasi-geostrophic (QG) models. First, the performances of the CL and CF methods are discussed with respect to different strength forcings, ensemble sizes, and covariance inflation factors. The experimental results show that the proposed CF method can obtain a more effective observation weight than the CL method and can reduce the errors caused by spurious correlations. Additionally, using power spectral density (PSD) as a performance evaluation index, the robustness of the proposed fuzzy logic localization method is demonstrated. However, the application of the fuzzy logic-based localization methodology to a real atmospheric model remains to be tested.

scholarly journals Background error covariance with balance constraints for aerosol species and applications in data assimilation

2015 ◽  
Vol 8 (11) ◽  
pp. 10053-10088
Author(s):  
Z. Zang ◽  
Z. Hao ◽  
Y. Li ◽  
X. Pan ◽  
W. You ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Elemental Carbon ◽  
Statistical Regression ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Balance Analysis ◽  
Forecasting System ◽  
Cross Correlations ◽  
The Impact

Abstract. Balance constraints are important for a background error covariance (BEC) in data assimilation to spread information between different variables and produce balance analysis fields. Using statistical regression, we develop the balance constraint for the BEC of aerosol variables and apply it to a data assimilation and forecasting system for the WRF/Chem model. One-month products from the WRF/Chem model are employed for BEC statistics with the NMC method. The cross-correlations among the original variables are generally high. The highest correlation between elemental carbon and organic carbon without balance constraints is approximately 0.9. However, the correlations for the unbalanced variables are less than 0.2 with the balance constraints. Data assimilation and forecasting experiments for evaluating the impact of balance constraints are performed with the observations of the surface PM2.5 concentrations and speciated concentrations along an aircraft flight track. The speciated increments of the experiment with balance constraints are more coincident than the speciated increments of the experiment without balance constraints, for the observation information can spread across variables by balance constraints in the former experiment. The forecast results of the experiment with balance constraints show significant and durable improvements from the 3rd hour to the 18th hour compared with the forecast results of the experiment without the balance constraints. However, the forecasts of these two experiments are similar during the first 3 h. The results suggest that the balance constraint is significantly positive for the aerosol assimilation and forecasting.

scholarly journals A Comparison between the 4DVAR and the Ensemble Kalman Filter Techniques for Radar Data Assimilation

2005 ◽  
Vol 133 (11) ◽  
pp. 3081-3094 ◽  
Author(s):  
A. Caya ◽  
J. Sun ◽  
C. Snyder
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Model ◽  
Radar Data ◽  
Forecast Errors ◽  
Error Covariance ◽  
Convective Scale ◽  
Cloud Resolving Model ◽  
Comparable Accuracy

Abstract A four-dimensional variational data assimilation (4DVAR) algorithm is compared to an ensemble Kalman filter (EnKF) for the assimilation of radar data at the convective scale. Using a cloud-resolving model, simulated, imperfect radar observations of a supercell storm are assimilated under the assumption of a perfect forecast model. Overall, both assimilation schemes perform well and are able to recover the supercell with comparable accuracy, given radial-velocity and reflectivity observations where rain was present. 4DVAR produces generally better analyses than the EnKF given observations limited to a period of 10 min (or three volume scans), particularly for the wind components. In contrast, the EnKF typically produces better analyses than 4DVAR after several assimilation cycles, especially for model variables not functionally related to the observations. The advantages of the EnKF in later cycles arise at least in part from the fact that the 4DVAR scheme implemented here does not use a forecast from a previous cycle as background or evolve its error covariance. Possible reasons for the initial advantage of 4DVAR are deficiencies in the initial ensemble used by the EnKF, the temporal smoothness constraint used in 4DVAR, and nonlinearities in the evolution of forecast errors over the assimilation window.

A Multicase Comparative Assessment of the Ensemble Kalman Filter for Assimilation of Radar Observations. Part II: Short-Range Ensemble Forecasts

2010 ◽  
Vol 138 (4) ◽  
pp. 1273-1292 ◽  
Author(s):  
Altuğ Aksoy ◽  
David C. Dowell ◽  
Chris Snyder
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Cloud Model ◽  
Wrf Model ◽  
Radar Data ◽  
Negative Effects ◽  
Ensemble Forecasts ◽  
Radar Observations

Abstract The quality of convective-scale ensemble forecasts, initialized from analysis ensembles obtained through the assimilation of radar observations using an ensemble Kalman filter (EnKF), is investigated for cases whose behaviors span supercellular, linear, and multicellular organization. This work is the companion to Part I, which focused on the quality of analyses during the 60-min analysis period. Here, the focus is on 30-min ensemble forecasts initialized at the end of that period. As in Part I, the Weather Research and Forecasting (WRF) model is employed as a simplified cloud model at 2-km horizontal grid spacing. Various observation-space and state-space verification metrics, computed both for ensemble means and individual ensemble members, are employed to assess the quality of ensemble forecasts comparatively across cases. While the cases exhibit noticeable differences in predictability, the forecast skill in each case, as measured by various metrics, decays on a time scale of tens of minutes. The ensemble spread also increases rapidly but significant outlier members or clustering among members are not encountered. Forecast quality is seen to be influenced to varying degrees by the respective initial soundings. While radar data assimilation is able to partially mitigate some of the negative effects in some situations, the supercell case, in particular, remains difficult to predict even after 60 min of data assimilation.

Sign in / Sign up

Export Citation Format

Share Document