Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

Thomas Milewski; Michel S. Bourqui

doi:10.1175/2011mwr3540.1

Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

Monthly Weather Review ◽

10.1175/2011mwr3540.1 ◽

2011 ◽

Vol 139 (11) ◽

pp. 3389-3404 ◽

Cited By ~ 17

Author(s):

Thomas Milewski ◽

Michel S. Bourqui

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Climate Model ◽

Observation Error ◽

Background Error ◽

Error Covariance Matrix ◽

Error Covariance ◽

Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

Balance and Ensemble Kalman Filter Localization Techniques

Monthly Weather Review ◽

10.1175/2010mwr3328.1 ◽

2011 ◽

Vol 139 (2) ◽

pp. 511-522 ◽

Cited By ~ 136

Author(s):

Steven J. Greybush ◽

Eugenia Kalnay ◽

Takemasa Miyoshi ◽

Kayo Ide ◽

Brian R. Hunt

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Weather Prediction ◽

Length Scale ◽

Observation Error ◽

Long Distance ◽

Error Covariance Matrix ◽

Error Covariance

Abstract In ensemble Kalman filter (EnKF) data assimilation, localization modifies the error covariance matrices to suppress the influence of distant observations, removing spurious long-distance correlations. In addition to allowing efficient parallel implementation, this takes advantage of the atmosphere’s lower dimensionality in local regions. There are two primary methods for localization. In B localization, the background error covariance matrix elements are reduced by a Schur product so that correlations between grid points that are far apart are removed. In R localization, the observation error covariance matrix is multiplied by a distance-dependent function, so that far away observations are considered to have infinite error. Successful numerical weather prediction depends upon well-balanced initial conditions to avoid spurious propagation of inertial-gravity waves. Previous studies note that B localization can disrupt the relationship between the height gradient and the wind speed of the analysis increments, resulting in an analysis that can be significantly ageostrophic. This study begins with a comparison of the accuracy and geostrophic balance of EnKF analyses using no localization, B localization, and R localization with simple one-dimensional balanced waves derived from the shallow-water equations, indicating that the optimal length scale for R localization is shorter than for B localization, and that for the same length scale R localization is more balanced. The comparison of localization techniques is then expanded to the Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY) global atmospheric model. Here, natural imbalance of the slow manifold must be contrasted with undesired imbalance introduced by data assimilation. Performance of the two techniques is comparable, also with a shorter optimal localization distance for R localization than for B localization.

An estimate of the inflation factor and analysis sensitivity in the ensemble Kalman filter

Nonlinear Processes in Geophysics ◽

10.5194/npg-24-329-2017 ◽

2017 ◽

Vol 24 (3) ◽

pp. 329-341 ◽

Cited By ~ 1

Author(s):

Guocan Wu ◽

Xiaogu Zheng

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Forecast Error ◽

Model Errors ◽

Error Covariance Matrix ◽

Spatially Correlated ◽

Error Covariance ◽

Inflation Factor ◽

Analysis Error

Abstract. The ensemble Kalman filter (EnKF) is a widely used ensemble-based assimilation method, which estimates the forecast error covariance matrix using a Monte Carlo approach that involves an ensemble of short-term forecasts. While the accuracy of the forecast error covariance matrix is crucial for achieving accurate forecasts, the estimate given by the EnKF needs to be improved using inflation techniques. Otherwise, the sampling covariance matrix of perturbed forecast states will underestimate the true forecast error covariance matrix because of the limited ensemble size and large model errors, which may eventually result in the divergence of the filter. In this study, the forecast error covariance inflation factor is estimated using a generalized cross-validation technique. The improved EnKF assimilation scheme is tested on the atmosphere-like Lorenz-96 model with spatially correlated observations, and is shown to reduce the analysis error and increase its sensitivity to the observations.

Data assimilation with multiple types of observation boreholes via the ensemble Kalman filter embedded within stochastic moment equations

Hydrology and Earth System Sciences ◽

10.5194/hess-25-1689-2021 ◽

2021 ◽

Vol 25 (4) ◽

pp. 1689-1709

Author(s):

Chuan-An Xia ◽

Xiaodong Luo ◽

Bill X. Hu ◽

Monica Riva ◽

Alberto Guadagnini

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Type A ◽

Observation Error ◽

Type B ◽

Moment Equations ◽

Error Covariance Matrix ◽

Monitoring Wells ◽

Error Covariance

Abstract. We employ an approach based on the ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices corresponding to (i) multi-node monitoring wells equipped with packers (Type A) and (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality, (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality, (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when a simplified flow model is adopted, and (iv) when compared with the standard Monte Carlo-based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

Data assimilation with multiple types of observation boreholes via ensemble Kalman filter embedded within stochastic moment equations

10.5194/hess-2020-588 ◽

2020 ◽

Author(s):

Chuan-An Xia ◽

Xiaodong Luo ◽

Bill X. Hu ◽

Monica Riva ◽

Alberto Guadagnini

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Type A ◽

Observation Error ◽

Type B ◽

Moment Equations ◽

Error Covariance Matrix ◽

Monitoring Wells ◽

Error Covariance

Abstract. We employ an approach based on ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices, corresponding to (i) multi-node monitoring wells equipped with packers (Type A), (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality; (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality; (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when an incomplete/simplified flow model is adopted; and (iv) when compared with the standard Monte Carlo -based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

Sampling Errors in Ensemble Kalman Filtering. Part I: Theory

Monthly Weather Review ◽

10.1175/2007mwr2323.1 ◽

2008 ◽

Vol 136 (8) ◽

pp. 3035-3049 ◽

Cited By ~ 35

Author(s):

William Sacher ◽

Peter Bartello

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Test Case ◽

Sampling Errors ◽

Background Error ◽

Theoretical Understanding ◽

Error Covariance Matrix ◽

Error Covariance ◽

Filter Divergence

Abstract This paper discusses the quality of the analysis given by the ensemble Kalman filter in a perfect model context when ensemble sizes are limited. The overall goal is to improve the theoretical understanding of the problem of systematic errors in the analysis variance due to the limited size of the ensemble, as well as the potential of the so-called double-ensemble Kalman filter, covariance inflation, and randomly perturbed analysis techniques to produce a stable analysis—that is to say, one not subject to filter divergence. This is achieved by expressing the error of the ensemble mean and the analysis error covariance matrix in terms of the sampling noise in the background error covariance matrix (owing to the finite ensemble estimation) and by comparing these errors for all methods. Theoretical predictions are confirmed with a simple scalar test case. In light of the analytical results obtained, the expression of the optimal covariance inflation factor is proposed in terms of the limited ensemble size and the Kalman gain.

An Extension of the Ensemble Kalman Filter for Estimating the Observation Error Covariance Matrix Based on the Variational Bayes’s Method

Monthly Weather Review ◽

10.1175/mwr-d-16-0139.1 ◽

2016 ◽

Vol 145 (1) ◽

pp. 199-213 ◽

Cited By ~ 5

Author(s):

Akio Nakabayashi ◽

Genta Ueno

Keyword(s):

Kalman Filter ◽

Maximum Likelihood ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Numerical Experiments ◽

Observation Error ◽

Error Covariance Matrix ◽

Error Covariance ◽

Long Run ◽

The Stability

Abstract This paper presents an extension of the ensemble Kalman filter (EnKF) that can simultaneously estimate the state vector and the observation error covariance matrix by using the variational Bayes’s (VB) method. In numerical experiments, this capability is examined for a time-variant observation error covariance matrix, and it is noteworthy that this method works well even when the true observation error covariance matrix is nondiagonal. In addition, two complementary studies are presented. First, the stability of a long-run assimilation is demonstrated when there are unmodeled disturbances. Second, a maximum-likelihood (ML) method is derived and demonstrated for optimizing the hyperparameters used in this method.

Evaluation of Surface Analyses and Forecasts with a Multiscale Ensemble Kalman Filter in Regions of Complex Terrain

Monthly Weather Review ◽

10.1175/2010mwr3612.1 ◽

2011 ◽

Vol 139 (6) ◽

pp. 2008-2024 ◽

Cited By ~ 24

Author(s):

Brian C. Ancell ◽

Clifford F. Mass ◽

Gregory J. Hakim

Keyword(s):

Kalman Filter ◽

High Resolution ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Complex Terrain ◽

Error Variance ◽

Small Scale ◽

Observation Error ◽

Grid Spacing ◽

Background Error

Abstract Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30–50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%–30%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested.

Using analysis state to construct a forecast error covariance matrix in ensemble Kalman filter assimilation

Advances in Atmospheric Sciences ◽

10.1007/s00376-012-2133-5 ◽

2013 ◽

Vol 30 (5) ◽

pp. 1303-1312 ◽

Cited By ~ 7

Author(s):

Xiaogu Zheng ◽

Guocan Wu ◽

Shupeng Zhang ◽

Xiao Liang ◽

Yongjiu Dai ◽

...

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Forecast Error ◽

Error Covariance Matrix ◽

Error Covariance

Efficient Adaptive Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to the Control of Ocean Mesoscale Signals

Monthly Weather Review ◽

10.1175/2009mwr3085.1 ◽

2010 ◽

Vol 138 (3) ◽

pp. 932-950 ◽

Cited By ~ 22

Author(s):

Jean-Michel Brankart ◽

Emmanuel Cosme ◽

Charles-Emmanuel Testut ◽

Pierre Brasseur ◽

Jacques Verron

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Error Estimates ◽

Forecast Error ◽

Observation Error ◽

Square Root ◽

Error Statistics ◽

Error Covariance Matrix ◽

Error Covariance ◽

Optimal Estimates

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

An improved variational Data Assimilation method for ocean models with limited number of observations.

10.5194/egusphere-egu2020-2683 ◽

2020 ◽

Author(s):

Lewis Sampson ◽

Jose M. Gonzalez-Ondina ◽

Georgy Shapiro

Keyword(s):

Data Assimilation ◽

Covariance Matrix ◽

Observational Data ◽

Inner Product ◽

Product Analysis ◽

Background Error ◽

Error Covariance Matrix ◽

Error Covariance ◽

Ocean Prediction ◽

The Stability

Data assimilation (DA) is a critical component for most state-of-the-art ocean prediction systems, which optimally combines model data and observational measurements to obtain an improved estimate of the modelled variables, by minimizing a cost function. The calculation requires the knowledge of the background error covariance matrix (BECM) as a weight for the quality of the model results, and an observational error covariance matrix (OECM) which weights the observational data.Computing the BECM would require knowing the true values of the physical variables, which is not feasible. Instead, the BECM is estimated from model results and observations by using methods like National Meteorological Centre (NMC) or the Hollingsworth and L&#246;nnberg (1984) (H-L). These methods have some shortcomings which make them unfit in some situations, which includes being fundamentally one-dimensional and making a suboptimal use of observations.We have produced a novel method for error estimation, using an analysis of observations minus background data (innovations), which attempts to improve on some of these shortcomings. In particular, our method better infers information from observations, requiring less data to produce statistically robust results. We do this by minimizing a linear combination of functions to fit the data using a specifically tailored inner product, referred to as an inner product analysis (IPA).We are able to produce quality BECM estimations even in data sparse domains, with notably better results in conditions of scarce observational data. By using a sample of observations, with decreasing sample size, we show that the stability and efficiency of our method, when compared to that of the H-L approach, does not deteriorate nearly as much as the number of data points decrease. We have found that we are able to continually produce error estimates with a reduced set of data, whereas the H-L method will begin to produce spurious values for smaller samples.Our method works very well in combination with standard tools like NEMOVar by providing the required standard deviations and length-scales ratios. We have successfully ran this in the Arabian Sea for multiple seasons and compared the results with the H-L (in optimal conditions, when plenty of data is available), spatially the methods perform equally well. When we look at the root mean square error (RMSE) we see very similar performances, with each method giving better results for some seasons and worse for others.