Enhancing the impact of IASI observations through an updated observation-error covariance matrix

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

Download Full-text

Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

Monthly Weather Review ◽

10.1175/2011mwr3540.1 ◽

2011 ◽

Vol 139 (11) ◽

pp. 3389-3404 ◽

Cited By ~ 17

Author(s):

Thomas Milewski ◽

Michel S. Bourqui

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Climate Model ◽

Observation Error ◽

Background Error ◽

Error Covariance Matrix ◽

Error Covariance ◽

Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

Download Full-text

Generalized background error covariance matrix model (GEN_BE v2.0)

Geoscientific Model Development ◽

10.5194/gmd-8-669-2015 ◽

2015 ◽

Vol 8 (3) ◽

pp. 669-696 ◽

Cited By ~ 20

Author(s):

G. Descombes ◽

T. Auligné ◽

F. Vandenberghe ◽

D. M. Barker ◽

J. Barré

Keyword(s):

Data Assimilation ◽

Covariance Matrix ◽

Chemical Species ◽

Variational Data Assimilation ◽

Control Variables ◽

Background Error ◽

Error Covariance Matrix ◽

Background Error Covariance ◽

Error Covariance ◽

The Impact

Abstract. The specification of state background error statistics is a key component of data assimilation since it affects the impact observations will have on the analysis. In the variational data assimilation approach, applied in geophysical sciences, the dimensions of the background error covariance matrix (B) are usually too large to be explicitly determined and B needs to be modeled. Recent efforts to include new variables in the analysis such as cloud parameters and chemical species have required the development of the code to GENerate the Background Errors (GEN_BE) version 2.0 for the Weather Research and Forecasting (WRF) community model. GEN_BE allows for a simpler, flexible, robust, and community-oriented framework that gathers methods used by some meteorological operational centers and researchers. We present the advantages of this new design for the data assimilation community by performing benchmarks of different modeling of B and showing some of the new features in data assimilation test cases. As data assimilation for clouds remains a challenge, we present a multivariate approach that includes hydrometeors in the control variables and new correlated errors. In addition, the GEN_BE v2.0 code is employed to diagnose error parameter statistics for chemical species, which shows that it is a tool flexible enough to implement new control variables. While the generation of the background errors statistics code was first developed for atmospheric research, the new version (GEN_BE v2.0) can be easily applied to other domains of science and chosen to diagnose and model B. Initially developed for variational data assimilation, the model of the B matrix may be useful for variational ensemble hybrid methods as well.

Download Full-text

Ozone-sensitive channel selection over IASI full spectrum with correlated observation errors for NWP

10.5194/amt-2019-242 ◽

2019 ◽

Author(s):

Olivier Coopmann ◽

Vincent Guidard ◽

Béatrice Josse ◽

Virginie Marécal ◽

Nadia Fourrié

Keyword(s):

Covariance Matrix ◽

Weather Prediction ◽

Channel Selection ◽

Spectral Interval ◽

Observation Error ◽

Full Spectrum ◽

Error Covariance Matrix ◽

Error Covariance ◽

One Year ◽

Selection Of

Abstract. The Infrared Atmospheric Sounding Interferometer (IASI) onboard the Metop satellites provides 8461 channels in the infrared spectrum, covering the spectral interval 645–2760 cm−1 at a resolution of 0.5 cm−1. The high volume of data observation resulting from IASI presents many challenges. In current Numerical Weather Prediction (NWP) models, assimilating all channels is not feasible, due to data transmission, data storage and significant computational costs. One of the methods for reducing the data volume is the channel selection. Many NWP centres use a subset of 314 IASI channels including 15 ozone-sensitive channels. However, this channel selection has been carried out assuming uncorrelated observation errors. In addition, these ozone-sensitive channels have been selected only for ozone information. The objective of this study is to carry out a new selection of IASI ozone-sensitive channels from the full spectrum over a spectral range of 1000–1070 cm−1, in a direct radiance assimilation framework. This selection is done with a full observation error covariance matrix to take into account cross-channel error correlations. A sensitivity method based on the channel spectral sensitivity to variables and a statistical approach based on the Degrees of Freedom for Signal (DFS) have been chosen. To be representative of atmospheric variability, 345 profiles from around the world over a one-year period were selected. The new selection, is evaluated in a One-Dimensional Variational (1D-Var) analyses framework. This selection highlights a new set of 15 IASI ozone-sensitive channels. The results are very encouraging since by adding these 15 channels to 122 operational channels, temperature and humidity analyses are improved by 13.8 % and 20.9 % respectively. Obviously, these 15 channels significantly improve ozone analyses. In addition to considering inter-channel observation error correlations, the channel selection method uses a robust background error covariance matrix that takes into account temperature, humidity and ozone errors using a lagged forecast method over a one-year period. The new selection of IASI ozone-sensitive channels will be soon used in the global 4D-Var ARPEGE (Action de Recherche Petite Echelle Grande Echelle) data assimilation system.

Download Full-text

Generalized Background Error covariance matrix model (GEN_BE v2.0)

Geoscientific Model Development Discussions ◽

10.5194/gmdd-7-4291-2014 ◽

2014 ◽

Vol 7 (4) ◽

pp. 4291-4352

Author(s):

G. Descombes ◽

T. Auligné ◽

F. Vandenberghe ◽

D. M. Barker

Keyword(s):

Data Assimilation ◽

Covariance Matrix ◽

Chemical Species ◽

Variational Data Assimilation ◽

Control Variables ◽

Background Error ◽

Error Covariance Matrix ◽

Background Error Covariance ◽

Error Covariance ◽

The Impact

Abstract. The specification of state background error statistics is a key component of data assimilation since it affects the impact observations will have on the analysis. In the variational data assimilation approach, applied in geophysical sciences, the dimensions of the background error covariance matrix (B) are usually too large to be explicitly determined and B needs to be modeled. Recent efforts to include new variables in the analysis such as cloud parameters and chemical species have required the development of the code to GENerate the Background Errors (GEN_BE) version 2.0 for the Weather Research and Forecasting (WRF) community model to allow for a simpler, flexible, robust, and community-oriented framework that gathers methods used by meteorological operational centers and researchers. We present the advantages of this new design for the data assimilation community by performing benchmarks and showing some of the new features on data assimilation test cases. As data assimilation for clouds remains a challenge, we present a multivariate approach that includes hydrometeors in the control variables and new correlated errors. In addition, the GEN_BE v2.0 code is employed to diagnose error parameter statistics for chemical species, which shows that it is a tool flexible enough to involve new control variables. While the generation of the background errors statistics code has been first developed for atmospheric research, the new version (GEN_BE v2.0) can be easily extended to other domains of science and be chosen as a testbed for diagnostic and new modeling of B. Initially developed for variational data assimilation, the model of the B matrix may be useful for variational ensemble hybrid methods as well.

Download Full-text

Data assimilation with multiple types of observation boreholes via the ensemble Kalman filter embedded within stochastic moment equations

Hydrology and Earth System Sciences ◽

10.5194/hess-25-1689-2021 ◽

2021 ◽

Vol 25 (4) ◽

pp. 1689-1709

Author(s):

Chuan-An Xia ◽

Xiaodong Luo ◽

Bill X. Hu ◽

Monica Riva ◽

Alberto Guadagnini

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Type A ◽

Observation Error ◽

Type B ◽

Moment Equations ◽

Error Covariance Matrix ◽

Monitoring Wells ◽

Error Covariance

Abstract. We employ an approach based on the ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices corresponding to (i) multi-node monitoring wells equipped with packers (Type A) and (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality, (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality, (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when a simplified flow model is adopted, and (iv) when compared with the standard Monte Carlo-based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

Download Full-text

Data assimilation with multiple types of observation boreholes via ensemble Kalman filter embedded within stochastic moment equations

10.5194/hess-2020-588 ◽

2020 ◽

Author(s):

Chuan-An Xia ◽

Xiaodong Luo ◽

Bill X. Hu ◽

Monica Riva ◽

Alberto Guadagnini

Keyword(s):

Kalman Filter ◽

Covariance Matrix ◽

Ensemble Kalman Filter ◽

Type A ◽

Observation Error ◽

Type B ◽

Moment Equations ◽

Error Covariance Matrix ◽

Monitoring Wells ◽

Error Covariance

Abstract. We employ an approach based on ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices, corresponding to (i) multi-node monitoring wells equipped with packers (Type A), (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality; (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality; (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when an incomplete/simplified flow model is adopted; and (iv) when compared with the standard Monte Carlo -based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

Download Full-text