scholarly journals A Projection Method for the Estimation of Error Covariance Matrices for Variational Data Assimilation in Ocean Modelling

2021 ◽  
Vol 9 (12) ◽  
pp. 1461
Author(s):  
Jose M. Gonzalez-Ondina ◽  
Lewis Sampson ◽  
Georgy I. Shapiro
Keyword(s):  
Data Assimilation ◽  
Projection Method ◽  
Covariance Function ◽  
Covariance Matrices ◽  
New Method ◽  
Observation Error ◽  
Covariance Functions ◽  
Error Covariance ◽  
Best Fit ◽  
True Values

Data assimilation methods are an invaluable tool for operational ocean models. These methods are often based on a variational approach and require the knowledge of the spatial covariances of the background errors (differences between the numerical model and the true values) and the observation errors (differences between true and measured values). Since the true values are never known in practice, the error covariance matrices containing values of the covariance functions at different locations, are estimated approximately. Several methods have been devised to compute these matrices, one of the most widely used is the one developed by Hollingsworth and Lönnberg (H-L). This method requires to bin (combine) the data points separated by similar distances, compute covariances in each bin and then to find a best fit covariance function. While being a helpful tool, the H-L method has its limitations. We have developed a new mathematical method for computing the background and observation error covariance functions and therefore the error covariance matrices. The method uses functional analysis which allows to overcome some shortcomings of the H-L method, for example, the assumption of statistical isotropy. It also eliminates the intermediate steps used in the H-L method such as binning the innovations (differences between observations and the model), and the computation of innovation covariances for each bin, before the best-fit curve can be found. We show that the new method works in situations where the standard H-L method experiences difficulties, especially when observations are scarce. It gives a better estimate than the H-L in a synthetic idealised case where the true covariance function is known. We also demonstrate that in many cases the new method allows to use the separable convolution mathematical algorithm to increase the computational speed significantly, up to an order of magnitude. The Projection Method (PROM) also allows computing 2D and 3D covariance functions in addition to the standard 1D case.

scholarly journals Multiscale Representation of Observation Error Statistics in Data Assimilation

Sensors ◽  
2020 ◽  
Vol 20 (5) ◽  
pp. 1460
Author(s):  
Vincent Chabot ◽  
Maëlle Nodet ◽  
Arthur Vidard
Keyword(s):  
Data Assimilation ◽  
Real Life ◽  
Covariance Matrices ◽  
Multiscale Modelling ◽  
Observation Error ◽  
Error Statistics ◽  
Life Data ◽  
Promising Tool ◽  
Error Covariance ◽  
Real Life Data

Accounting for realistic observation errors is a known bottleneck in data assimilation, because dealing with error correlations is complex. Following a previous study on this subject, we propose to use multiscale modelling, more precisely wavelet transform, to address this question. This study aims to investigate the problem further by addressing two issues arising in real-life data assimilation: how to deal with partially missing data (e.g., concealed by an obstacle between the sensor and the observed system), and how to solve convergence issues associated with complex observation error covariance matrices? Two adjustments relying on wavelets modelling are proposed to deal with those, and offer significant improvements. The first one consists of adjusting the variance coefficients in the frequency domain to account for masked information. The second one consists of a gradual assimilation of frequencies. Both of these fully rely on the multiscale properties associated with wavelet covariance modelling. Numerical results on twin experiments show that multiscale modelling is a promising tool to account for correlations in observation errors in realistic applications.

scholarly journals A Review of Innovation-Based Methods to Jointly Estimate Model and Observation Error Covariance Matrices in Ensemble Data Assimilation

2020 ◽  
Vol 148 (10) ◽  
pp. 3973-3994 ◽  
Author(s):  
Pierre Tandeo ◽  
Pierre Ailliot ◽  
Marc Bocquet ◽  
Alberto Carrassi ◽  
Takemasa Miyoshi ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Current Data ◽  
Covariance Matrices ◽  
Estimation Methods ◽  
Observation Error ◽  
Correlated Noise ◽  
Unified Framework ◽  
Error Covariance ◽  
Error Terms ◽  
Observation Space

AbstractData assimilation combines forecasts from a numerical model with observations. Most of the current data assimilation algorithms consider the model and observation error terms as additive Gaussian noise, specified by their covariance matrices and , respectively. These error covariances, and specifically their respective amplitudes, determine the weights given to the background (i.e., the model forecasts) and to the observations in the solution of data assimilation algorithms (i.e., the analysis). Consequently, and matrices significantly impact the accuracy of the analysis. This review aims to present and to discuss, with a unified framework, different methods to jointly estimate the and matrices using ensemble-based data assimilation techniques. Most of the methods developed to date use the innovations, defined as differences between the observations and the projection of the forecasts onto the observation space. These methods are based on two main statistical criteria: 1) the method of moments, in which the theoretical and empirical moments of the innovations are assumed to be equal, and 2) methods that use the likelihood of the observations, themselves contained in the innovations. The reviewed methods assume that innovations are Gaussian random variables, although extension to other distributions is possible for likelihood-based methods. The methods also show some differences in terms of levels of complexity and applicability to high-dimensional systems. The conclusion of the review discusses the key challenges to further develop estimation methods for and . These challenges include taking into account time-varying error covariances, using limited observational coverage, estimating additional deterministic error terms, or accounting for correlated noise.

scholarly journals EnKF and 4D-Var data assimilation with chemical transport model BASCOE (version 05.06)

2016 ◽  
Vol 9 (8) ◽  
pp. 2893-2908 ◽  
Author(s):  
Sergey Skachko ◽  
Richard Ménard ◽  
Quentin Errera ◽  
Yves Christophe ◽  
Simon Chabrillat
Keyword(s):  
Data Assimilation ◽  
Transport Model ◽  
Chemical Transport ◽  
Error Variance ◽  
Model Error ◽  
Chemical Processes ◽  
Observation Error ◽  
Chemical Transport Model ◽  
Error Covariance ◽  
Chemical Tracer

Abstract. We compare two optimized chemical data assimilation systems, one based on the ensemble Kalman filter (EnKF) and the other based on four-dimensional variational (4D-Var) data assimilation, using a comprehensive stratospheric chemistry transport model (CTM). This work is an extension of the Belgian Assimilation System for Chemical ObsErvations (BASCOE), initially designed to work with a 4D-Var data assimilation. A strict comparison of both methods in the case of chemical tracer transport was done in a previous study and indicated that both methods provide essentially similar results. In the present work, we assimilate observations of ozone, HCl, HNO3, H2O and N2O from EOS Aura-MLS data into the BASCOE CTM with a full description of stratospheric chemistry. Two new issues related to the use of the full chemistry model with EnKF are taken into account. One issue is a large number of error variance parameters that need to be optimized. We estimate an observation error variance parameter as a function of pressure level for each observed species using the Desroziers method. For comparison purposes, we apply the same estimate procedure in the 4D-Var data assimilation, where both scale factors of the background and observation error covariance matrices are estimated using the Desroziers method. However, in EnKF the background error covariance is modelled using the full chemistry model and a model error term which is tuned using an adjustable parameter. We found that it is adequate to have the same value of this parameter based on the chemical tracer formulation that is applied for all observed species. This is an indication that the main source of model error in chemical transport model is due to the transport. The second issue in EnKF with comprehensive atmospheric chemistry models is the noise in the cross-covariance between species that occurs when species are weakly chemically related at the same location. These errors need to be filtered out in addition to a localization based on distance. The performance of two data assimilation methods was assessed through an 8-month long assimilation of limb sounding observations from EOS Aura MLS. This paper discusses the differences in results and their relation to stratospheric chemical processes. Generally speaking, EnKF and 4D-Var provide results of comparable quality but differ substantially in the presence of model error or observation biases. If the erroneous chemical modelling is associated with moderately fast chemical processes, but whose lifetimes are longer than the model time step, then EnKF performs better, while 4D-Var develops spurious increments in the chemically related species. If, however, the observation biases are significant, then 4D-Var is more robust and is able to reject erroneous observations while EnKF does not.

scholarly journals Assimilation of surface NO<sub>2</sub> and O<sub>3</sub> observations into the SILAM chemistry transport model

2015 ◽  
Vol 8 (2) ◽  
pp. 191-203 ◽  
Author(s):  
J. Vira ◽  
M. Sofiev
Keyword(s):  
Transport Model ◽  
Time Of Day ◽  
Covariance Matrices ◽  
Atmospheric Composition ◽  
Integrated Modelling ◽  
Daily Maximum ◽  
Observation Error ◽  
Data Set ◽  
Chemistry Transport Model ◽  
Error Covariance

Abstract. This paper describes the assimilation of trace gas observations into the chemistry transport model SILAM (System for Integrated modeLling of Atmospheric coMposition) using the 3D-Var method. Assimilation results for the year 2012 are presented for the prominent photochemical pollutants ozone (O3) and nitrogen dioxide (NO2). Both species are covered by the AirBase observation database, which provides the observational data set used in this study. Attention was paid to the background and observation error covariance matrices, which were obtained primarily by the iterative application of a posteriori diagnostics. The diagnostics were computed separately for 2 months representing summer and winter conditions, and further disaggregated by time of day. This enabled the derivation of background and observation error covariance definitions, which included both seasonal and diurnal variation. The consistency of the obtained covariance matrices was verified using χ2 diagnostics. The analysis scores were computed for a control set of observation stations withheld from assimilation. Compared to a free-running model simulation, the correlation coefficient for daily maximum values was improved from 0.8 to 0.9 for O3 and from 0.53 to 0.63 for NO2.

scholarly journals Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

2011 ◽  
Vol 139 (11) ◽  
pp. 3389-3404 ◽  
Author(s):  
Thomas Milewski ◽  
Michel S. Bourqui
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Climate Model ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

scholarly journals Construction of non-diagonal background error covariance matrices for global chemical data assimilation

2010 ◽  
Vol 3 (4) ◽  
pp. 1783-1827 ◽  
Author(s):  
K. Singh ◽  
M. Jardak ◽  
A. Sandu ◽  
K. Bowman ◽  
M. Lee ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Computational Procedure ◽  
Covariance Matrices ◽  
Chemical Data ◽  
Spatial Correlations ◽  
Transport Models ◽  
Background Error ◽  
Spatial Error ◽  
Background Error Covariance ◽  
Error Covariance

Abstract. Chemical data assimilation attempts to optimally use noisy observations along with imperfect model predictions to produce a better estimate of the chemical state of the atmosphere. It is widely accepted that a key ingredient for successful data assimilation is a realistic estimation of the background error distribution. Particularly important is the specification of the background error covariance matrix, which contains information about the magnitude of the background errors and about their correlations. Most models currently use diagonal background covariance matrices. As models evolve toward finer resolutions, the diagonal background covariance matrices become increasingly inaccurate, since they captures less of the spatial error correlations. This paper discusses an efficient computational procedure for constructing non-diagonal background error covariance matrices which account for the spatial correlations of errors. The benefits of using the non-diagonal covariance matrices for variational data assimilation with chemical transport models are illustrated.

scholarly journals Assimilation of surface NO<sub>2</sub> and O<sub>3</sub> observations into the SILAM chemistry transport model

2014 ◽  
Vol 7 (4) ◽  
pp. 5589-5621 ◽  
Author(s):  
J. Vira ◽  
M. Sofiev
Keyword(s):  
Transport Model ◽  
Model Simulation ◽  
Time Of Day ◽  
Covariance Matrices ◽  
Daily Maximum ◽  
Observation Error ◽  
Chemistry Transport Model ◽  
Error Covariance ◽  
Seasonal And Diurnal Variation ◽  
Free Running

Abstract. This paper describes assimilation of trace gas observations into the chemistry transport model SILAM using the 3D-Var method. Assimilation results for year 2012 are presented for the prominent photochemical pollutants ozone (O3) and nitrogen dioxide (NO2). Both species are covered by the Airbase observation database, which provides the observational dataset used in this study. Attention is paid to the background and observation error covariance matrices, which are obtained primarily by iterative application of a posteriori diagnostics. The diagnostics are computed separately for two months representing summer and winter conditions, and further disaggregated by time of day. This allows deriving background and observation error covariance definitions which include both seasonal and diurnal variation. The consistency of the obtained covariance matrices is verified using χ2 diagnostics. The analysis scores are computed for a control set of observation stations withheld from assimilation. Compared to a free-running model simulation, the correlation coefficient for daily maximum values is improved from 0.8 to 0.9 for O3 and from 0.53 to 0.63 for NO2.

scholarly journals Estimating Observation Error Statistics Using a Robust Filter Method for Data Assimilation

2020 ◽  
Author(s):  
Yue Wang ◽  
Yulong Bai
Keyword(s):  
Data Assimilation ◽  
Error Estimation ◽  
Filter Method ◽  
Robust Filtering ◽  
Observation Error ◽  
Error Statistics ◽  
H Infinity ◽  
Error Covariance ◽  
Lorenz 96 ◽  
Data Assimilation System

For the data assimilation algorithms, the observation error covariance plays an important role, because they control the weight that is given to the model forecast and to the observation in the solution, i.e., the analysis. In order to easily calculate, we often assume observation to be a diagonal matrix, however, the observation errors are correlated to the state and have a certain dependence on time, such as certain observing types which are remotely sensed. In this work, we obtain the time-dependent and correlated observation error by the method of observation error estimation in the data assimilation system. We combine the ensemble time-local H-infinity filter (EnTLHF) with an estimate of observation error covariance matrix, named ensemble time-local H-infinity filter with observation error covariance estimation (EnTLHF-R). In the experiment, a classical nonlinear Lorenz-96 model to evaluate the performance of new method is used. The results show that the robust filtering with observation error estimation is more accurate, more robust, and the filtering is more stable.

scholarly journals Observation error covariance specification in dynamical systems for data assimilation using recurrent neural networks

2021 ◽  
Author(s):  
Sibo Cheng ◽  
Mingming Qiu
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Dynamical Systems ◽  
Data Assimilation ◽  
Recurrent Neural Networks ◽  
Time Series Data ◽  
Series Data ◽  
Observation Data ◽  
Observation Error ◽  
Error Covariance

AbstractData assimilation techniques are widely used to predict complex dynamical systems with uncertainties, based on time-series observation data. Error covariance matrices modeling is an important element in data assimilation algorithms which can considerably impact the forecasting accuracy. The estimation of these covariances, which usually relies on empirical assumptions and physical constraints, is often imprecise and computationally expensive, especially for systems of large dimensions. In this work, we propose a data-driven approach based on long short term memory (LSTM) recurrent neural networks (RNN) to improve both the accuracy and the efficiency of observation covariance specification in data assimilation for dynamical systems. Learning the covariance matrix from observed/simulated time-series data, the proposed approach does not require any knowledge or assumption about prior error distribution, unlike classical posterior tuning methods. We have compared the novel approach with two state-of-the-art covariance tuning algorithms, namely DI01 and D05, first in a Lorenz dynamical system and then in a 2D shallow water twin experiments framework with different covariance parameterization using ensemble assimilation. This novel method shows significant advantages in observation covariance specification, assimilation accuracy, and computational efficiency.

Sign in / Sign up

Export Citation Format

Share Document