scholarly journals Estimating Observation Error Statistics Using a Robust Filter Method for Data Assimilation

2020 ◽  
Author(s):  
Yue Wang ◽  
Yulong Bai
Keyword(s):  
Data Assimilation ◽  
Error Estimation ◽  
Filter Method ◽  
Robust Filtering ◽  
Observation Error ◽  
Error Statistics ◽  
H Infinity ◽  
Error Covariance ◽  
Lorenz 96 ◽  
Data Assimilation System

For the data assimilation algorithms, the observation error covariance plays an important role, because they control the weight that is given to the model forecast and to the observation in the solution, i.e., the analysis. In order to easily calculate, we often assume observation to be a diagonal matrix, however, the observation errors are correlated to the state and have a certain dependence on time, such as certain observing types which are remotely sensed. In this work, we obtain the time-dependent and correlated observation error by the method of observation error estimation in the data assimilation system. We combine the ensemble time-local H-infinity filter (EnTLHF) with an estimate of observation error covariance matrix, named ensemble time-local H-infinity filter with observation error covariance estimation (EnTLHF-R). In the experiment, a classical nonlinear Lorenz-96 model to evaluate the performance of new method is used. The results show that the robust filtering with observation error estimation is more accurate, more robust, and the filtering is more stable.

scholarly journals Multiscale Representation of Observation Error Statistics in Data Assimilation

Sensors ◽  
2020 ◽  
Vol 20 (5) ◽  
pp. 1460
Author(s):  
Vincent Chabot ◽  
Maëlle Nodet ◽  
Arthur Vidard
Keyword(s):  
Data Assimilation ◽  
Real Life ◽  
Covariance Matrices ◽  
Multiscale Modelling ◽  
Observation Error ◽  
Error Statistics ◽  
Life Data ◽  
Promising Tool ◽  
Error Covariance ◽  
Real Life Data

Accounting for realistic observation errors is a known bottleneck in data assimilation, because dealing with error correlations is complex. Following a previous study on this subject, we propose to use multiscale modelling, more precisely wavelet transform, to address this question. This study aims to investigate the problem further by addressing two issues arising in real-life data assimilation: how to deal with partially missing data (e.g., concealed by an obstacle between the sensor and the observed system), and how to solve convergence issues associated with complex observation error covariance matrices? Two adjustments relying on wavelets modelling are proposed to deal with those, and offer significant improvements. The first one consists of adjusting the variance coefficients in the frequency domain to account for masked information. The second one consists of a gradual assimilation of frequencies. Both of these fully rely on the multiscale properties associated with wavelet covariance modelling. Numerical results on twin experiments show that multiscale modelling is a promising tool to account for correlations in observation errors in realistic applications.

scholarly journals Improving Variational Data Assimilation through Background and Observation Error Adjustments

2018 ◽  
Vol 146 (2) ◽  
pp. 485-501 ◽  
Author(s):  
Jann Paul Mattern ◽  
Christopher A. Edwards ◽  
Andrew M. Moore
Keyword(s):  
Data Assimilation ◽  
Three Dimensional ◽  
Ocean Model ◽  
Variational Data Assimilation ◽  
Observation Error ◽  
Observation Window ◽  
Error Covariance ◽  
Short Term Forecasting ◽  
Data Assimilation System ◽  
Assimilation System

Abstract A procedure to objectively adjust the error covariance matrices of a variational data assimilation system is presented. It is based on popular diagnostics that utilize differences between observations and prior and posterior model solutions at the observation locations. In the application to a data assimilation system that combines a three-dimensional, physical–biogeochemical ocean model with large datasets of physical and chlorophyll a observations, the tuning procedure leads to a decrease in the posterior model-observation misfit and small improvements in short-term forecasting skill. It also increases the consistency of the data assimilation system with respect to diagnostics, based on linear estimation theory, and reduces signs of overfitting. The tuning procedure is easy to implement and only relies on information that is either prescribed to the data assimilation system or can be obtained from a series of short data assimilation experiments. The implementation includes a lognormal representation for biogeochemical variables and associated modifications to the diagnostics. Furthermore, the effect of the length of the observation window (number and distribution of observations) used to compute the diagnostics and the effect of neglecting model dynamics in the tuning procedure are examined.

scholarly journals Accounting for Correlated Observation Error in a Dual-Formulation 4D Variational Data Assimilation System

2017 ◽  
Vol 145 (3) ◽  
pp. 1019-1032 ◽  
Author(s):  
William F. Campbell ◽  
Elizabeth A. Satterfield ◽  
Benjamin Ruston ◽  
Nancy L. Baker
Keyword(s):  
Data Assimilation ◽  
Positive Impact ◽  
Error Variance ◽  
Variational Data Assimilation ◽  
Advanced Technology ◽  
Radiosonde Data ◽  
Observation Error ◽  
Error Covariance ◽  
Data Assimilation System ◽  
Assimilation System

Appropriate specification of the error statistics for both observational data and short-term forecasts is necessary to produce an optimal analysis. Observation error stems from instrument error, forward model error, and error of representation. All sources of observation error, particularly error of representation, can lead to nonzero correlations. While correlated forecast error has been accounted for since the early days of atmospheric data assimilation, observation error has typically been treated as uncorrelated until relatively recently. Thinning, averaging, and/or inflation of the assigned observation error variance have been employed to compensate for unaccounted error correlations, especially for high-resolution satellite data. In this study, the benefits of accounting for nonzero vertical (interchannel) correlation for both the Advanced Technology Microwave Satellite (ATMS) and Infrared Atmospheric Sounding Interferometer (IASI) in the NRL Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) are assessed. The vertical observation error covariance matrix for the ATMS and IASI instruments was estimated using the Desroziers method. The results suggest lowering the assigned error variance and introducing strong correlations, especially in the moisture-sensitive channels. Strong positive impact on forecast skill (verified against both the ECMWF analyses and high-quality radiosonde data) is shown in both the ATMS and IASI instruments. Additionally, the convergence of the iterative solver in NAVDAS-AR can be improved by small modifications to the observation error covariance matrices, resulting in further reduction in RMS error.

scholarly journals Simulation of Monsoon Depressions Using WRF-VAR: Impact of Different Background Error Statistics and Lateral Boundary Conditions

2014 ◽  
Vol 142 (10) ◽  
pp. 3586-3613 ◽  
Author(s):  
A. Routray ◽  
S. C. Kar ◽  
P. Mali ◽  
K. Sowjanya
Keyword(s):  
Data Assimilation ◽  
Boundary Conditions ◽  
Initial Conditions ◽  
Variational Data Assimilation ◽  
Error Statistics ◽  
Background Error ◽  
Single Observation ◽  
Monsoon Depressions ◽  
Data Assimilation System ◽  
Assimilation System

Abstract In a variational data assimilation system, background error statistics (BES) spread the influence of the observations in space and filter analysis increments through dynamic balance or statistical relationships. In a data-sparse region such as the Bay of Bengal, BES play an important role in defining the location and structure of monsoon depressions (MDs). In this study, the Indian-region-specific BES have been computed for the Weather Research and Forecasting (WRF) three-dimensional variational data assimilation system. A comparative study using single observation tests is carried out using the computed BES and global BES within the WRF system. Both sets of BES are used in the assimilation cycles and forecast runs for simulating the meteorological features associated with the MDs. Numerical experiments have been conducted to assess the relative impact of various BES in the analysis and simulations of the MDs. The results show that use of regional BES in the assimilation cycle has a positive impact on the prediction of the location, propagation, and development of rainbands associated with the MDs. The track errors of MDs are smaller when domain-specific BES are used in the assimilation cycle. Additional experiments have been conducted using data from the Interim European Centre for Medium-Range Weather Forecasts Re-Analysis (ERA-Interim) as initial and boundary conditions (IBCs) in the assimilation cycle. The results indicate that the use of domain-dependent BES and high-resolution ERA-I data as IBCs further improved the initial conditions for the model leading to better forecasts of the MDs.

scholarly journals EnKF and 4D-Var data assimilation with chemical transport model BASCOE (version 05.06)

2016 ◽  
Vol 9 (8) ◽  
pp. 2893-2908 ◽  
Author(s):  
Sergey Skachko ◽  
Richard Ménard ◽  
Quentin Errera ◽  
Yves Christophe ◽  
Simon Chabrillat
Keyword(s):  
Data Assimilation ◽  
Transport Model ◽  
Chemical Transport ◽  
Error Variance ◽  
Model Error ◽  
Chemical Processes ◽  
Observation Error ◽  
Chemical Transport Model ◽  
Error Covariance ◽  
Chemical Tracer

Abstract. We compare two optimized chemical data assimilation systems, one based on the ensemble Kalman filter (EnKF) and the other based on four-dimensional variational (4D-Var) data assimilation, using a comprehensive stratospheric chemistry transport model (CTM). This work is an extension of the Belgian Assimilation System for Chemical ObsErvations (BASCOE), initially designed to work with a 4D-Var data assimilation. A strict comparison of both methods in the case of chemical tracer transport was done in a previous study and indicated that both methods provide essentially similar results. In the present work, we assimilate observations of ozone, HCl, HNO3, H2O and N2O from EOS Aura-MLS data into the BASCOE CTM with a full description of stratospheric chemistry. Two new issues related to the use of the full chemistry model with EnKF are taken into account. One issue is a large number of error variance parameters that need to be optimized. We estimate an observation error variance parameter as a function of pressure level for each observed species using the Desroziers method. For comparison purposes, we apply the same estimate procedure in the 4D-Var data assimilation, where both scale factors of the background and observation error covariance matrices are estimated using the Desroziers method. However, in EnKF the background error covariance is modelled using the full chemistry model and a model error term which is tuned using an adjustable parameter. We found that it is adequate to have the same value of this parameter based on the chemical tracer formulation that is applied for all observed species. This is an indication that the main source of model error in chemical transport model is due to the transport. The second issue in EnKF with comprehensive atmospheric chemistry models is the noise in the cross-covariance between species that occurs when species are weakly chemically related at the same location. These errors need to be filtered out in addition to a localization based on distance. The performance of two data assimilation methods was assessed through an 8-month long assimilation of limb sounding observations from EOS Aura MLS. This paper discusses the differences in results and their relation to stratospheric chemical processes. Generally speaking, EnKF and 4D-Var provide results of comparable quality but differ substantially in the presence of model error or observation biases. If the erroneous chemical modelling is associated with moderately fast chemical processes, but whose lifetimes are longer than the model time step, then EnKF performs better, while 4D-Var develops spurious increments in the chemically related species. If, however, the observation biases are significant, then 4D-Var is more robust and is able to reject erroneous observations while EnKF does not.

scholarly journals Improvements in Forecasting Intense Rainfall: Results from the FRANC (Forecasting Rainfall Exploiting New Data Assimilation Techniques and Novel Observations of Convection) Project

Atmosphere ◽  
2019 ◽  
Vol 10 (3) ◽  
pp. 125 ◽  
Author(s):  
Sarah Dance ◽  
Susan Ballard ◽  
Ross Bannister ◽  
Peter Clark ◽  
Hannah Cloke ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Doppler Radar ◽  
Weather Prediction ◽  
Observation Error ◽  
Error Statistics ◽  
Convective Rainfall ◽  
Background Error ◽  
Spatially Correlated ◽  
Operational Forecasts ◽  
The Impact

The FRANC project (Forecasting Rainfall exploiting new data Assimilation techniques and Novel observations of Convection) has researched improvements in numerical weather prediction of convective rainfall via the reduction of initial condition uncertainty. This article provides an overview of the project’s achievements. We highlight new radar techniques: correcting for attenuation of the radar return; correction for beams that are over 90% blocked by trees or towers close to the radar; and direct assimilation of radar reflectivity and refractivity. We discuss the treatment of uncertainty in data assimilation: new methods for estimation of observation uncertainties with novel applications to Doppler radar winds, Atmospheric Motion Vectors, and satellite radiances; a new algorithm for implementation of spatially-correlated observation error statistics in operational data assimilation; and innovative treatment of moist processes in the background error covariance model. We present results indicating a link between the spatial predictability of convection and convective regimes, with potential to allow improved forecast interpretation. The research was carried out as a partnership between University researchers and the Met Office (UK). We discuss the benefits of this approach and the impact of our research, which has helped to improve operational forecasts for convective rainfall events.

scholarly journals Efficient Adaptive Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to the Control of Ocean Mesoscale Signals

2010 ◽  
Vol 138 (3) ◽  
pp. 932-950 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Error Estimates ◽  
Forecast Error ◽  
Observation Error ◽  
Square Root ◽  
Error Statistics ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Optimal Estimates

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

Sign in / Sign up

Export Citation Format

Share Document