Vertical Covariance Localization for Satellite Radiances in Ensemble Kalman Filters

2010 ◽  
Vol 138 (1) ◽  
pp. 282-290 ◽  
Author(s):  
William F. Campbell ◽  
Craig H. Bishop ◽  
Daniel Hodyss
Keyword(s):  
Kalman Filters ◽  
Forecast Error ◽  
Model Space ◽  
Error Variance ◽  
Observation Error ◽  
Ensemble Kalman Filters ◽  
Weighting Functions ◽  
Error Covariance ◽  
Space Localization ◽  
Satellite Channels

Abstract A widely used observation space covariance localization method is shown to adversely affect satellite radiance assimilation in ensemble Kalman filters (EnKFs) when compared to model space covariance localization. The two principal problems are that distance and location are not well defined for integrated measurements, and that neighboring satellite channels typically have broad, overlapping weighting functions, which produce true, nonzero correlations that localization in radiance space can incorrectly eliminate. The limitations of the method are illustrated in a 1D conceptual model, consisting of three vertical levels and a two-channel satellite instrument. A more realistic 1D model is subsequently tested, using the 30 vertical levels from the Navy Operational Global Atmospheric Prediction System (NOGAPS), the Advanced Microwave Sounding Unit A (AMSU-A) weighting functions for channels 6–11, and the observation error variance and forecast error covariance from the NRL Atmospheric Variational Data Assimilation System (NAVDAS). Analyses from EnKFs using radiance space localization are compared with analyses from raw EnKFs, EnKFs using model space localization, and the optimal analyses using the NAVDAS forecast error covariance as a proxy for the true forecast error covariance. As measured by mean analysis error variance reduction, radiance space localization is inferior to model space localization for every ensemble size and meaningful observation error variance tested. Furthermore, given as many satellite channels as vertical levels, radiance space localization cannot recover the true temperature state with perfect observations, whereas model space localization can.

scholarly journals Model Space Localization Is Not Always Better Than Observation Space Localization for Assimilation of Satellite Radiances

2015 ◽  
Vol 143 (10) ◽  
pp. 3948-3955 ◽  
Author(s):  
Lili Lei ◽  
Jeffrey S. Whitaker
Keyword(s):  
Localization Length ◽  
Model Space ◽  
Error Variance ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance ◽  
Space Localization ◽  
1D Model ◽  
Observation Space ◽  
Vertical Localization

Abstract Covariance localization is an essential component of ensemble-based data assimilation systems for large geophysical applications with limited ensemble sizes. For integral observations like the satellite radiances, where the concepts of location or vertical distance are not well defined, vertical localization in observation space is not as straightforward as in model space. The detailed differences between model space and observation space localizations are examined using a real radiance observation. Counterintuitive analysis increments can be obtained with model space localization; the magnitude of the increment can increase and the increment can change sign when the localization scale decreases. This occurs when there are negative background-error covariances and a predominately positive forward operator. Too narrow model space localization can neglect the negative background-error covariances and result in the counterintuitive analysis increments. An idealized 1D model with integral observations and known true error covariance is then used to compare errors resulting from model space and observation space localizations. Although previous studies have suggested that observation space localization is inferior to model space localization for satellite radiances, the results from the 1D model reveal that observation space localization can have advantages over model space localization when there are negative background-error covariances. Differences between model space and observation space localizations disappear as ensemble size, observation error variance, and localization scale increase. Thus, large ensemble sizes and vertical localization length scales may be needed to more effectively assimilate radiance observations.

scholarly journals Efficient Parameterization of the Observation Error Covariance Matrix for Square Root or Ensemble Kalman Filters: Application to Ocean Altimetry

2009 ◽  
Vol 137 (6) ◽  
pp. 1908-1927 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Clément Ubelmann ◽  
Charles-Emmanuel Testut ◽  
Emmanuel Cosme ◽  
Pierre Brasseur ◽  
...  
Keyword(s):  
Covariance Matrix ◽  
Kalman Filters ◽  
Signal Amplitude ◽  
Observation Error ◽  
Square Root ◽  
Ensemble Kalman Filters ◽  
Error Covariance Matrix ◽  
Standard Algorithm ◽  
Error Covariance ◽  
North Brazil

Abstract In the Kalman filter standard algorithm, the computational complexity of the observational update is proportional to the cube of the number y of observations (leading behavior for large y). In realistic atmospheric or oceanic applications, involving an increasing quantity of available observations, this often leads to a prohibitive cost and to the necessity of simplifying the problem by aggregating or dropping observations. If the filter error covariance matrices are in square root form, as in square root or ensemble Kalman filters, the standard algorithm can be transformed to be linear in y, providing that the observation error covariance matrix is diagonal. This is a significant drawback of this transformed algorithm and often leads to an assumption of uncorrelated observation errors for the sake of numerical efficiency. In this paper, it is shown that the linearity of the transformed algorithm in y can be preserved for other forms of the observation error covariance matrix. In particular, quite general correlation structures (with analytic asymptotic expressions) can be simulated simply by augmenting the observation vector with differences of the original observations, such as their discrete gradients. Errors in ocean altimetric observations are spatially correlated, as for instance orbit or atmospheric errors along the satellite track. Adequately parameterizing these correlations can directly improve the quality of observational updates and the accuracy of the associated error estimates. In this paper, the example of the North Brazil Current circulation is used to demonstrate the importance of this effect, which is especially significant in that region of moderate ratio between signal amplitude and observation noise, and to show that the efficient parameterization that is proposed for the observation error correlations is appropriate to take it into account. Adding explicit gradient observations also receives a physical justification. This parameterization is thus proved to be useful to ocean data assimilation systems that are based on square root or ensemble Kalman filters, as soon as the number of observations becomes penalizing, and if a sophisticated parameterization of the observation error correlations is required.

scholarly journals EnKF and 4D-Var data assimilation with chemical transport model BASCOE (version 05.06)

2016 ◽  
Vol 9 (8) ◽  
pp. 2893-2908 ◽  
Author(s):  
Sergey Skachko ◽  
Richard Ménard ◽  
Quentin Errera ◽  
Yves Christophe ◽  
Simon Chabrillat
Keyword(s):  
Data Assimilation ◽  
Transport Model ◽  
Chemical Transport ◽  
Error Variance ◽  
Model Error ◽  
Chemical Processes ◽  
Observation Error ◽  
Chemical Transport Model ◽  
Error Covariance ◽  
Chemical Tracer

Abstract. We compare two optimized chemical data assimilation systems, one based on the ensemble Kalman filter (EnKF) and the other based on four-dimensional variational (4D-Var) data assimilation, using a comprehensive stratospheric chemistry transport model (CTM). This work is an extension of the Belgian Assimilation System for Chemical ObsErvations (BASCOE), initially designed to work with a 4D-Var data assimilation. A strict comparison of both methods in the case of chemical tracer transport was done in a previous study and indicated that both methods provide essentially similar results. In the present work, we assimilate observations of ozone, HCl, HNO3, H2O and N2O from EOS Aura-MLS data into the BASCOE CTM with a full description of stratospheric chemistry. Two new issues related to the use of the full chemistry model with EnKF are taken into account. One issue is a large number of error variance parameters that need to be optimized. We estimate an observation error variance parameter as a function of pressure level for each observed species using the Desroziers method. For comparison purposes, we apply the same estimate procedure in the 4D-Var data assimilation, where both scale factors of the background and observation error covariance matrices are estimated using the Desroziers method. However, in EnKF the background error covariance is modelled using the full chemistry model and a model error term which is tuned using an adjustable parameter. We found that it is adequate to have the same value of this parameter based on the chemical tracer formulation that is applied for all observed species. This is an indication that the main source of model error in chemical transport model is due to the transport. The second issue in EnKF with comprehensive atmospheric chemistry models is the noise in the cross-covariance between species that occurs when species are weakly chemically related at the same location. These errors need to be filtered out in addition to a localization based on distance. The performance of two data assimilation methods was assessed through an 8-month long assimilation of limb sounding observations from EOS Aura MLS. This paper discusses the differences in results and their relation to stratospheric chemical processes. Generally speaking, EnKF and 4D-Var provide results of comparable quality but differ substantially in the presence of model error or observation biases. If the erroneous chemical modelling is associated with moderately fast chemical processes, but whose lifetimes are longer than the model time step, then EnKF performs better, while 4D-Var develops spurious increments in the chemically related species. If, however, the observation biases are significant, then 4D-Var is more robust and is able to reject erroneous observations while EnKF does not.

scholarly journals Efficient Adaptive Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to the Control of Ocean Mesoscale Signals

2010 ◽  
Vol 138 (3) ◽  
pp. 932-950 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Error Estimates ◽  
Forecast Error ◽  
Observation Error ◽  
Square Root ◽  
Error Statistics ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Optimal Estimates

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

scholarly journals On Serial Observation Processing in Localized Ensemble Kalman Filters

2015 ◽  
Vol 143 (5) ◽  
pp. 1554-1567 ◽  
Author(s):  
Lars Nerger
Keyword(s):  
Ensemble Kalman Filter ◽  
Kalman Filters ◽  
Large Scale ◽  
The State ◽  
Ensemble Kalman Filters ◽  
Filter Performance ◽  
Error Covariance ◽  
Analysis Process ◽  
Scale Models ◽  
Serial Observation

Abstract Ensemble square root filters can either assimilate all observations that are available at a given time at once, or assimilate the observations in batches or one at a time. For large-scale models, the filters are typically applied with a localized analysis step. This study demonstrates that the interaction of serial observation processing and localization can destabilize the analysis process, and it examines under which conditions the instability becomes significant. The instability results from a repeated inconsistent update of the state error covariance matrix that is caused by the localization. The inconsistency is present in all ensemble Kalman filters, except for the classical ensemble Kalman filter with perturbed observations. With serial observation processing, its effect is small in cases when the assimilation changes the ensemble of model states only slightly. However, when the assimilation has a strong effect on the state estimates, the interaction of localization and serial observation processing can significantly deteriorate the filter performance. In realistic large-scale applications, when the assimilation changes the states only slightly and when the distribution of the observations is irregular and changing over time, the instability is likely not significant.

scholarly journals A New Data Assimilation Scheme: The Space-Expanded Ensemble Localization Kalman Filter

2013 ◽  
Vol 2013 ◽  
pp. 1-6 ◽  
Author(s):  
Hongze Leng ◽  
Junqiang Song ◽  
Fengshun Lu ◽  
Xiaoqun Cao
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Forecast Error ◽  
Three Dimensional ◽  
Full Rank ◽  
Observation Error ◽  
Model Framework ◽  
Background Error ◽  
Error Covariance ◽  
Expanded Ensemble

This study considers a new hybrid three-dimensional variational (3D-Var) and ensemble Kalman filter (EnKF) data assimilation (DA) method in a non-perfect-model framework, named space-expanded ensemble localization Kalman filter (SELKF). In this method, the localization operation is directly applied to the ensemble anomalies with a Schur Product, rather than to the full error covariance of the state in the EnKF. Meanwhile, the correction space of analysis increment is expanded to a space with larger dimension, and the rank of the forecast error covariance is significantly increased. This scheme can reduce the spurious correlations in the covariance and approximate the full-rank background error covariance well. Furthermore, a deterministic scheme is used to generate the analysis anomalies. The results show that the SELKF outperforms the perturbed EnKF given a relatively small ensemble size, especially when the length scale is relatively long or the observation error covariance is relatively small.

scholarly journals Investigating the Use of Ensemble Variance to Predict Observation Error of Representation

2017 ◽  
Vol 145 (2) ◽  
pp. 653-667 ◽  
Author(s):  
Elizabeth Satterfield ◽  
Daniel Hodyss ◽  
David D. Kuhl ◽  
Craig H. Bishop
Keyword(s):  
Data Assimilation ◽  
Forecast Model ◽  
Error Variance ◽  
Observation Error ◽  
Coarse Resolution ◽  
Error Covariance ◽  
Flow Dependence ◽  
Representation Error ◽  
Resolution Model ◽  
Predictive Relationship

Data assimilation schemes combine observational data with a short-term model forecast to produce an analysis. However, many characteristics of the atmospheric states described by the observations and the model differ. Observations often measure a higher-resolution state than coarse-resolution model grids can describe. Hence, the observations may measure aspects of gradients or unresolved eddies that are poorly resolved by the filtered version of reality represented by the model. This inconsistency, known as observation representation error, must be accounted for in data assimilation schemes. In this paper the ability of the ensemble to predict the variance of the observation error of representation is explored, arguing that the portion of representation error being detected by the ensemble variance is that portion correlated to the smoothed features that the coarse-resolution forecast model is able to predict. This predictive relationship is explored using differences between model states and their spectrally truncated form, as well as commonly used statistical methods to estimate observation error variances. It is demonstrated that the ensemble variance is a useful predictor of the observation error variance of representation and that it could be used to account for flow dependence in the observation error covariance matrix.

scholarly journals Efficient Local Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to a Basin-Scale Ocean Turbulent Flow

2011 ◽  
Vol 139 (2) ◽  
pp. 474-493 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Turbulent Flow ◽  
Covariance Matrix ◽  
Kalman Filters ◽  
Square Root ◽  
Ensemble Kalman Filters ◽  
Error Covariance Matrix ◽  
Square Roots ◽  
Schur Product ◽  
Error Covariance ◽  
Basin Scale

Abstract In large-sized atmospheric or oceanic applications of square root or ensemble Kalman filters, it is often necessary to introduce the prior assumption that long-range correlations are negligible and force them to zero using a local parameterization, supplementing the ensemble or reduced-rank representation of the covariance. One classic algorithm to perform this operation consists of taking the Schur product of the ensemble covariance matrix by a local support correlation matrix. However, with this parameterization, the square root of the forecast error covariance matrix is no more directly available, so that any observational update algorithm requiring this square root must include an additional step to compute local square roots from the Schur product. This computation generates an additional numerical cost or produces high-rank square roots, which may deprive the observational update from its original efficiency. In this paper, it is shown how efficient local square root parameterizations can be obtained, for use with a specific square root formulation (i.e., eigenbasis algorithm) of the observational update. Comparisons with the classic algorithm are provided, mainly in terms of consistency, accuracy, and computational complexity. As an application, the resulting parameterization is used to estimate maps of dynamic topography characterizing a basin-scale ocean turbulent flow. Even with this moderate-sized system (a 2200-km-wide square basin with 100-km-wide mesoscale eddies), it is observed that more than 1000 ensemble members are necessary to faithfully represent the global correlation patterns, and that a local parameterization is needed to produce correct covariances with moderate-sized ensembles. Comparisons with the exact solution show that the use of local square roots is able to improve the accuracy of the updated ensemble mean and the consistency of the updated ensemble variance. With the eigenbasis algorithm, optimal adaptive estimates of scaling factors for the forecast and observation error covariance matrix can also be obtained locally at negligible additional numerical cost. Finally, a comparison of the overall computational cost illustrates the decisive advantage that efficient local square root parameterizations may have to deal simultaneously with a larger number of observations and avoid data thinning as much as possible.

Atmospheric River Reconnaissance Observation Impact in the Navy Global Forecast System

2020 ◽  
Vol 148 (2) ◽  
pp. 763-782 ◽  
Author(s):  
Rebecca E. Stone ◽  
Carolyn A. Reynolds ◽  
James D. Doyle ◽  
Rolf H. Langland ◽  
Nancy L. Baker ◽  
...  
Keyword(s):  
North American ◽  
Forecast Error ◽  
Error Variance ◽  
Observation Error ◽  
Atmospheric Rivers ◽  
Atmospheric River ◽  
The North ◽  
Beneficial Impact ◽  
Observation Impact ◽  
The Impact

Abstract Atmospheric rivers, often associated with impactful weather along the west coast of North America, can be a challenge to forecast even on short time scales. This is attributed, at least in part, to the scarcity of eastern Pacific in situ observations. We examine the impact of assimilating dropsonde observations collected during the Atmospheric River (AR) Reconnaissance 2018 field program on the Navy Global Environmental Model (NAVGEM) analyses and forecasts. We compare NAVGEM’s representation of the ARs to the observations, and examine whether the observation–background difference statistics are similar to the observation error variance specified in the data assimilation system. Forecast sensitivity observation impact is determined for each dropsonde variable, and compared to the impacts of the North American radiosonde network. We find that the reconnaissance soundings have significant beneficial impact, with per observation impact more than double that of the North American radiosonde network. Temperature and wind observations have larger total and per observation impact than moisture observations. In our experiment, the 24-h global forecast error reduction from the reconnaissance soundings can be comparable to the reduction from the North American radiosonde network for the field program dates that include at least two flights.

Sign in / Sign up

Export Citation Format

Share Document