Empirical determination of the covariance of forecast errors: an empirical justification and reformulation of Hybrid covariance models

2021 ◽  
Author(s):  
Diego Saul Carrio Carrio ◽  
Craig Bishop ◽  
Shunji Kotsuki
Keyword(s):  
Covariance Matrix ◽  
Error Variance ◽  
Separation Distance ◽  
Covariance Model ◽  
Ensemble Size ◽  
True Error ◽  
Error Covariance ◽  
Covariance Models ◽  
Small Ensemble ◽  
Increasing Function

<p>The replacement of climatological background error covariance models with Hybrid error covariance models that linearly combine a localized ensemble covariance matrix and a climatological error covariance matrix has led to significant forecast improvements at several forecasting centres. To deepen understanding of why the Hybrid’s superficially ad-hoc mix of ensemble based covariances and climatological covariances yielded such significant improvements, we derive the linear state estimation equations that minimize analysis error variance given an imperfect ensemble covariance. For high dimensional models, the computational cost of the very large sample sizes required to empirically estimate the terms in these equations is prohibitive. However, a reasonable and computationally feasible approximation to these equations can be obtained from empirical estimates of the true error covariance between two model variables given an imperfect ensemble covariance between the same two variables.   Here, using a Data Assimilation (DA) system featuring a simplified Global Circulation Model (SPEEDY), pseudo-observations of known error variance and an ensemble data assimilation scheme (LETKF),  we quantitatively demonstrate that the traditional Hybrid used by many operational centres is a much better approximation to the true covariance given the ensemble covariance than either the static climatological covariance or the localized ensemble covariance. These quantitative findings help explain why operational centres have found such large forecast improvements when switching from a static error covariance model to a Hybrid forecast error covariance model. Another fascinating finding of our empirical study is that the form of current Hybrid error covariance models is fundamentally incorrect in that the weight given to the static covariance matrix is independent of the separation distance of model variables. Our results show that this weight should be an increasing function of variable separation distance.  It is found that for ensemble covariances significantly different to zero, the true error covariance of spatially separated variables is an approximately linear function of the corresponding ensemble covariance, However, for small ensemble sizes and ensemble covariances near zero, the true covariance is an increasing function of the magnitude of the ensemble covariance and reaches a local minimum at the precise point where the ensemble covariance is equal to zero. It is hypothesized that this behaviour is a consequence of small ensemble size and, specifically, associated spurious fluctuations of the ensemble covariances and variances. Consistent with this hypothesis, this local minimum is almost eliminated by quadrupling the ensemble size.</p>

scholarly journals Observation-Informed Generalized Hybrid Error Covariance Models

2018 ◽  
Vol 146 (11) ◽  
pp. 3605-3622 ◽  
Author(s):  
Elizabeth A. Satterfield ◽  
Daniel Hodyss ◽  
David D. Kuhl ◽  
Craig H. Bishop
Keyword(s):  
Data Assimilation ◽  
Covariance Matrix ◽  
Sample Variance ◽  
Calibration Function ◽  
Covariance Model ◽  
True Error ◽  
Ensemble Data Assimilation ◽  
Error Covariance ◽  
Ensemble Data ◽  
Prior Variance

Abstract Because of imperfections in ensemble data assimilation schemes, one cannot assume that the ensemble-derived covariance matrix is equal to the true error covariance matrix. Here, we describe a simple and intuitively compelling method to fit calibration functions of the ensemble sample variance to the mean of the distribution of true error variances, given an ensemble estimate. We demonstrate that the use of such calibration functions is consistent with theory showing that, when sampling error in the prior variance estimate is considered, the gain that minimizes the posterior error variance uses the expected true prior variance, given an ensemble sample variance. Once the calibration function has been fitted, it can be combined with ensemble-based and climatologically based error correlation information to obtain a generalized hybrid error covariance model. When the calibration function is chosen to be a linear function of the ensemble variance, the generalized hybrid error covariance model is the widely used linear hybrid consisting of a weighted sum of a climatological and an ensemble-based forecast error covariance matrix. However, when the calibration function is chosen to be, say, a cubic function of the ensemble sample variance, the generalized hybrid error covariance model is a nonlinear function of the ensemble estimate. We consider idealized univariate data assimilation and multivariate cycling ensemble data assimilation to demonstrate that the generalized hybrid error covariance model closely approximates the optimal weights found through computationally expensive tuning in the linear case and, in the nonlinear case, outperforms any plausible linear model.

scholarly journals Parametric covariance dynamics for the nonlinear diffusive Burgers equation

2018 ◽  
Vol 25 (3) ◽  
pp. 481-495 ◽  
Author(s):  
Olivier Pannekoucke ◽  
Marc Bocquet ◽  
Richard Ménard
Keyword(s):  
Kalman Filter ◽  
Burgers Equation ◽  
Diffusion Tensor ◽  
Error Variance ◽  
Covariance Model ◽  
Computationally Efficient ◽  
Closure Model ◽  
Error Covariance ◽  
Efficient Alternative ◽  
Linear Covariance

Abstract. The parametric Kalman filter (PKF) is a computationally efficient alternative method to the ensemble Kalman filter. The PKF relies on an approximation of the error covariance matrix by a covariance model with a space–time evolving set of parameters. This study extends the PKF to nonlinear dynamics using the diffusive Burgers equation as an application, focusing on the forecast step of the assimilation cycle. The covariance model considered is based on the diffusion equation, with the diffusion tensor and the error variance as evolving parameters. An analytical derivation of the parameter dynamics highlights a closure issue. Therefore, a closure model is proposed based on the kurtosis of the local correlation functions. Numerical experiments compare the PKF forecast with the statistics obtained from a large ensemble of nonlinear forecasts. These experiments strengthen the closure model and demonstrate the ability of the PKF to reproduce the tangent linear covariance dynamics, at a low numerical cost.

scholarly journals Parametric covariance dynamics for the nonlinear diffusive Burgers' equation

2018 ◽  
Author(s):  
Olivier Pannekoucke ◽  
Marc Bocquet ◽  
Richard Ménard
Keyword(s):  
Kalman Filter ◽  
Burgers Equation ◽  
Diffusion Tensor ◽  
Error Variance ◽  
Covariance Model ◽  
Computationally Efficient ◽  
Closure Model ◽  
Error Covariance ◽  
Efficient Alternative ◽  
Linear Covariance

Abstract. The parametric Kalman filter (PKF) is a computationally efficient alternative method to the ensemble Kalman filter (EnKF). The PKF relies on an approximation of the error covariance matrix by a covariance model with space-time evolving set of parameters. This study extends the PKF to nonlinear dynamics using the diffusive Burgers' equation as an application, focusing on the forecast step of the assimilation cycle. The covariance model considered is based on the diffusion equation, with the diffusion tensor and the error variance as evolving parameter. An analytical derivation of the parameter dynamics highlights a closure issue. Therefore, a closure model is proposed based on the so-called kurtosis of the local correlation functions. Numerical experiments compare the PKF forecast with the statistics obtained from an large ensemble of nonlinear forecasts. These experiments strengthen the closure model and demonstrate the ability of the PKF to reproduce the tangent-linear covariance dynamics, at a low numerical cost.

scholarly journals Facilitating Inversion of the Error Covariance Models for the Wide-Swath Altimeters

2020 ◽  
Vol 12 (11) ◽  
pp. 1823
Author(s):  
Max Yaremchuk ◽  
Joseph M. D’Addezio ◽  
Gregg Jacobs
Keyword(s):  
Covariance Matrix ◽  
Cost Savings ◽  
Strongly Correlated ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Covariance Models ◽  
Ocean Topography ◽  
Cent Error ◽  
Along Track ◽  
Wide Swath

Wide-swath satellite altimeter observations are contaminated by errors caused by the uncertainties in the geometry and orientation of the on-board interferometer. These errors are strongly correlated across the track, while also having similar error structures in the along-track direction. We describe a method for modifying the geometric component of the error covariance matrix which facilitates accuracy in the removal of the respective error modes from the signal and improves computational efficiency of the data assimilation schemes involving wide-swath altimeter observations. The method has been tested using the Surface Water and Ocean Topography simulator. We show substantial computer cost savings in the pseudo-inversion of the respective error covariance matrix. This efficiency improvement comes with a few per cent error in the approximation of the original covariance model simulating uncertainties in the geometry and orientation of the on-board interferometer.

scholarly journals The High-Rank Ensemble Transform Kalman Filter

2019 ◽  
Vol 147 (8) ◽  
pp. 3025-3043 ◽  
Author(s):  
Bo Huang ◽  
Xuguang Wang ◽  
Craig H. Bishop
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
High Rank ◽  
Background Error ◽  
Localization Method ◽  
Error Covariance ◽  
Higher Rank ◽  
Small Ensemble ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract The ensemble Kalman filter is typically implemented either by applying the localization on the background error covariance matrix (B-localization) or by inflating the observation error variances (R-localization). A mathematical demonstration suggests that for the same effective localization function, the background error covariance matrix from the B-localization method shows a higher rank than the R-localization method. The B-localization method is realized in the ensemble transform Kalman filter (ETKF) by extending the background ensemble perturbations through modulation (MP-localization). Specifically, the modulation functions are constructed from the leading eigenvalues and eigenvectors of the original B-localization matrix. Because of its higher rank than the classic R-localized ETKF, the B-/MP-localized ETKF is termed as the high-rank ETKF (HETKF). The performances of the HETKF and R-localized ETKF were compared through cycled data assimilation experiments using the Lorenz model II. The results show that the HETKF outperforms the R-localized ETKF especially for a small ensemble. The improved analysis in the HETKF is likely associated with the higher rank from the B-/MP-localization method, since its higher rank is expected to contribute more positively to alleviating the rank deficiency issue and thus improve the analysis for a small ensemble. The HETKF is less sensitive to the localization length scales and inflation factors. Furthermore, the experiments suggest that the above conclusion comparing the HETKF and R-localized ETKF does not depend on how the analyzed ensemble perturbations are subselected in the HETKF.

scholarly journals Hidden Error Variance Theory. Part I: Exposition and Analytic Model

2013 ◽  
Vol 141 (5) ◽  
pp. 1454-1468 ◽  
Author(s):  
Craig H. Bishop ◽  
Elizabeth A. Satterfield
Keyword(s):  
Posterior Distribution ◽  
Forecast Error ◽  
Error Variance ◽  
Empirical Estimation ◽  
Empirical Observation ◽  
Inverse Gamma ◽  
Error Covariance ◽  
Error Distributions ◽  
Forecast Error Variance ◽  
Covariance Models

Abstract A conundrum of predictability research is that while the prediction of flow-dependent error distributions is one of its main foci, chaos fundamentally hides flow-dependent forecast error distributions from empirical observation. Empirical estimation of such error distributions requires a large sample of error realizations given the same flow-dependent conditions. However, chaotic elements of the flow and the observing network make it impossible to collect a large enough conditioned error sample to empirically define such distributions and their variance. Such conditional variances are “hidden.” Here, an exposition of the problem is developed from an ensemble Kalman filter data assimilation system applied to a 10-variable nonlinear chaotic model and 25 000 replicate models. The 25 000 replicates reveal the error variances that would otherwise be hidden. It is found that the inverse-gamma distribution accurately approximates the posterior distribution of conditional error variances given an imperfect ensemble variance and provides a reasonable approximation to the prior climatological distribution of conditional error variances. A new analytical model shows how the properties of a likelihood distribution of ensemble variances given a true conditional error variance determine the posterior distribution of error variances given an ensemble variance. The analytically generated distributions are shown to satisfactorily fit empirically determined distributions. The theoretical analysis yields a rigorous interpretation and justification of hybrid error variance models that linearly combine static and flow-dependent estimates of forecast error variance; in doing so, it also helps justify and inform hybrid error covariance models.

Effects of Accounting-Method Choices on Subjective Performance-Measure Weighting Decisions: Experimental Evidence on Precision and Error Covariance

2005 ◽  
Vol 80 (4) ◽  
pp. 1163-1192 ◽  
Author(s):  
Ranjani Krishnan ◽  
Joan L. Luft ◽  
Michael D. Shields
Keyword(s):  
Performance Measures ◽  
Error Variance ◽  
Spillover Effects ◽  
Performance Measure ◽  
Incentive System ◽  
Experimental Setting ◽  
Error Covariance ◽  
Psychology Theory ◽  
Optimal Weighting ◽  
Similarity And Difference

Performance-measure weights for incentive compensation are often determined subjectively. Determining these weights is a cognitively difficult task, and archival research shows that observed performance-measure weights are only partially consistent with the predictions of agency theory. Ittner et al. (2003) have concluded that psychology theory can help to explain such inconsistencies. In an experimental setting based on Feltham and Xie (1994), we use psychology theories of reasoning to predict distinctive patterns of similarity and difference between optimal and actual subjective performance-measure weights. The following predictions are supported. First, in contrast to a number of prior studies, most individuals' decisions are significantly influenced by the performance measures' error variance (precision) and error covariance. Second, directional errors in the use of these measurement attributes are relatively frequent, resulting in a mean underreaction to an accounting change that alters performance measurement error. Third, individuals seem insufficiently aware that a change in the accounting for one measure has spillover effects on the optimal weighting of the other measure in a two-measure incentive system. In consequence, they make performance-measure weighting decisions that are likely to result in misallocations of agent effort.

Enhancing the impact of IASI observations through an updated observation-error covariance matrix

2016 ◽  
Vol 142 (697) ◽  
pp. 1767-1780 ◽  
Author(s):  
Niels Bormann ◽  
Massimo Bonavita ◽  
Rossana Dragani ◽  
Reima Eresmaa ◽  
Marco Matricardi ◽  
...  
Keyword(s):  
Covariance Matrix ◽  
Observation Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
The Impact

Impact of AMSU-A Radiances in a Column Ensemble Kalman Filter

2018 ◽  
Vol 146 (12) ◽  
pp. 3949-3976 ◽  
Author(s):  
Herschel L. Mitchell ◽  
P. L. Houtekamer ◽  
Sylvain Heilliette
Keyword(s):  
Covariance Matrix ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Microwave Sounding Unit ◽  
The Matrix ◽  
Microwave Sounding ◽  
Background Temperature ◽  
The Difference

Abstract A column EnKF, based on the Canadian global EnKF and using the RTTOV radiative transfer (RT) model, is employed to investigate issues relating to the EnKF assimilation of Advanced Microwave Sounding Unit-A (AMSU-A) radiance measurements. Experiments are performed with large and small ensembles, with and without localization. Three different descriptions of background temperature error are considered: 1) using analytical vertical modes and hypothetical spectra, 2) using the vertical modes and spectrum of a covariance matrix obtained from the global EnKF after 2 weeks of cycling, and 3) using the vertical modes and spectrum of the static background error covariance matrix employed to initiate a global data assimilation cycle. It is found that the EnKF performs well in some of the experiments with background error description 1, and yields modest error reductions with background error description 3. However, the EnKF is virtually unable to reduce the background error (even when using a large ensemble) with background error description 2. To analyze these results, the different background error descriptions are viewed through the prism of the RT model by comparing the trace of the matrix , where is the RT model and is the background error covariance matrix. Indeed, this comparison is found to explain the difference in the results obtained, which relates to the degree to which deep modes are, or are not, present in the different background error covariances. The results suggest that, after 2 weeks of cycling, the global EnKF has virtually eliminated all background error structures that can be “seen” by the AMSU-A radiances.

Sign in / Sign up

Export Citation Format

Share Document