scholarly journals Four-Dimensional Data Assimilation and Balanced Dynamics

2006 ◽  
Vol 63 (7) ◽  
pp. 1840-1858 ◽  
Author(s):  
Lisa J. Neef ◽  
Saroja M. Polavarapu ◽  
Theodore G. Shepherd
Keyword(s):  
Data Assimilation ◽  
Degrees Of Freedom ◽  
Forecast Error ◽  
Wave Mode ◽  
Slow Manifold ◽  
Error Statistics ◽  
Ensemble Kalman Filters ◽  
Error Covariance ◽  
Balanced Dynamics ◽  
Four Dimensional Data Assimilation

Abstract The problem of spurious excitation of gravity waves in the context of four-dimensional data assimilation is investigated using a simple model of balanced dynamics. The model admits a chaotic vortical mode coupled to a comparatively fast gravity wave mode, and can be initialized such that the model evolves on a so-called slow manifold, where the fast motion is suppressed. Identical twin assimilation experiments are performed, comparing the extended and ensemble Kalman filters (EKF and EnKF, respectively). The EKF uses a tangent linear model (TLM) to estimate the evolution of forecast error statistics in time, whereas the EnKF uses the statistics of an ensemble of nonlinear model integrations. Specifically, the case is examined where the true state is balanced, but observation errors project onto all degrees of freedom, including the fast modes. It is shown that the EKF and EnKF will assimilate observations in a balanced way only if certain assumptions hold, and that, outside of ideal cases (i.e., with very frequent observations), dynamical balance can easily be lost in the assimilation. For the EKF, the repeated adjustment of the covariances by the assimilation of observations can easily unbalance the TLM, and destroy the assumptions on which balanced assimilation rests. It is shown that an important factor is the choice of initial forecast error covariance matrix. A balance-constrained EKF is described and compared to the standard EKF, and shown to offer significant improvement for observation frequencies where balance in the standard EKF is lost. The EnKF is advantageous in that balance in the error covariances relies only on a balanced forecast ensemble, and that the analysis step is an ensemble-mean operation. Numerical experiments show that the EnKF may be preferable to the EKF in terms of balance, though its validity is limited by ensemble size. It is also found that overobserving can lead to a more unbalanced forecast ensemble and thus to an unbalanced analysis.

scholarly journals Linear versus Nonlinear Filtering with Scale-Selective Corrections for Balanced Dynamics in a Simple Atmospheric Model

2012 ◽  
Vol 69 (11) ◽  
pp. 3405-3419 ◽  
Author(s):  
Aneesh C. Subramanian ◽  
Ibrahim Hoteit ◽  
Bruce Cornuelle ◽  
Arthur J. Miller ◽  
Hajoon Song
Keyword(s):  
Nonlinear Model ◽  
Nonlinear Filtering ◽  
Atmospheric Model ◽  
Wave Mode ◽  
Slow Manifold ◽  
Bayesian Filtering ◽  
Ensemble Kalman Filters ◽  
Importance Resampling ◽  
Balanced Dynamics ◽  
Better Than

Abstract This paper investigates the role of the linear analysis step of the ensemble Kalman filters (EnKF) in disrupting the balanced dynamics in a simple atmospheric model and compares it to a fully nonlinear particle-based filter (PF). The filters have a very similar forecast step but the analysis step of the PF solves the full Bayesian filtering problem while the EnKF analysis only applies to Gaussian distributions. The EnKF is compared to two flavors of the particle filter with different sampling strategies, the sequential importance resampling filter (SIRF) and the sequential kernel resampling filter (SKRF). The model admits a chaotic vortical mode coupled to a comparatively fast gravity wave mode. It can also be configured either to evolve on a so-called slow manifold, where the fast motion is suppressed, or such that the fast-varying variables are diagnosed from the slow-varying variables as slaved modes. Identical twin experiments show that EnKF and PF capture the variables on the slow manifold well as the dynamics is very stable. PFs, especially the SKRF, capture slaved modes better than the EnKF, implying that a full Bayesian analysis estimates the nonlinear model variables better. The PFs perform significantly better in the fully coupled nonlinear model where fast and slow variables modulate each other. This suggests that the analysis step in the PFs maintains the balance in both variables much better than the EnKF. It is also shown that increasing the ensemble size generally improves the performance of the PFs but has less impact on the EnKF after a sufficient number of members have been used.

scholarly journals On the Sensitivity Equations of Four-Dimensional Variational (4D-Var) Data Assimilation

2008 ◽  
Vol 136 (8) ◽  
pp. 3050-3065 ◽  
Author(s):  
Dacian N. Daescu
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Computational Effort ◽  
Water Model ◽  
Error Statistics ◽  
Background Error ◽  
Practical Applications ◽  
Error Covariance ◽  
Sensitivity Equations ◽  
The Impact

Abstract The equations of the forecast sensitivity to observations and to the background estimate in a four-dimensional variational data assimilation system (4D-Var DAS) are derived from the first-order optimality condition in unconstrained minimization. Estimation of the impact of uncertainties in the specification of the error statistics is considered by evaluating the sensitivity to the observation and background error covariance matrices. The information provided by the error covariance sensitivity analysis is used to identify the input components for which improved estimates of the statistical properties of the errors are of most benefit to the analysis and forecast. A close relationship is established between the sensitivities within each input pair data/error covariance such that once the observation and background sensitivities are available the evaluation of the sensitivity to the specification of the corresponding error statistics requires little additional computational effort. The relevance of the 4D-Var sensitivity equations to assess the data impact in practical applications is discussed. Computational issues are addressed and idealized 4D-Var experiments are set up with a finite-volume shallow-water model to illustrate the theoretical concepts. Time-dependent observation sensitivity and potential applications to improve the model forecast are presented. Guidance provided by the sensitivity fields is used to adjust a 4D-Var DAS to achieve forecast error reduction through assimilation of supplementary data and through an accurate specification of a few of the background error variances.

scholarly journals Multivariate Error Covariance Estimates by Monte Carlo Simulation for Assimilation Studies in the Pacific Ocean

2005 ◽  
Vol 133 (8) ◽  
pp. 2310-2334 ◽  
Author(s):  
Anna Borovikov ◽  
Michele M. Rienecker ◽  
Christian L. Keppenne ◽  
Gregory C. Johnson
Keyword(s):  
Monte Carlo ◽  
Data Assimilation ◽  
Forecast Error ◽  
Independent Set ◽  
Velocity Fields ◽  
Optimal Interpolation ◽  
Model Errors ◽  
Error Statistics ◽  
Practical Applications ◽  
Monte Carlo Techniques

Abstract One of the most difficult aspects of ocean-state estimation is the prescription of the model forecast error covariances. The paucity of ocean observations limits our ability to estimate the covariance structures from model–observation differences. In most practical applications, simple covariances are usually prescribed. Rarely are cross covariances between different model variables used. Here a comparison is made between a univariate optimal interpolation (UOI) scheme and a multivariate OI algorithm (MvOI) in the assimilation of ocean temperature profiles. In the UOI case only temperature is updated using a Gaussian covariance function. In the MvOI, salinity, zonal, and meridional velocities as well as temperature are updated using an empirically estimated multivariate covariance matrix. Earlier studies have shown that a univariate OI has a detrimental effect on the salinity and velocity fields of the model. Apparently, in a sequential framework it is important to analyze temperature and salinity together. For the MvOI an estimate of the forecast error statistics is made by Monte Carlo techniques from an ensemble of model forecasts. An important advantage of using an ensemble of ocean states is that it provides a natural way to estimate cross covariances between the fields of different physical variables constituting the model-state vector, at the same time incorporating the model’s dynamical and thermodynamical constraints as well as the effects of physical boundaries. Only temperature observations from the Tropical Atmosphere–Ocean array have been assimilated in this study. To investigate the efficacy of the multivariate scheme, two data assimilation experiments are validated with a large independent set of recently published subsurface observations of salinity, zonal velocity, and temperature. For reference, a control run with no data assimilation is used to check how the data assimilation affects systematic model errors. While the performance of the UOI and MvOI is similar with respect to the temperature field, the salinity and velocity fields are greatly improved when the multivariate correction is used, as is evident from the analyses of the rms differences between these fields and independent observations. The MvOI assimilation is found to improve upon the control run in generating water masses with properties close to the observed, while the UOI fails to maintain the temperature and salinity structure.

How is the Balance of a Forecast Ensemble Affected by Adaptive and Nonadaptive Localization Schemes?

2015 ◽  
Vol 143 (9) ◽  
pp. 3680-3699 ◽  
Author(s):  
Ross N. Bannister
Keyword(s):  
Forecast Error ◽  
Covariance Matrices ◽  
Error Statistics ◽  
Error Covariance ◽  
Localization Scheme ◽  
Adaptive Schemes ◽  
Convective Scale ◽  
Free Parameters ◽  
Hydrostatic Balance ◽  
First Time

Abstract This paper investigates the effect on balance of a number of Schur product–type localization schemes that have been designed with the primary function of reducing spurious far-field correlations in forecast error statistics. The localization schemes studied comprise a nonadaptive scheme (where the moderation matrix is decomposed in a spectral basis), and two adaptive schemes: a simplified version of Smoothed Ensemble Correlations Raised to a Power (SENCORP) and Ensemble Correlations Raised to a Power (ECO-RAP). The paper shows, the author believes for the first time, how the degree of balance (geostrophic and hydrostatic) implied by the error covariance matrices localized by these schemes can be diagnosed. Here it is considered that an effective localization scheme is one that reduces spurious correlations adequately, but also minimizes disruption of balance (where the “correct” degree of balance or imbalance is assumed to be possessed by the unlocalized ensemble). By varying free parameters that describe each scheme (e.g., the degree of truncation in the schemes that use the spectral basis, the “order” of each scheme, and the degree of ensemble smoothing), it is found that a particular configuration of the ECO-RAP scheme is best suited to the convective-scale system studied. According to the diagnostics this ECO-RAP configuration still weakens geostrophic and hydrostatic balance, but overall this is less so than for other schemes.

Theoretical Application of Ensemble Kalman Filter to Adsorptive Solute Cr(VI) Transfer from Soil into Surface Runoff

2014 ◽  
Vol 919-921 ◽  
pp. 1257-1261
Author(s):  
Chao Qun Tan ◽  
Ju Xiu Tong ◽  
Bill X. Hu ◽  
Jin Zhong Yang
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Surface Runoff ◽  
Amplification Factor ◽  
Forecast Error ◽  
Assimilation Effect ◽  
Solute Transfer ◽  
Error Covariance ◽  
Theoretical Application

This paper mainly discusses some details when applying data assimilation method via an ensemble Kalman filter (EnKF) to improve prediction of adsorptive solute Cr(VI) transfer from soil into runoff. Based on this work, we could make better use of our theoretical model to predict adsorptive solute transfer from soil into surface runoff in practice. The results show that the ensemble number of 100 is reasonable, considering assimilation effect and efficiency after selecting its number from 25 to 225 at an interval of 25. While the initial ensemble value makes little difference to data assimilation (DA) results. Besides, DA results could be improved by multiplying an amplification factor to forecast error covariance matrix due to underestimation of forecast error.

scholarly journals Multiscale Representation of Observation Error Statistics in Data Assimilation

Sensors ◽  
2020 ◽  
Vol 20 (5) ◽  
pp. 1460
Author(s):  
Vincent Chabot ◽  
Maëlle Nodet ◽  
Arthur Vidard
Keyword(s):  
Data Assimilation ◽  
Real Life ◽  
Covariance Matrices ◽  
Multiscale Modelling ◽  
Observation Error ◽  
Error Statistics ◽  
Life Data ◽  
Promising Tool ◽  
Error Covariance ◽  
Real Life Data

Accounting for realistic observation errors is a known bottleneck in data assimilation, because dealing with error correlations is complex. Following a previous study on this subject, we propose to use multiscale modelling, more precisely wavelet transform, to address this question. This study aims to investigate the problem further by addressing two issues arising in real-life data assimilation: how to deal with partially missing data (e.g., concealed by an obstacle between the sensor and the observed system), and how to solve convergence issues associated with complex observation error covariance matrices? Two adjustments relying on wavelets modelling are proposed to deal with those, and offer significant improvements. The first one consists of adjusting the variance coefficients in the frequency domain to account for masked information. The second one consists of a gradual assimilation of frequencies. Both of these fully rely on the multiscale properties associated with wavelet covariance modelling. Numerical results on twin experiments show that multiscale modelling is a promising tool to account for correlations in observation errors in realistic applications.

scholarly journals The Development of a Hybrid EnKF-3DVAR Algorithm for Storm-Scale Data Assimilation

2013 ◽  
Vol 2013 ◽  
pp. 1-12 ◽  
Author(s):  
Jidong Gao ◽  
Ming Xue ◽  
David J. Stensrud
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Control Variable ◽  
Radar Data ◽  
Ensemble Forecasts ◽  
Advanced Regional Prediction System ◽  
Error Covariance ◽  
Variable Approach ◽  
Regional Prediction ◽  
Rms Errors

A hybrid 3DVAR-EnKF data assimilation algorithm is developed based on 3DVAR and ensemble Kalman filter (EnKF) programs within the Advanced Regional Prediction System (ARPS). The hybrid algorithm uses the extended alpha control variable approach to combine the static and ensemble-derived flow-dependent forecast error covariances. The hybrid variational analysis is performed using an equal weighting of static and flow-dependent error covariance as derived from ensemble forecasts. The method is first applied to the assimilation of simulated radar data for a supercell storm. Results obtained using 3DVAR (with static covariance entirely), hybrid 3DVAR-EnKF, and the EnKF are compared. When data from a single radar are used, the EnKF method provides the best results for the model dynamic variables, while the hybrid method provides the best results for hydrometeor related variables in term of rms errors. Although storm structures can be established reasonably well using 3DVAR, the rms errors are generally worse than seen from the other two methods. With two radars, the results from 3DVAR are closer to those from EnKF. Our tests indicate that the hybrid scheme can reduce the storm spin-up time because it fits the observations, especially the reflectivity observations, better than the EnKF and the 3DVAR at the beginning of the assimilation cycles.

scholarly journals Efficient Adaptive Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to the Control of Ocean Mesoscale Signals

2010 ◽  
Vol 138 (3) ◽  
pp. 932-950 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Error Estimates ◽  
Forecast Error ◽  
Observation Error ◽  
Square Root ◽  
Error Statistics ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Optimal Estimates

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

scholarly journals Localized Ensemble-Based Tangent Linear Models and Their Use in Propagating Hybrid Error Covariance Models

2016 ◽  
Vol 144 (4) ◽  
pp. 1383-1405 ◽  
Author(s):  
Sergey Frolov ◽  
Craig H. Bishop
Keyword(s):  
Data Assimilation ◽  
Short Duration ◽  
Linear Models ◽  
Forecast Error ◽  
Weather Forecast ◽  
Step Function ◽  
Rank Deficiency ◽  
One Dimensional ◽  
Error Covariance ◽  
Covariance Models

Abstract Hybrid error covariance models that blend climatological estimates of forecast error covariances with ensemble-based, flow-dependent forecast error covariances have led to significant reductions in forecast error when employed in 4DVAR data assimilation schemes. Tangent linear models (TLMs) designed to predict the differences between perturbed and unperturbed simulations of the weather forecast are a key component of such 4DVAR schemes. However, many forecasting centers have found that TLMs and their adjoints do not scale well computationally and are difficult to create and maintain—particularly for coupled ocean–wave–ice–atmosphere models. In this paper, the authors create ensemble-based TLMs (ETLMs) and test their ability to propagate both climatological and flow-dependent parts of hybrid error covariance models. These tests demonstrate that rank deficiency limits the utility of unlocalized ETLMs. High-rank, time-evolving, flow-adaptive localization functions are constructed and tested using recursive application of short-duration ETLMs, each of which is localized using a static localization. Since TLM operators do not need to be semipositive definite, the authors experiment with a variety of localization approaches including step function localization. The step function localization leads to a local formulation that was found to be highly effective. In tests using simple one-dimensional models with both dispersive and nondispersive dynamics, it is shown that practical ETLM configurations were effective at propagating covariances as far as four error correlation scales.

Sign in / Sign up

Export Citation Format

Share Document