Accounting for Model Errors in Ensemble Data Assimilation

2009 ◽  
Vol 137 (10) ◽  
pp. 3407-3419 ◽  
Author(s):  
Hong Li ◽  
Eugenia Kalnay ◽  
Takemasa Miyoshi ◽  
Christopher M. Danforth
Keyword(s):  
Kalman Filter ◽  
Forecast Errors ◽  
Random Errors ◽  
Model Errors ◽  
State Dependent ◽  
Constant Bias ◽  
Perfect Model ◽  
Ensemble Transform Kalman Filter ◽  
Low Dimensional ◽  
Bias Removal

Abstract This study addresses the issue of model errors with the ensemble Kalman filter. Observations generated from the NCEP–NCAR reanalysis fields are assimilated into a low-resolution AGCM. Without an effort to account for model errors, the performance of the local ensemble transform Kalman filter (LETKF) is seriously degraded when compared with the perfect-model scenario. Several methods to account for model errors, including model bias and system noise, are investigated. The results suggest that the two pure bias removal methods considered [Dee and Da Silva (DdSM) and low dimensional (LDM)] are not able to beat the multiplicative or additive inflation schemes used to account for the effects of total model errors. In contrast, when the bias removal methods are augmented by additive noise representing random errors (DdSM+ and LDM+), they outperform the pure inflation schemes. Of these augmented methods, the LDM+, where the constant bias, diurnal bias, and state-dependent errors are estimated from a large sample of 6-h forecast errors, gives the best results. The advantage of the LDM+ over other methods is larger in data-sparse regions than in data-dense regions.

scholarly journals Comparison between Local Ensemble Transform Kalman Filter and PSAS in the NASA finite volume GCM – perfect model experiments

2008 ◽  
Vol 15 (4) ◽  
pp. 645-659 ◽  
Author(s):  
J. Liu ◽  
E. J. Fertig ◽  
H. Li ◽  
E. Kalnay ◽  
B. R. Hunt ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Finite Volume ◽  
Circulation Model ◽  
Forecast Errors ◽  
Global Circulation Model ◽  
Perfect Model ◽  
Grid Points ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Assimilation System

Abstract. This paper compares the performance of the Local Ensemble Transform Kalman Filter (LETKF) with the Physical-Space Statistical Analysis System (PSAS) under a perfect model scenario. PSAS is a 3D-Var assimilation system used operationally in the Goddard Earth Observing System Data Assimilation System (GEOS-4 DAS). The comparison is carried out using simulated winds and geopotential height observations and the finite volume Global Circulation Model with 72 grid points zonally, 46 grid points meridionally and 55 vertical levels. With forty ensemble members, the LETKF obtains analyses and forecasts with significantly lower RMS errors than those from PSAS, especially over the Southern Hemisphere and oceans. This observed advantage of the LETKF over PSAS is due to the ability of the 40-member ensemble LETKF to capture flow-dependent errors and thus create a good estimate of the evolving background uncertainty. An initial decrease of the forecast errors in the Northern Hemisphere observed in the PSAS but not in the LETKF suggests that the LETKF analysis is more balanced.

The Geometry of Model Error

2008 ◽  
Vol 65 (6) ◽  
pp. 1749-1772 ◽  
Author(s):  
Kevin Judd ◽  
Carolyn A. Reynolds ◽  
Thomas E. Rosmond ◽  
Leonard A. Smith
Keyword(s):  
Weather Forecasting ◽  
Model Space ◽  
Forecast Model ◽  
Model Error ◽  
Forecast Errors ◽  
Model Errors ◽  
Direction Error ◽  
Forecasting System ◽  
Projection Error ◽  
Low Dimensional

Abstract This paper investigates the nature of model error in complex deterministic nonlinear systems such as weather forecasting models. Forecasting systems incorporate two components, a forecast model and a data assimilation method. The latter projects a collection of observations of reality into a model state. Key features of model error can be understood in terms of geometric properties of the data projection and a model attracting manifold. Model error can be resolved into two components: a projection error, which can be understood as the model’s attractor being in the wrong location given the data projection, and direction error, which can be understood as the trajectories of the model moving in the wrong direction compared to the projection of reality into model space. This investigation introduces some new tools and concepts, including the shadowing filter, causal and noncausal shadow analyses, and various geometric diagnostics. Various properties of forecast errors and model errors are described with reference to low-dimensional systems, like Lorenz’s equations; then, an operational weather forecasting system is shown to have the same predicted behavior. The concepts and tools introduced show promise for the diagnosis of model error and the improvement of ensemble forecasting systems.

scholarly journals Strengths and Weaknesses of MOS, Running-Mean Bias Removal, and Kalman Filter Techniques for Improving Model Forecasts over the Western United States

2007 ◽  
Vol 22 (6) ◽  
pp. 1304-1318 ◽  
Author(s):  
William Y. Y. Cheng ◽  
W. James Steenburgh
Keyword(s):  
United States ◽  
Kalman Filter ◽  
Initial Conditions ◽  
Weather Prediction ◽  
Western United States ◽  
Training Period ◽  
Cold Pool ◽  
Model Errors ◽  
Model Biases ◽  
Bias Removal

Abstract Despite improvements in numerical weather prediction, model errors, particularly near the surface, are unavoidable due to imperfect model physics, initial conditions, and boundary conditions. Here, three techniques for improving the accuracy of 2-m temperature, 2-m dewpoint, and 10-m wind forecasts by the Eta/North American Meso (NAM) Model are evaluated: (i) traditional model output statistics (ETAMOS), requiring a relatively long training period; (ii) the Kalman filter (ETAKF), requiring a relatively short initial training period (∼4–5 days); and (iii) 7-day running mean bias removal (ETA7DBR), requiring a 7-day training period. Forecasts based on the ETAKF and ETA7DBR methods were produced for more than 2000 MesoWest observing sites in the western United States. However, the evaluation presented in this study was based on subjective forecaster assessments and objective verification at 145 ETAMOS stations during summer 2004 and winter 2004/05. For the 145-site sample, ETAMOS produces the most accurate cumulative temperature, dewpoint, and wind speed and direction forecasts, followed by ETAKF and ETA7DBR, which have similar accuracy. Selected case studies illustrate that ETAMOS produces superior forecasts when model biases change dramatically, such as during large-scale pattern changes, but that ETAKF and ETA7DBR produce superior forecasts during quiescent cool season patterns when persistent valley and basin cold pools exist. During quiescent warm season patterns, the accuracy of all three methods is similar. Although the improved ETAKF cold pool forecasts are noteworthy, particularly since the Kalman filter can help better define cold pool structure by producing forecasts for locations without long-term records, alternative approaches are needed to improve forecasts during periods when model biases change dramatically.

scholarly journals Growth of Forecast Errors from Covariances Modeled by 4DVAR and ETKF Methods

2011 ◽  
Vol 139 (5) ◽  
pp. 1505-1518 ◽  
Author(s):  
Chiara Piccolo
Keyword(s):  
Weather Forecasting ◽  
Initial Conditions ◽  
Forecast Error ◽  
Forecast Errors ◽  
Model Errors ◽  
Background Error ◽  
Background Error Covariance ◽  
Error Covariance ◽  
Numerical Weather Forecasting ◽  
Ensemble Transform Kalman Filter

Numerical weather forecasting errors grow with time. Error growth results from the amplification of small perturbations due to atmospheric instability or from model deficiencies during model integration. In current NWP systems, the dimension of the forecast error covariance matrices is far too large for these matrices to be represented explicitly. They must be approximated. This paper focuses on comparing the growth of forecast error from covariances modeled by the Met Office operational four-dimensional variational data assimilation (4DVAR) and ensemble transform Kalman filter (ETKF) methods over a period of 24 h. The growth of forecast errors implied by 4DVAR is estimated by drawing a random sample of initial conditions from a Gaussian distribution with the standard deviations given by the background error covariance matrix and then evolving the sample forward in time using linearized dynamics. The growth of the forecast error modeled by the ETKF is estimated by propagating the full nonlinear model in time starting from initial conditions generated by an ETKF. This method includes model errors in two ways: by using an inflation factor and by adding model perturbations through a stochastic physics scheme. Finally, these results are compared with a benchmark of the climatological error. The forecast error predicted by the implicit evolution of 4DVAR does not grow, regardless of the dataset used to generate the static background error covariance statistics. The forecast error predicted by the ETKF grows more rapidly because the ETKF selects balanced initial perturbations, which project onto rapidly growing modes. Finally, in both cases it is not possible to disentangle the contribution of the initial condition error from the model error.

scholarly journals The Gaussian Approach to Adaptive Covariance Inflation and Its Implementation with the Local Ensemble Transform Kalman Filter

2011 ◽  
Vol 139 (5) ◽  
pp. 1519-1535 ◽  
Author(s):  
Takemasa Miyoshi
Keyword(s):  
Kalman Filter ◽  
General Circulation ◽  
Forecast Error ◽  
Grid Point ◽  
Circulation Model ◽  
Error Variance ◽  
Model Errors ◽  
Variable Model ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

In ensemble Kalman filters, the underestimation of forecast error variance due to limited ensemble size and other sources of imperfection is commonly treated by empirical covariance inflation. To avoid manual optimization of multiplicative inflation parameters, previous studies proposed adaptive inflation approaches using observations. Anderson applied Bayesian estimation theory to the probability density function of inflation parameters. Alternatively, Li et al. used the innovation statistics of Desroziers et al. and applied a Kalman filter analysis update to the inflation parameters based on the Gaussian assumption. In this study, Li et al.’s Gaussian approach is advanced to include the variance of the estimated inflation as derived from the central limit theorem. It is shown that the Gaussian approach is an accurate approximation of Anderson’s general Bayesian approach. An advanced implementation of the Gaussian approach with the local ensemble transform Kalman filter is proposed, where the adaptive inflation parameters are computed simultaneously with the ensemble transform matrix at each grid point. The spatially and temporally varying adaptive inflation technique is implemented with the Lorenz 40-variable model and a low-resolution atmospheric general circulation model; numerical experiments show promising results both with and without model errors.

scholarly journals A Robust Extended Kalman Filter Applied to Ultrawideband Positioning

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Chuanyang Wang ◽  
Houzeng Han ◽  
Jian Wang ◽  
Hang Yu ◽  
Deng Yang
Keyword(s):  
Kalman Filter ◽  
Extended Kalman Filter ◽  
Robust Estimation ◽  
Hypothesis Test ◽  
Systematic Errors ◽  
Random Errors ◽  
Model Errors ◽  
Global Test ◽  
Practical Applications ◽  
The Individual

Ultrawideband (UWB) is well-suited for indoor positioning due to its high resolution and good penetration through objects. The observation model of UWB positioning is nonlinear. As one of nonlinear filter algorithms, extended Kalman filter (EKF) is widely used to estimate the position. In practical applications, the dynamic estimation is subject to the outliers caused by gross errors. However, the EKF cannot resist the effect of gross errors. The innovation will become abnormally large and the performance and the reliability of the filter algorithm are inevitably influenced. In this study, a robust EKF (REKF) method accompanied by hypothesis test and robust estimation is proposed. To judge the validity of model, the global test based on Mahalanobis distance is implemented to assess whether the test statistical term exceeds the threshold for outlier detection. To reduce and eliminate the effects of the individual outlier, the robust estimation using scheme III of the Institute of Geodesy and Geophysics of China (IGGIII) based on local test of the normalized residual is performed. Meanwhile, three kinds of stochastic models for outliers are expressed by modeling the contaminated distributions. Furthermore, the simulation and measurement experiments are performed to verify the effectiveness and feasibility of the proposed REKF for resisting the outliers. Simulation experiment results are given to demonstrate that the outliers following all the three kinds of contaminated distributions can be detected. The proposed REKF can effectively control the influences of the outliers being treated as systematic errors and large variance random errors. When the outliers come from the thick-tailed distribution, the robust estimation does not play a role, and the REKF are equivalent to the EKF method. The measured experiment results show that the outliers will be generated in the nonline-of-sight environment whose impact is abnormally serious. The robust estimation can provide relatively reliable optimized residuals and control the influences of the outliers caused by gross errors. We can believe that the proposed REKF is effective to resist the effects of outliers and improves the positioning accuracy compared with least-squares (LS) and EKF method. Moreover, the adaptive filter and ranging error model should be considered to compensate the state model errors and ranging systematic errors respectively. Then, the measurement outliers will be detected more correctly, and the robust estimation will be used effectively.

Model forecast error correction based on the Local Dynamical Analog method: an example application to the ENSO forecast

2020 ◽  
Author(s):  
Zhaolu Hou ◽  
Bin Zuo ◽  
Shaoqing Zhang ◽  
Fei Huang ◽  
Ruiqiang Ding ◽  
...  
Keyword(s):  
Error Correction ◽  
Forecast Error ◽  
Forecast Errors ◽  
Model Errors ◽  
Model Forecast ◽  
New Approach ◽  
Correct Model ◽  
State Dependent ◽  
Dependent Model ◽  
Scalar Time Series

<p>Numerical forecasts always have associated errors. Analogue correction methods combine numerical simulations with statistical analyses to reduce model forecast errors. However, identifying appropriate analogues remains a challenging task. Here, we use the Local Dynamical Analog (LDA) method to locate analogues and correct model forecast errors. As an example, an ENSO model forecast error correction experiment confirms that the LDA method locates more dynamical analogues of states of interest and better corrects forecast errors than do other methods. This is because the LDA method ensures similarity of the initial states and the evolution of both states. In addition, the LDA method can be applied using a scalar time series, which reduces the complexity of the dynamical system. Model forecast error correction using the LDA method provides a new approach to correcting state-dependent model errors and can be readily integrated with other advanced models.</p>

scholarly journals System bias correction of short-term hub-height wind forecasts using the Kalman filter

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Jingjing Xu ◽  
Ziniu Xiao ◽  
Zhaohui Lin ◽  
Ming Li
Keyword(s):  
Kalman Filter ◽  
Power Systems ◽  
Wind Power ◽  
Bias Correction ◽  
Wind Farm ◽  
Wrf Model ◽  
Forecast Errors ◽  
Random Errors ◽  
Power Prediction ◽  
Short Term

AbstractWind energy is a fluctuating source for power systems, which poses challenges to grid planning for the wind power industry. To improve the short-term wind forecasts at turbine height, the bias correction approach Kalman filter (KF) is applied to 72-h wind speed forecasts from the WRF model in Zhangbei wind farm for a period over two years. The KF approach shows a remarkable ability in improving the raw forecasts by decreasing the root-mean-square error by 16% from 3.58 to 3.01 m s−1, the mean absolute error by 14% from 2.71 to 2.34 m s−1, the bias from 0.22 to − 0.19 m s−1, and improving the correlation from 0.58 to 0.66. The KF significantly reduces random errors of the model, showing the capability to deal with the forecast errors associated with physical processes which cannot be accurately handled by the numerical model. In addition, the improvement of the bias correction is larger for wind speeds sensitive to wind power generation. So the KF approach is suitable for short-term wind power prediction.

scholarly journals Examination of Analysis and Forecast Errors of High-Resolution Assimilation, Bias Removal, and Digital Filter Initialization with an Ensemble Kalman Filter

2012 ◽  
Vol 140 (12) ◽  
pp. 3992-4004 ◽  
Author(s):  
Brian C. Ancell
Keyword(s):  
Kalman Filter ◽  
High Resolution ◽  
Digital Filter ◽  
Ensemble Kalman Filter ◽  
Weather Prediction ◽  
Surface Wind ◽  
Forecast Errors ◽  
Temperature Bias ◽  
High Resolution Analysis ◽  
Bias Removal

Abstract Mesoscale atmospheric data assimilation is becoming an integral part of numerical weather prediction. Modern computational resources now allow assimilation and subsequent forecasting experiments ranging from resolutions of tens of kilometers over regional domains to smaller grids that employ storm-scale assimilation. To assess the value of the high-resolution capabilities involved with assimilation and forecasting at different scales, analyses and forecasts must be carefully evaluated to understand 1) whether analysis benefits gained at finer scales persist into the forecast relative to downscaled runs begun from lower-resolution analyses, 2) how the positive analysis effects of bias removal evolve into the forecast, and 3) how digital filter initialization affects analyses and forecasts. This study applies a 36- and 4-km ensemble Kalman filter over 112 assimilation cycles to address these important issues, which could all be relevant to a variety of short-term, high-resolution, real-time forecasting applications. It is found that with regard to surface wind and temperature, analysis improvements gained at higher resolution persist throughout the 12-h forecast window relative to downscaled, high-resolution forecasts begun from analyses on the coarser grid. Aloft, however, no forecast improvements were found with the high-resolution analysis/forecast runs. Surface wind and temperature bias removal, while clearly improving surface analyses, degraded surface forecasts and showed little forecast influence aloft. Digital filter initialization degraded temperature analyses with or without bias removal, degraded wind analyses when bias removal was used, but improved wind analyses when bias removal was absent. No forecast improvements were found with digital filter initialization. The consequences of these results with regard to operational assimilation/forecasting systems on nested grids are discussed.

Characteristics of Target Areas Selected by the Ensemble Transform Kalman Filter for Medium-Range Forecasts of High-Impact Winter Weather

2010 ◽  
Vol 138 (7) ◽  
pp. 2803-2824 ◽  
Author(s):  
Sharanya J. Majumdar ◽  
Kathryn J. Sellwood ◽  
Daniel Hodyss ◽  
Zoltan Toth ◽  
Yucheng Song
Keyword(s):  
Kalman Filter ◽  
Storm Track ◽  
High Impact ◽  
Lead Times ◽  
Forecast Errors ◽  
Long Distance ◽  
Multiple Targets ◽  
Medium Range ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform

Abstract The characteristics of “target” locations of tropospheric wind and temperature identified by a modified version of the ensemble transform Kalman filter (ETKF), in order to reduce 0–7-day forecast errors over North America, are explored from the perspective of a field program planner. Twenty cases of potential high-impact weather over the continent were investigated, using a 145-member ensemble comprising perturbations from NCEP, ECMWF, and the Canadian Meteorological Centre (CMC). Multiple targets were found to exist in the midlatitude storm track. In half of the cases, distinctive targets could be traced upstream near Japan at lead times of 4–7 days. In these cases, the flow was predominantly zonal and a coherent Rossby wave packet was present over the northern Pacific Ocean. The targets at the longest lead times were often located within propagating areas of baroclinic energy conversion far upstream. As the lead time was reduced, these targets were found to diminish in importance, with downstream targets corresponding to a separate synoptic system gaining in prominence. This shift in optimal targets is sometimes consistent with the radiation of ageostrophic geopotential fluxes and transfer of eddy kinetic energy downstream, associated with downstream baroclinic development. Concurrently, multiple targets arise due to spurious long-distance correlations in the ETKF. The targets were least coherent in blocked flows, in which the ETKF is known to be least reliable. The effectiveness of targeting in the medium range requires evaluation, using data such as those collected during the winter phase of The Observing System Research and Predictability Experiment (THORPEX) Pacific Asian Regional Field Campaign (T-PARC) in 2009.

Sign in / Sign up

Export Citation Format

Share Document