scholarly journals Investigation of Local Weighting Filtering on Randomization Technique Estimates in a Data Assimilation System

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1563
Author(s):  
Xiang Xing ◽  
Bainian Liu ◽  
Weimin Zhang ◽  
Xiaoqun Cao ◽  
Jingzhe Sun
Keyword(s):  
Data Assimilation ◽  
Weather Prediction ◽  
Computational Cost ◽  
Atmospheric Model ◽  
Small Scale ◽  
State Variables ◽  
Model Framework ◽  
Background Error ◽  
Filtering Method ◽  
Weighted Correlation

Mainstream numerical weather prediction (NWP) centers usually estimate the standard deviations of background error by using a randomization technique to calibrate specific parameters of the background error covariance model in variational data assimilation (VAR) systems. However, the sampling size of the randomization technique is typically several orders of magnitude smaller than that of model state variables, and using finite-sized estimates as a proxy for the truth can lead to sampling noise, which may contaminate the estimation of the standard deviation. The sampling noise is firstly investigated in an atmospheric model to show that the sampling noise has a symmetrical structure oscillating around the truth on a small scale. To alleviate the sampling noise, a heterogeneous local weighting filtering is proposed based on distance-weighted correlation and similarity-weighted correlation. Local weighting filtering is easy to implement in the VAR operational systems and has a low computational cost in the post-processing of reducing the sampling noise. The validity and performance of local weighting filtering method are examined in a realistic model framework to show that the proposed filtering is able to eliminate most of the sampling noise dramatically, the details of the filtered results are more visible, and the accuracy of the filtered results is almost the same as that estimated from the larger sample. The signal-to-noise ratio of the optimal filtered field is improved by nearly 20%. A comparison with the widely used spectral filtering approach in the operational system is considered, showing that the proposed filtering method is more efficient to implement in the filtering procedure and exhibits very good performance in terms of preserving the local anisotropic features of the estimates. These attractive results show the potential efficiency of the local weighting filtering method for solving the noise issue in the randomization technique.

scholarly journals Linear Filtering of Sample Covariances for Ensemble-Based Data Assimilation. Part II: Application to a Convective-Scale NWP Model

2015 ◽  
Vol 143 (5) ◽  
pp. 1644-1664 ◽  
Author(s):  
Benjamin Ménétrier ◽  
Thibaut Montmerle ◽  
Yann Michel ◽  
Loïk Berre
Keyword(s):  
Data Assimilation ◽  
Weather Prediction ◽  
Computational Cost ◽  
Localization Length ◽  
Optimality Criteria ◽  
Global Scale ◽  
Linear Filtering ◽  
Background Error ◽  
Full Generality ◽  
Convective Scale

Abstract In Part I of this two-part study, a new theory for optimal linear filtering of covariances sampled from an ensemble of forecasts was detailed. This method, especially designed for data assimilation (DA) schemes in numerical weather prediction (NWP) systems, has the advantage of using optimality criteria that involve sample estimated quantities and filter output only. In this second part, the theory is tested with real background error covariances computed using a large ensemble data assimilation (EDA) at the convective scale coupled with a large EDA at the global scale, based respectively on the Applications of Research to Operations at Mesoscale (AROME) and ARPEGE operational NWP systems. Background error variances estimated with a subset of this ensemble are filtered and evaluated against values obtained with the remaining members, which are considered as an independent reference. Algorithms presented in Part I show relevant results, with the homogeneous filtering being quasi optimal. Heterogeneous filtering is also successfully tested with different local criteria, yet at a higher computational cost, showing the full generality of the method. As a second application, horizontal and vertical localization functions are diagnosed from the ensemble, providing pertinent localization length scales that consistently depend on the number of members, on the meteorological variables, and on the vertical levels.

scholarly journals The Impact of Length-Scale Variation When Diagnosing the Standard Deviations of Background Error in a 4D-Var System and Filtering Method Investigation

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Xiang Xing ◽  
Bainian Liu ◽  
Weimin Zhang ◽  
Xiaoqun Cao ◽  
Hongze Leng
Keyword(s):  
Standard Deviation ◽  
Data Assimilation ◽  
Variational Data Assimilation ◽  
Small Scale ◽  
Model Parameters ◽  
Spatial Averaging ◽  
Background Error ◽  
Operational System ◽  
Filtering Method ◽  
Standard Deviations

The four-dimensional variational data assimilation (4D-Var) method has been widely employed as an operational scheme in mainstream numerical weather prediction (NWP) centers. In addition to the ensemble data assimilation method, the randomization technique is still used to diagnose the standard deviations of background error in variational data assimilation (VAR) systems; however, such randomization techniques induce sampling noise, which may contaminate the quality of the standard deviations. First, this paper studies the properties of the sampling noise induced by the randomization technique. The results show that the sampling noise is on a small scale displaying high-frequency oscillations around the estimate compared with the estimate and this difference motivates the use of filtering techniques to eliminate the sampling noise effects. The characteristics of the standard deviation field of the control variables are also investigated, and the standard deviation fields of different model parameters have different scales and vary with the vertical model levels. To eliminate such sampling noise, the spectral filtering method used widely in the operational system and a modified spatial averaging approach are investigated. Although both methods have splendid performance in eliminating sampling noise, the spatial averaging approach is more efficient and easier to implement in operational systems. In addition, the optimal filtered results from the spatial averaging approach are dependent on model parameters and vertical levels, which is consistent with the variation in the standard deviation field. Finally, the spatial averaging approach is tested on the operational system at the global scale based on the YH4DVAR and the global NWP system, and the results indicate that the spatial averaging approach has positive effects on both analysis and forecast quality.

An Efficient Dual-Resolution Approach for Ensemble Data Assimilation and Tests with Simulated Doppler Radar Data

2008 ◽  
Vol 136 (3) ◽  
pp. 945-963 ◽  
Author(s):  
Jidong Gao ◽  
Ming Xue
Keyword(s):  
High Resolution ◽  
Data Assimilation ◽  
Radial Velocity ◽  
Computational Cost ◽  
Radar Data ◽  
Horizontal Resolution ◽  
Velocity Data ◽  
Background Error ◽  
Cross Covariance ◽  
Radial Velocity Data

Abstract A new efficient dual-resolution (DR) data assimilation algorithm is developed based on the ensemble Kalman filter (EnKF) method and tested using simulated radar radial velocity data for a supercell storm. Radar observations are assimilated on both high-resolution and lower-resolution grids using the EnKF algorithm with flow-dependent background error covariances estimated from the lower-resolution ensemble. It is shown that the flow-dependent and dynamically evolved background error covariances thus estimated are effective in producing quality analyses on the high-resolution grid. The DR method has the advantage of being able to significantly reduce the computational cost of the EnKF analysis. In the system, the lower-resolution ensemble provides the flow-dependent background error covariance, while the single-high-resolution forecast and analysis provides the benefit of higher resolution, which is important for resolving the internal structures of thunderstorms. The relative smoothness of the covariance obtained from the lower 4-km-resolution ensemble does not appear to significantly degrade the quality of analysis. This is because the cross covariance among different variables is of first-order importance for “retrieving” unobserved variables from the radar radial velocity data. For the DR analysis, an ensemble size of 40 appears to be a reasonable choice with the use of a 4-km horizontal resolution in the ensemble and a 1-km resolution in the high-resolution analysis. Several sensitivity tests show that the DR EnKF system is quite robust to different observation errors. A 4-km thinned data resolution is a compromise that is acceptable under the constraint of real-time applications. A data density of 8 km leads to a significant degradation in the analysis.

Evaluation of Surface Analyses and Forecasts with a Multiscale Ensemble Kalman Filter in Regions of Complex Terrain

2011 ◽  
Vol 139 (6) ◽  
pp. 2008-2024 ◽  
Author(s):  
Brian C. Ancell ◽  
Clifford F. Mass ◽  
Gregory J. Hakim
Keyword(s):  
Kalman Filter ◽  
High Resolution ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Complex Terrain ◽  
Error Variance ◽  
Small Scale ◽  
Observation Error ◽  
Grid Spacing ◽  
Background Error

Abstract Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30–50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%–30%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested.

scholarly journals Background Error in WRF Model

2020 ◽  
Vol 16 ◽  
Keyword(s):  
Data Assimilation ◽  
Physical Properties ◽  
Covariance Matrix ◽  
Numerical Weather Prediction ◽  
Weather Prediction ◽  
Wrf Model ◽  
Background Error ◽  
Numerical Weather ◽  
Error Covariance ◽  
First Time

WRF model have been tuned and tested over Georgia’s territory for years. First time in Georgia theprocess of data assimilation in Numerical weather prediction is developing. This work presents how forecasterror statistics appear in the data assimilation problem through the background error covariance matrix – B, wherethe variances and correlations associated with model forecasts are estimated. Results of modeling of backgrounderror covariance matrix for control variables using WRF model over Georgia with desired domain configurationare discussed and presented. The modeling was implemented in two different 3DVAR systems (WRFDA andGSI) and results were checked by pseudo observation benchmark cases using also default global and regional BEmatrixes. The mathematical and physical properties of the covariances are also reviewed.

scholarly journals Development of a Hybrid En3DVar Data Assimilation System and Comparisons with 3DVar and EnKF for Radar Data Assimilation with Observing System Simulation Experiments

2018 ◽  
Vol 146 (1) ◽  
pp. 175-198 ◽  
Author(s):  
Rong Kong ◽  
Ming Xue ◽  
Chengsi Liu
Keyword(s):  
Data Assimilation ◽  
System Simulation ◽  
Radar Data ◽  
Mixing Ratio ◽  
State Variables ◽  
Ensemble Size ◽  
Simulation Experiments ◽  
Background Error ◽  
Variational Minimization ◽  
Water Vapor Mixing Ratio

Abstract A hybrid ensemble–3DVar (En3DVar) system is developed and compared with 3DVar, EnKF, “deterministic forecast” EnKF (DfEnKF), and pure En3DVar for assimilating radar data through perfect-model observing system simulation experiments (OSSEs). DfEnKF uses a deterministic forecast as the background and is therefore parallel to pure En3DVar. Different results are found between DfEnKF and pure En3DVar: 1) the serial versus global nature and 2) the variational minimization versus direct filter updating nature of the two algorithms are identified as the main causes for the differences. For 3DVar (EnKF/DfEnKF and En3DVar), optimal decorrelation scales (localization radii) for static (ensemble) background error covariances are obtained and used in hybrid En3DVar. The sensitivity of hybrid En3DVar to covariance weights and ensemble size is examined. On average, when ensemble size is 20 or larger, a 5%–10% static covariance gives the best results, while for smaller ensembles, more static covariance is beneficial. Using an ensemble size of 40, EnKF and DfEnKF perform similarly, and both are better than pure and hybrid En3DVar overall. Using 5% static error covariance, hybrid En3DVar outperforms pure En3DVar for most state variables but underperforms for hydrometeor variables, and the improvement (degradation) is most notable for water vapor mixing ratio qυ (snow mixing ratio qs). Overall, EnKF/DfEnKF performs the best, 3DVar performs the worst, and static covariance only helps slightly via hybrid En3DVar.

scholarly journals Progress toward the Application of a Localized Particle Filter for Numerical Weather Prediction

2019 ◽  
Vol 147 (4) ◽  
pp. 1107-1126 ◽  
Author(s):  
Jonathan Poterjoy ◽  
Louis Wicker ◽  
Mark Buehner
Keyword(s):  
Data Assimilation ◽  
Particle Filter ◽  
Numerical Weather Prediction ◽  
Large Scale ◽  
Sequential Monte Carlo ◽  
Prediction Models ◽  
Weather Prediction ◽  
Coupled Systems ◽  
State Variables ◽  
Numerical Weather

Abstract A series of papers published recently by the first author introduce a nonlinear filter that operates effectively as a data assimilation method for large-scale geophysical applications. The method uses sequential Monte Carlo techniques adopted by particle filters, which make no parametric assumptions for the underlying prior and posterior error distributions. The filter also treats the underlying dynamical system as a set of loosely coupled systems to effectively localize the effect observations have on posterior state estimates. This property greatly reduces the number of particles—or ensemble members—required for its implementation. For these reasons, the method is called the local particle filter. The current manuscript summarizes algorithmic advances made to the local particle filter following recent tests performed over a hierarchy of dynamical systems. The revised filter uses modified vector weight calculations and probability mapping techniques from earlier studies, and new strategies for improving filter stability in situations where state variables are observed infrequently with very accurate measurements. Numerical experiments performed on low-dimensional data assimilation problems provide evidence that supports the theoretical benefits of the new improvements. As a proof of concept, the revised particle filter is also tested on a high-dimensional application from a real-time weather forecasting system at the NOAA/National Severe Storms Laboratory (NSSL). The proposed changes have large implications for researchers applying the local particle filter for real applications, such as data assimilation in numerical weather prediction models.

scholarly journals Scale-Dependent Representation of the Information Content of Observations in the Global Ensemble Kalman Filter Data Assimilation

2016 ◽  
Vol 144 (8) ◽  
pp. 2927-2945
Author(s):  
Nedjeljka Žagar ◽  
Jeffrey Anderson ◽  
Nancy Collins ◽  
Timothy Hoar ◽  
Kevin Raeder ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Information Content ◽  
Ensemble Kalman Filter ◽  
Large Scale ◽  
Variance Reduction ◽  
Weather Prediction ◽  
Current Data ◽  
Model Framework ◽  
The Tropics

Abstract Global data assimilation systems for numerical weather prediction (NWP) are characterized by significant uncertainties in tropical analysis fields. Furthermore, the largest spread of global ensemble forecasts in the short range on all scales is in the tropics. The presented results suggest that these properties hold even in the perfect-model framework and the ensemble Kalman filter data assimilation with a globally homogeneous network of wind and temperature profiles. The reasons for this are discussed by using the modal analysis, which provides information about the scale dependency of analysis and forecast uncertainties and information about the efficiency of data assimilation to reduce the prior uncertainties in the balanced and inertio-gravity dynamics. The scale-dependent representation of variance reduction of the prior ensemble by the data assimilation shows that the peak efficiency of data assimilation is on the synoptic scales in the midlatitudes that are associated with quasigeostrophic dynamics. In contrast, the variance associated with the inertia–gravity modes is less successfully reduced on all scales. A smaller information content of observations on planetary scales with respect to the synoptic scales is discussed in relation to the large-scale tropical uncertainties that current data assimilation methodologies do not address successfully. In addition, it is shown that a smaller reduction of the large-scale uncertainties in the prior state for NWP in the tropics than in the midlatitudes is influenced by the applied radius for the covariance localization.

scholarly journals Improvements in Forecasting Intense Rainfall: Results from the FRANC (Forecasting Rainfall Exploiting New Data Assimilation Techniques and Novel Observations of Convection) Project

Atmosphere ◽  
2019 ◽  
Vol 10 (3) ◽  
pp. 125 ◽  
Author(s):  
Sarah Dance ◽  
Susan Ballard ◽  
Ross Bannister ◽  
Peter Clark ◽  
Hannah Cloke ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Doppler Radar ◽  
Weather Prediction ◽  
Observation Error ◽  
Error Statistics ◽  
Convective Rainfall ◽  
Background Error ◽  
Spatially Correlated ◽  
Operational Forecasts ◽  
The Impact

The FRANC project (Forecasting Rainfall exploiting new data Assimilation techniques and Novel observations of Convection) has researched improvements in numerical weather prediction of convective rainfall via the reduction of initial condition uncertainty. This article provides an overview of the project’s achievements. We highlight new radar techniques: correcting for attenuation of the radar return; correction for beams that are over 90% blocked by trees or towers close to the radar; and direct assimilation of radar reflectivity and refractivity. We discuss the treatment of uncertainty in data assimilation: new methods for estimation of observation uncertainties with novel applications to Doppler radar winds, Atmospheric Motion Vectors, and satellite radiances; a new algorithm for implementation of spatially-correlated observation error statistics in operational data assimilation; and innovative treatment of moist processes in the background error covariance model. We present results indicating a link between the spatial predictability of convection and convective regimes, with potential to allow improved forecast interpretation. The research was carried out as a partnership between University researchers and the Met Office (UK). We discuss the benefits of this approach and the impact of our research, which has helped to improve operational forecasts for convective rainfall events.

E4DVar: Coupling an Ensemble Kalman Filter with Four-Dimensional Variational Data Assimilation in a Limited-Area Weather Prediction Model

2012 ◽  
Vol 140 (2) ◽  
pp. 587-600 ◽  
Author(s):  
Meng Zhang ◽  
Fuqing Zhang
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Prediction Model ◽  
Ensemble Kalman Filter ◽  
Weather Prediction ◽  
Coupled System ◽  
Limited Area ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Hybrid Data Assimilation

A hybrid data assimilation approach that couples the ensemble Kalman filter (EnKF) and four-dimensional variational (4DVar) methods is implemented for the first time in a limited-area weather prediction model. In this coupled system, denoted E4DVar, the EnKF and 4DVar systems run in parallel while feeding into each other. The multivariate, flow-dependent background error covariance estimated from the EnKF ensemble is used in the 4DVar minimization and the ensemble mean in the EnKF analysis is replaced by the 4DVar analysis, while updating the analysis perturbations for the next cycle of ensemble forecasts with the EnKF. Therefore, the E4DVar can obtain flow-dependent information from both the explicit covariance matrix derived from ensemble forecasts, as well as implicitly from the 4DVar trajectory. The performance of an E4DVar system is compared with the uncoupled 4DVar and EnKF for a limited-area model by assimilating various conventional observations over the contiguous United States for June 2003. After verifying the forecasts from each analysis against standard sounding observations, it is found that the E4DVar substantially outperforms both the EnKF and 4DVar during this active summer month, which featured several episodes of severe convective weather. On average, the forecasts produced from E4DVar analyses have considerably smaller errors than both of the stand-alone EnKF and 4DVar systems for forecast lead times up to 60 h.

Sign in / Sign up

Export Citation Format

Share Document