scholarly journals Implicit Equal-Weights Variational Particle Smoother

Atmosphere ◽  
2020 ◽  
Vol 11 (4) ◽  
pp. 338
Author(s):  
Pinqiang Wang ◽  
Mengbin Zhu ◽  
Yan Chen ◽  
Weimin Zhang
Keyword(s):  
Particle Filter ◽  
Weather Prediction ◽  
Ensemble Methods ◽  
Great Success ◽  
Ensemble Spread ◽  
Ensemble Kalman Filters ◽  
Error Covariance ◽  
Filter Methods ◽  
Geophysical Application ◽  
Prediction Systems

Under the motivation of the great success of four-dimensional variational (4D-Var) data assimilation methods and the advantages of ensemble methods (e.g., Ensemble Kalman Filters and Particle Filters) in numerical weather prediction systems, we introduce the implicit equal-weights particle filter scheme in the weak constraint 4D-Var framework which avoids the filter degeneracy through implicit sampling in high-dimensional situations. The new variational particle smoother (varPS) method has been tested and explored using the Lorenz96 model with dimensions N x = 40 , N x = 100 , N x = 250 , and N x = 400 . The results show that the new varPS method does not suffer from the curse of dimensionality by construction and the root mean square error (RMSE) in the new varPS is comparable with the ensemble 4D-Var method. As a combination of the implicit equal-weights particle filter and weak constraint 4D-Var, the new method improves the RMSE compared with the implicit equal-weights particle filter and LETKF (local ensemble transformed Kalman filter) methods and enlarges the ensemble spread compared with ensemble 4D-Var scheme. To overcome the difficulty of the implicit equal-weights particle filter in real geophysical application, the posterior error covariance matrix is estimated using a limited ensemble and can be calculated in parallel. In general, this new varPS performs slightly better in ensemble quality (the balance between the RMSE and ensemble spread) than the ensemble 4D-Var and has the potential to be applied into real geophysical systems.

scholarly journals Ensemble prediction of floods – catchment non-linearity and forecast probabilities

2007 ◽  
Vol 7 (4) ◽  
pp. 431-444 ◽  
Author(s):  
J. Komma ◽  
C. Reszler ◽  
G. Blöschl ◽  
T. Haiden
Keyword(s):  
Prediction Models ◽  
Weather Prediction ◽  
Ensemble Methods ◽  
Rain Gauge ◽  
Ensemble Prediction ◽  
Lead Times ◽  
Forecast Errors ◽  
Ensemble Spread ◽  
Relative Operating Characteristic ◽  
Data Set

Abstract. Quantifying the uncertainty of flood forecasts by ensemble methods is becoming increasingly important for operational purposes. The aim of this paper is to examine how the ensemble distribution of precipitation forecasts propagates in the catchment system, and to interpret the flood forecast probabilities relative to the forecast errors. We use the 622 km2 Kamp catchment in Austria as an example where a comprehensive data set, including a 500 yr and a 1000 yr flood, is available. A spatially-distributed continuous rainfall-runoff model is used along with ensemble and deterministic precipitation forecasts that combine rain gauge data, radar data and the forecast fields of the ALADIN and ECMWF numerical weather prediction models. The analyses indicate that, for long lead times, the variability of the precipitation ensemble is amplified as it propagates through the catchment system as a result of non-linear catchment response. In contrast, for lead times shorter than the catchment lag time (e.g. 12 h and less), the variability of the precipitation ensemble is decreased as the forecasts are mainly controlled by observed upstream runoff and observed precipitation. Assuming that all ensemble members are equally likely, the statistical analyses for five flood events at the Kamp showed that the ensemble spread of the flood forecasts is always narrower than the distribution of the forecast errors. This is because the ensemble forecasts focus on the uncertainty in forecast precipitation as the dominant source of uncertainty, and other sources of uncertainty are not accounted for. However, a number of analyses, including Relative Operating Characteristic diagrams, indicate that the ensemble spread is a useful indicator to assess potential forecast errors for lead times larger than 12 h.

scholarly journals Concrete ensemble Kalman filters with rigorous catastrophic filter divergence

2015 ◽  
Vol 112 (34) ◽  
pp. 10589-10594 ◽  
Author(s):  
David Kelly ◽  
Andrew J. Majda ◽  
Xin T. Tong
Keyword(s):  
Theoretical Perspective ◽  
Ensemble Methods ◽  
Dynamical Property ◽  
Forecast Model ◽  
High Dimensional ◽  
Great Success ◽  
Large Set ◽  
Ensemble Kalman Filters ◽  
Nonlinear Dynamical ◽  
Filter Divergence

The ensemble Kalman filter and ensemble square root filters are data assimilation methods used to combine high-dimensional, nonlinear dynamical models with observed data. Ensemble methods are indispensable tools in science and engineering and have enjoyed great success in geophysical sciences, because they allow for computationally cheap low-ensemble-state approximation for extremely high-dimensional turbulent forecast models. From a theoretical perspective, the dynamical properties of these methods are poorly understood. One of the central mysteries is the numerical phenomenon known as catastrophic filter divergence, whereby ensemble-state estimates explode to machine infinity, despite the true state remaining in a bounded region. In this article we provide a breakthrough insight into the phenomenon, by introducing a simple and natural forecast model that transparently exhibits catastrophic filter divergence under all ensemble methods and a large set of initializations. For this model, catastrophic filter divergence is not an artifact of numerical instability, but rather a true dynamical property of the filter. The divergence is not only validated numerically but also proven rigorously. The model cleanly illustrates mechanisms that give rise to catastrophic divergence and confirms intuitive accounts of the phenomena given in past literature.

Limited-Area Ensemble-Based Data Assimilation

2011 ◽  
Vol 139 (7) ◽  
pp. 2025-2045 ◽  
Author(s):  
Zhiyong Meng ◽  
Fuqing Zhang
Keyword(s):  
Data Assimilation ◽  
Variational Methods ◽  
Prediction Models ◽  
Weather Prediction ◽  
Limited Area ◽  
Ensemble Kalman Filters ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Error Covariance ◽  
Numerical Weather Prediction Models

Abstract Ensemble-based data assimilation is a state estimation technique that uses short-term ensemble forecasts to estimate flow-dependent background error covariance and is best known by varying forms of ensemble Kalman filters (EnKFs). The EnKF has recently emerged as one of the primary alternatives to the variational data assimilation methods widely used in both global and limited-area numerical weather prediction models. In addition to comparing the EnKF with variational methods, this article reviews recent advances and challenges in the development and applications of the EnKF, including its hybrid with variational methods, in limited-area models that resolve weather systems from convective to meso- and regional scales.

scholarly journals Efficient Assimilation of Simulated Observations in a High-Dimensional Geophysical System Using a Localized Particle Filter

2016 ◽  
Vol 144 (5) ◽  
pp. 2007-2020 ◽  
Author(s):  
Jonathan Poterjoy ◽  
Jeffrey L. Anderson
Keyword(s):  
Data Assimilation ◽  
Particle Filter ◽  
Kalman Filters ◽  
General Circulation ◽  
Prediction Models ◽  
Weather Prediction ◽  
Circulation Model ◽  
High Dimensional ◽  
Ensemble Kalman Filters ◽  
Gaussian Filters

This study presents the first application of a localized particle filter (PF) for data assimilation in a high-dimensional geophysical model. Particle filters form Monte Carlo approximations of model probability densities conditioned on observations, while making no assumptions about the underlying error distribution. Unlike standard PFs, the local PF uses a localization function to reduce the influence of distant observations on state variables, which significantly decreases the number of particles required to maintain the filter’s stability. Because the local PF operates effectively using small numbers of particles, it provides a possible alternative to Gaussian filters, such as ensemble Kalman filters, for large geophysical models. In the current study, the local PF is compared with stochastic and deterministic ensemble Kalman filters using a simplified atmospheric general circulation model. The local PF is found to provide stable filtering results over yearlong data assimilation experiments using only 25 particles. The local PF also outperforms the Gaussian filters when observation networks include measurements that have non-Gaussian errors or relate nonlinearly to the model state, like remotely sensed data used frequently in atmospheric analyses. Results from this study encourage further testing of the local PF on more complex geophysical systems, such as weather prediction models.

scholarly journals The Impact of Assimilating Satellite Radiance Observations in the Copernicus European Regional Reanalysis (CERRA)

2021 ◽  
Vol 13 (3) ◽  
pp. 426
Author(s):  
Zheng Qi Wang ◽  
Roger Randriamampianina
Keyword(s):  
Weather Prediction ◽  
Deterministic System ◽  
Observation Error ◽  
Key Factors ◽  
Background Error ◽  
Current Configuration ◽  
Error Covariance ◽  
Regional Reanalysis ◽  
The Impact ◽  
European Regional

The assimilation of microwave and infrared (IR) radiance satellite observations within numerical weather prediction (NWP) models have been an important component in the effort of improving the accuracy of analysis and forecast. Such capabilities were implemented during the development of the high-resolution Copernicus European Regional Reanalysis (CERRA), funded by the Copernicus Climate Change Services (C3S). The CERRA system couples the deterministic system with the ensemble data assimilation to provide periodic updates of the background error covariance matrix. Several key factors for the assimilation of radiances were investigated, including appropriate use of variational bias correction (VARBC), surface-sensitive AMSU-A observations and observation error correlation. Twenty-one-day impact studies during the summer and winter seasons were conducted. Generally, the assimilation of radiances has a small impact on the analysis, while greater impacts are observed on short-range (12 and 24-h) forecasts with an error reduction of 1–2% for the mid and high troposphere. Although, the current configuration provided less accurate forecasts from 09 and 18 UTC analysis times. With the increased thinning distances and the rejection of IASI observation over land, the errors in the analyses and 3 h forecasts on geopotential height were reduced up to 2%.

Exploring trends for the H SAF ASCAT root-zone soil moisture data records

2021 ◽  
Author(s):  
David Fairbairn ◽  
Patricia de Rosnay ◽  
Peter Weston
Keyword(s):  
Soil Moisture ◽  
Root Zone ◽  
Weather Prediction ◽  
Climate Change Impacts ◽  
Soil Moisture Data ◽  
Negative Root ◽  
In Situ Observations ◽  
Prediction Systems ◽  
Water Scarce

<p>Environmental (e.g. floods, droughts) and weather prediction systems rely on an accurate representation of soil moisture (SM). The EUMETSAT H SAF aims to provide high quality satellite-based hydrological products, including SM.<br>ECMWF is producing ASCAT root zone SM for H SAF. The production relies on an Extended Kalman filter to retrieve root zone SM from surface SM satellite data. A 10 km sampling reanalysis product (1992-2020) forced by ERA5 atmospheric fields (H141/H142) is produced for H SAF, which assimilates ERS/SCAT (1992-2006) and ASCAT-A/B/C (2007-2020) derived surface SM. The root-zone SM performance is validated using sparse in situ observations globally and generally demonstrates a positive and consistent correlation over the period. A negative trend in root-zone SM is found during summer and autumn months over much of Europe during the period (1992-2020). This is consistent with expected climate change impacts and is particularly alarming over the water-scarce Mediterranean region. The recent hot and dry summer of 2019 and dry spring of 2020 are well captured by negative root-zone SM anomalies. Plans for the future H SAF data record products will be presented, including the assimilation of high-resolution EPS-SCA-derived soil moisture data.</p>

scholarly journals Background Error in WRF Model

2020 ◽  
Vol 16 ◽  
Keyword(s):  
Data Assimilation ◽  
Physical Properties ◽  
Covariance Matrix ◽  
Numerical Weather Prediction ◽  
Weather Prediction ◽  
Wrf Model ◽  
Background Error ◽  
Numerical Weather ◽  
Error Covariance ◽  
First Time

WRF model have been tuned and tested over Georgia’s territory for years. First time in Georgia theprocess of data assimilation in Numerical weather prediction is developing. This work presents how forecasterror statistics appear in the data assimilation problem through the background error covariance matrix – B, wherethe variances and correlations associated with model forecasts are estimated. Results of modeling of backgrounderror covariance matrix for control variables using WRF model over Georgia with desired domain configurationare discussed and presented. The modeling was implemented in two different 3DVAR systems (WRFDA andGSI) and results were checked by pseudo observation benchmark cases using also default global and regional BEmatrixes. The mathematical and physical properties of the covariances are also reviewed.

scholarly journals Progress toward the Application of a Localized Particle Filter for Numerical Weather Prediction

2019 ◽  
Vol 147 (4) ◽  
pp. 1107-1126 ◽  
Author(s):  
Jonathan Poterjoy ◽  
Louis Wicker ◽  
Mark Buehner
Keyword(s):  
Data Assimilation ◽  
Particle Filter ◽  
Numerical Weather Prediction ◽  
Large Scale ◽  
Sequential Monte Carlo ◽  
Prediction Models ◽  
Weather Prediction ◽  
Coupled Systems ◽  
State Variables ◽  
Numerical Weather

Abstract A series of papers published recently by the first author introduce a nonlinear filter that operates effectively as a data assimilation method for large-scale geophysical applications. The method uses sequential Monte Carlo techniques adopted by particle filters, which make no parametric assumptions for the underlying prior and posterior error distributions. The filter also treats the underlying dynamical system as a set of loosely coupled systems to effectively localize the effect observations have on posterior state estimates. This property greatly reduces the number of particles—or ensemble members—required for its implementation. For these reasons, the method is called the local particle filter. The current manuscript summarizes algorithmic advances made to the local particle filter following recent tests performed over a hierarchy of dynamical systems. The revised filter uses modified vector weight calculations and probability mapping techniques from earlier studies, and new strategies for improving filter stability in situations where state variables are observed infrequently with very accurate measurements. Numerical experiments performed on low-dimensional data assimilation problems provide evidence that supports the theoretical benefits of the new improvements. As a proof of concept, the revised particle filter is also tested on a high-dimensional application from a real-time weather forecasting system at the NOAA/National Severe Storms Laboratory (NSSL). The proposed changes have large implications for researchers applying the local particle filter for real applications, such as data assimilation in numerical weather prediction models.

scholarly journals Non-Gaussian Probability Densities of Convection Initiation and Development Investigated Using a Particle Filter with a Storm-Scale Numerical Weather Prediction Model

2019 ◽  
Vol 148 (1) ◽  
pp. 3-20 ◽  
Author(s):  
Takuya Kawabata ◽  
Genta Ueno
Keyword(s):  
Prediction Model ◽  
Particle Filter ◽  
Goodness Of Fit ◽  
Weather Prediction ◽  
Potential Temperature ◽  
Error Estimator ◽  
Observation Error ◽  
Adaptive Observation ◽  
Non Gaussian ◽  
Convection Initiation

Abstract Non-Gaussian probability density functions (PDFs) in convection initiation (CI) and development were investigated using a particle filter with a storm-scale numerical prediction model and an adaptive observation error estimator (NHM-RPF). An observing system simulation experiment (OSSE) was conducted with a 90-min assimilation period and 1000 particles at a 2-km grid spacing. Pseudosurface observations of potential temperature (PT), winds, water vapor (QV), and pseudoradar observations of rainwater (QR) in the lower troposphere were created in a nature run that simulated a well-developed cumulonimbus. The results of the OSSE (PF) show a significant improvement in comparison to ensemble simulations without any observations. The Gaussianity of the PDFs for PF in the CI area was evaluated using the Bayesian information criterion to compare goodness-of-fit of Gaussian, two-Gaussian mixture, and histogram models. The PDFs are strongly non-Gaussian when NHM-RPF produces diverse particles over the CI period. The non-Gaussian PDF of the updraft is followed by the upper-bounded PDF of the relative humidity, which produces non-Gaussian PDFs of QV and PT. The PDFs of the cloud water and QR are strongly non-Gaussian throughout the experimental period. We conclude that the non-Gaussianity of the CI originated from the non-Gaussianity of the updraft. In addition, we show that the adaptive observation error estimator significantly contributes to the stability of PF and the robustness to many observations.

Sign in / Sign up

Export Citation Format

Share Document