Robust Data Assimilation in River Flow and Stage Estimation Based on Multiple Imputation Particle Filter

Particle filter has received increasing attention in data assimilation for estimating model states and parameters in cases of non-linear and non-Gaussian dynamic processes. Various modifications of the original particle filter have been suggested in the literature, including integrating particle filter with Markov Chain Monte Carlo (PF-MCMC) and, later, using genetic algorithm evolutionary operators as part of the state updating process. In this work, a modified genetic-based PF-MCMC approach for estimating the states and parameters simultaneously and without assuming Gaussian distribution for priors is presented. The method was tested on two simulation examples on the basis of the crop model AquaCrop-OS. In the first example, the method was compared to a PF-MCMC method in which states and parameters are updated sequentially and genetic operators are used only for state adjustments. The influence of ensemble size, measurement noise, and mutation and crossover parameters were also investigated. Accurate and stable estimations of the model states were obtained in all cases. Parameter estimation was more challenging than state estimation and not all parameters converged to their true value, especially when the parameter value had little influence on the measured variables. Overall, the proposed method showed more accurate and consistent parameter estimation than the PF-MCMC with sequential estimation, which showed highly conservative behavior. The superiority of the proposed method was more pronounced when the ensemble included a large number of particles and the measurement noise was low.

Download Full-text

CrocO_v1.0 : a Particle Filter to assimilate snowpack observations in a spatialised framework

10.5194/gmd-2020-130 ◽

2020 ◽

Author(s):

Bertrand Cluzet ◽

Matthieu Lafaysse ◽

Emmanuel Cosme ◽

Clément Albergel ◽

Louis-François Meunier ◽

...

Keyword(s):

Data Assimilation ◽

Particle Filter ◽

Near Infrared ◽

Snow Water Equivalent ◽

Sources Of Information ◽

Average Improvement ◽

Modelling Uncertainties ◽

Snow Water ◽

Spatio Temporal ◽

Snowpack Properties

Abstract. Monitoring the evolution of the snowpack properties in mountainous areas is crucial for avalanche hazard forecasting and water resources management. In-situ and remotely sensed observations provide precious information on the snowpack but usually offer a limited spatio-temporal coverage of bulk or surface variables only. In particular, visible-near infrared (VIS-NIR) reflectance observations can inform on the snowpack surface properties but are limited by terrain shading and clouds. Snowpack modelling enables to estimate any physical variable, virtually anywhere, but is affected by large errors and uncertainties. Data assimilation offers a way to combine both sources of information, and to propagate information from observed areas to non observed areas. Here, we present CrocO, (Crocus-Observations) an ensemble data assimilation system able to ingest any snowpack observation (applied as a first step to the height of snow (HS) and VIS-NIR reflectances) in a spatialised geometry. CrocO uses an ensemble of snowpack simulations to represent modelling uncertainties, and a Particle Filter (PF) to reduce them. The PF is known to collapse when assimilating a too large number of observations. Two variants of the PF were specifically implemented to ensure that observations information is propagated in space while tackling this issue. The global algorithm ingests all available observations with an iterative inflation of observation errors, while the klocal algorithm is a localised approach performing a selection of the observations to assimilate based on background correlation patterns. Experiments are carried out in a twin experiment setup, with synthetic observations of HS and VIS-NIR reflectances available in only a 1/6th of the simulation domain. Results show that compared against runs without assimilation, analyses exhibit an average improvement of snow water equivalent Continuous Rank Probability Score (CRPS) of 60 % when assimilating HS with a 40-member ensemble, and an average 20 % CRPS improvement when assimilating reflectance with a 160-member ensemble. Significant improvements are also obtained outside the observation domain. These promising results open a way for the assimilation of real observations of reflectance, or of any snowpack observations in a spatialised context.

Download Full-text

Progress toward the Application of a Localized Particle Filter for Numerical Weather Prediction

Monthly Weather Review ◽

10.1175/mwr-d-17-0344.1 ◽

2019 ◽

Vol 147 (4) ◽

pp. 1107-1126 ◽

Cited By ~ 9

Author(s):

Jonathan Poterjoy ◽

Louis Wicker ◽

Mark Buehner

Keyword(s):

Data Assimilation ◽

Particle Filter ◽

Numerical Weather Prediction ◽

Large Scale ◽

Sequential Monte Carlo ◽

Prediction Models ◽

Weather Prediction ◽

Coupled Systems ◽

State Variables ◽

Numerical Weather

Abstract A series of papers published recently by the first author introduce a nonlinear filter that operates effectively as a data assimilation method for large-scale geophysical applications. The method uses sequential Monte Carlo techniques adopted by particle filters, which make no parametric assumptions for the underlying prior and posterior error distributions. The filter also treats the underlying dynamical system as a set of loosely coupled systems to effectively localize the effect observations have on posterior state estimates. This property greatly reduces the number of particles—or ensemble members—required for its implementation. For these reasons, the method is called the local particle filter. The current manuscript summarizes algorithmic advances made to the local particle filter following recent tests performed over a hierarchy of dynamical systems. The revised filter uses modified vector weight calculations and probability mapping techniques from earlier studies, and new strategies for improving filter stability in situations where state variables are observed infrequently with very accurate measurements. Numerical experiments performed on low-dimensional data assimilation problems provide evidence that supports the theoretical benefits of the new improvements. As a proof of concept, the revised particle filter is also tested on a high-dimensional application from a real-time weather forecasting system at the NOAA/National Severe Storms Laboratory (NSSL). The proposed changes have large implications for researchers applying the local particle filter for real applications, such as data assimilation in numerical weather prediction models.

Download Full-text

A particle filter scheme for multivariate data assimilation into a point-scale snowpack model in an Alpine environment

The Cryosphere ◽

10.5194/tc-12-2287-2018 ◽

2018 ◽

Vol 12 (7) ◽

pp. 2287-2306 ◽

Cited By ~ 8

Author(s):

Gaia Piazzi ◽

Guillaume Thirel ◽

Lorenzo Campo ◽

Simone Gabellani

Keyword(s):

Data Assimilation ◽

Particle Filter ◽

Multivariate Data ◽

Model Parameters ◽

Parameter Perturbation ◽

Point Scale ◽

Meteorological Forcing ◽

System Sensitivity ◽

Filter Performance ◽

The Impact

Abstract. The accuracy of hydrological predictions in snow-dominated regions deeply depends on the quality of the snowpack simulations, with dynamics that strongly affect the local hydrological regime, especially during the melting period. With the aim of reducing the modelling uncertainty, data assimilation techniques are increasingly being implemented for operational purposes. This study aims to investigate the performance of a multivariate sequential importance resampling – particle filter scheme, designed to jointly assimilate several ground-based snow observations. The system, which relies on a multilayer energy-balance snow model, has been tested at three Alpine sites: Col de Porte (France), Torgnon (Italy), and Weissfluhjoch (Switzerland). The implementation of a multivariate data assimilation scheme faces several challenging issues, which are here addressed and extensively discussed: (1) the effectiveness of the perturbation of the meteorological forcing data in preventing the sample impoverishment; (2) the impact of the parameter perturbation on the filter updating of the snowpack state; the system sensitivity to (3) the frequency of the assimilated observations, and (4) the ensemble size.The perturbation of the meteorological forcing data generally turns out to be insufficient for preventing the sample impoverishment of the particle sample, which is highly limited when jointly perturbating key model parameters. However, the parameter perturbation sharpens the system sensitivity to the frequency of the assimilated observations, which can be successfully relaxed by introducing indirectly estimated information on snow-mass-related variables. The ensemble size is found not to greatly impact the filter performance in this point-scale application.

Download Full-text

Application of ensemble transform data assimilation methods for parameter estimation in reservoir modeling

Nonlinear Processes in Geophysics ◽

10.5194/npg-25-731-2018 ◽

2018 ◽

Vol 25 (4) ◽

pp. 731-746 ◽

Cited By ~ 1

Author(s):

Sangeetika Ruchi ◽

Svetlana Dubinkina

Keyword(s):

Parameter Estimation ◽

Kalman Filter ◽

Data Assimilation ◽

Particle Filter ◽

Reservoir Modeling ◽

Model Parameters ◽

Uncertain Parameters ◽

Initial Ensemble ◽

Leading Mode ◽

Ensemble Transform

Abstract. Over the years data assimilation methods have been developed to obtain estimations of uncertain model parameters by taking into account a few observations of a model state. The most reliable Markov chain Monte Carlo (MCMC) methods are computationally expensive. Sequential ensemble methods such as ensemble Kalman filters and particle filters provide a favorable alternative. However, ensemble Kalman filter has an assumption of Gaussianity. Ensemble transform particle filter does not have this assumption and has proven to be highly beneficial for an initial condition estimation and a small number of parameter estimations in chaotic dynamical systems with non-Gaussian distributions. In this paper we employ ensemble transform particle filter (ETPF) and ensemble transform Kalman filter (ETKF) for parameter estimation in nonlinear problems with 1, 5, and 2500 uncertain parameters and compare them to importance sampling (IS). The large number of uncertain parameters is of particular interest for subsurface reservoir modeling as it allows us to parameterize permeability on the grid. We prove that the updated parameters obtained by ETPF lie within the range of an initial ensemble, which is not the case for ETKF. We examine the performance of ETPF and ETKF in a twin experiment setup, where observations of pressure are synthetically created based on the known values of parameters. For a small number of uncertain parameters (one and five) ETPF performs comparably to ETKF in terms of the mean estimation. For a large number of uncertain parameters (2500) ETKF is robust with respect to the initial ensemble, while ETPF is sensitive due to sampling error. Moreover, for the high-dimensional test problem ETPF gives an increase in the root mean square error after data assimilation is performed. This is resolved by applying distance-based localization, which however deteriorates a posterior estimation of the leading mode by largely increasing the variance due to a combination of less varying localized weights, not keeping the imposed bounds on the modes via the Karhunen–Loeve expansion, and the main variability explained by the leading mode. A possible remedy is instead of applying localization to use only leading modes that are well estimated by ETPF, which demands knowledge of which mode to truncate.

Download Full-text

Review article: Comparison of local particle filters and new implementations

Nonlinear Processes in Geophysics ◽

10.5194/npg-25-765-2018 ◽

2018 ◽

Vol 25 (4) ◽

pp. 765-807 ◽

Cited By ~ 10

Author(s):

Alban Farchi ◽

Marc Bocquet

Keyword(s):

Data Assimilation ◽

Particle Filter ◽

Particle Filtering ◽

Transport Theory ◽

Optimal Transport ◽

Curse Of Dimensionality ◽

Local Analysis ◽

Problem Size ◽

Yield Data ◽

Non Gaussian

Abstract. Particle filtering is a generic weighted ensemble data assimilation method based on sequential importance sampling, suited for nonlinear and non-Gaussian filtering problems. Unless the number of ensemble members scales exponentially with the problem size, particle filter (PF) algorithms experience weight degeneracy. This phenomenon is a manifestation of the curse of dimensionality that prevents the use of PF methods for high-dimensional data assimilation. The use of local analyses to counteract the curse of dimensionality was suggested early in the development of PF algorithms. However, implementing localisation in the PF is a challenge, because there is no simple and yet consistent way of gluing together locally updated particles across domains. In this article, we review the ideas related to localisation and the PF in the geosciences. We introduce a generic and theoretical classification of local particle filter (LPF) algorithms, with an emphasis on the advantages and drawbacks of each category. Alongside the classification, we suggest practical solutions to the difficulties of local particle filtering, which lead to new implementations and improvements in the design of LPF algorithms. The LPF algorithms are systematically tested and compared using twin experiments with the one-dimensional Lorenz 40-variables model and with a two-dimensional barotropic vorticity model. The results illustrate the advantages of using the optimal transport theory to design the local analysis. With reasonable ensemble sizes, the best LPF algorithms yield data assimilation scores comparable to those of typical ensemble Kalman filter algorithms, even for a mildly nonlinear system.

Download Full-text