scholarly journals Application of ensemble transform data assimilation methods for parameter estimation in nonlinear problems

2018 ◽  
Author(s):  
Sangeetika Ruchi ◽  
Svetlana Dubinkina
Keyword(s):  
Parameter Estimation ◽  
Data Assimilation ◽  
Nonlinear Problems ◽  
Model Parameters ◽  
Ensemble Size ◽  
Uncertain Model ◽  
Initial Ensemble ◽  
Non Gaussian ◽  
Posterior Estimation ◽  
Ensemble Transform

Abstract. Over the years data assimilation methods have been developed to obtain estimations of uncertain model parameters by taking into account a few observations of a model state. However, most of these computationally affordable methods have assumptions of Gaussianity, e.g. an Ensemble Kalman Filter. Ensemble Transform Particle Filter does not have the assumption of Gaussianity and has proven to be highly beneficial for an initial condition estimation and a small number of parameter estimation in chaotic dynamical systems with non-Gaussian distributions. In this paper we employ Ensemble Transform Particle Smoother (ETPS) and Ensemble Transform Kalman Smoother (ETKS) for parameter estimation in nonlinear problems with 1, 5, and 2500 uncertain parameters and compare them to importance sampling (IS). We prove that the updated parameters obtained by ETPS lie within the range of an initial ensemble, which is not the case for ETKS. We examine the performance of ETPS and ETKS in a twin experiment setup, where observations of pressure are synthetically created based on the know values of parameters. The numerical experiments demonstrate that the ETKS provides good estimations of the mean parameters but not of the posterior distributions and as the ensemble size increases the posterior does not improve. ETPS provides good approximations of the posterior and as the ensemble size increases the posterior converges to the posterior obtained by IS with a large ensemble. ETKS is very robust while ETPS is very sensitive with respect to the initial ensemble. An issue of an increase in the root mean square error after data assimilation is performed in ETPS for a high-dimensional test problem is resolved by applying distance-based localization, which however deteriorated the posterior estimation.

scholarly journals Application of ensemble transform data assimilation methods for parameter estimation in reservoir modeling

2018 ◽  
Vol 25 (4) ◽  
pp. 731-746 ◽  
Author(s):  
Sangeetika Ruchi ◽  
Svetlana Dubinkina
Keyword(s):  
Parameter Estimation ◽  
Kalman Filter ◽  
Data Assimilation ◽  
Particle Filter ◽  
Reservoir Modeling ◽  
Model Parameters ◽  
Uncertain Parameters ◽  
Initial Ensemble ◽  
Leading Mode ◽  
Ensemble Transform

Abstract. Over the years data assimilation methods have been developed to obtain estimations of uncertain model parameters by taking into account a few observations of a model state. The most reliable Markov chain Monte Carlo (MCMC) methods are computationally expensive. Sequential ensemble methods such as ensemble Kalman filters and particle filters provide a favorable alternative. However, ensemble Kalman filter has an assumption of Gaussianity. Ensemble transform particle filter does not have this assumption and has proven to be highly beneficial for an initial condition estimation and a small number of parameter estimations in chaotic dynamical systems with non-Gaussian distributions. In this paper we employ ensemble transform particle filter (ETPF) and ensemble transform Kalman filter (ETKF) for parameter estimation in nonlinear problems with 1, 5, and 2500 uncertain parameters and compare them to importance sampling (IS). The large number of uncertain parameters is of particular interest for subsurface reservoir modeling as it allows us to parameterize permeability on the grid. We prove that the updated parameters obtained by ETPF lie within the range of an initial ensemble, which is not the case for ETKF. We examine the performance of ETPF and ETKF in a twin experiment setup, where observations of pressure are synthetically created based on the known values of parameters. For a small number of uncertain parameters (one and five) ETPF performs comparably to ETKF in terms of the mean estimation. For a large number of uncertain parameters (2500) ETKF is robust with respect to the initial ensemble, while ETPF is sensitive due to sampling error. Moreover, for the high-dimensional test problem ETPF gives an increase in the root mean square error after data assimilation is performed. This is resolved by applying distance-based localization, which however deteriorates a posterior estimation of the leading mode by largely increasing the variance due to a combination of less varying localized weights, not keeping the imposed bounds on the modes via the Karhunen–Loeve expansion, and the main variability explained by the leading mode. A possible remedy is instead of applying localization to use only leading modes that are well estimated by ETPF, which demands knowledge of which mode to truncate.

scholarly journals Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization

2017 ◽  
Vol 145 (11) ◽  
pp. 4575-4592 ◽  
Author(s):  
Craig H. Bishop ◽  
Jeffrey S. Whitaker ◽  
Lili Lei
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Satellite Data ◽  
Localization Length ◽  
Model Space ◽  
Ensemble Size ◽  
Length Scales ◽  
Ensemble Transform Kalman Filter ◽  
Ensemble Transform ◽  
Expanded Ensemble

To ameliorate suboptimality in ensemble data assimilation, methods have been introduced that involve expanding the ensemble size. Such expansions can incorporate model space covariance localization and/or estimates of climatological or model error covariances. Model space covariance localization in the vertical overcomes problematic aspects of ensemble-based satellite data assimilation. In the case of the ensemble transform Kalman filter (ETKF), the expanded ensemble size associated with vertical covariance localization would also enable the simultaneous update of entire vertical columns of model variables from hyperspectral and multispectral satellite sounders. However, if the original formulation of the ETKF were applied to an expanded ensemble, it would produce an analysis ensemble that was the same size as the expanded forecast ensemble. This article describes a variation on the ETKF called the gain ETKF (GETKF) that takes advantage of covariances from the expanded ensemble, while producing an analysis ensemble that has the required size of the unexpanded forecast ensemble. The approach also yields an inflation factor that depends on the localization length scale that causes the GETKF to perform differently to an ensemble square root filter (EnSRF) using the same expanded ensemble. Experimentation described herein shows that the GETKF outperforms a range of alternative ETKF-based solutions to the aforementioned problems. In cycling data assimilation experiments with a newly developed storm-track version of the Lorenz-96 model, the GETKF analysis root-mean-square error (RMSE) matches the EnSRF RMSE at shorter than optimal localization length scales but is superior in that it yields smaller RMSEs for longer localization length scales.

scholarly journals Simultaneous state-parameter estimation of rainfall-induced landslide displacement using data assimilation

2019 ◽  
Author(s):  
Jing Wang ◽  
Guigen Nie ◽  
Shengjun Gao ◽  
Changhu Xue
Keyword(s):  
Parameter Estimation ◽  
Data Assimilation ◽  
Prediction Method ◽  
State Parameter ◽  
Joint Estimation ◽  
Model Parameters ◽  
Displacement Estimation ◽  
Landslide Stability ◽  
Landslide Displacement Prediction ◽  
Using Data

Abstract. Landslide displacement prediction has great practical engineering significance to landslide stability evaluation and early warning. The evolution of landslide is a complex dynamic process, applying classical prediction method will result in significant error. Data assimilation method offers a new way to merge multi-source data with the model. However, data assimilation is still deficient in the ability to meet the demand of dynamic landslide system. In this paper, simultaneous state-parameter estimation (SSPE) using particle filter-based data assimilation is applied to predict displacement of the landslide. Landslide SSPE assimilation strategy can make use of time-series displacements and hydrological information for the joint estimation of landslide displacement and model parameters, which can improve the performance considerably. We select Xishan Village, Sichuan province, China as experiment site to test SSPE assimilation strategy. Based on the comparison of actual monitoring data with prediction values, results strongly suggest the effectiveness and feasibility of SSPE assimilation strategy in short-term landslide displacement estimation.

scholarly journals Optimal solution error covariance in highly nonlinear problems of variational data assimilation

2012 ◽  
Vol 19 (2) ◽  
pp. 177-184 ◽  
Author(s):  
V. Shutyaev ◽  
I. Gejadze ◽  
G. J. M. Copeland ◽  
F.-X. Le Dimet
Keyword(s):  
Nonlinear Dynamics ◽  
Data Assimilation ◽  
Optimal Solution ◽  
Nonlinear Problems ◽  
Variational Data Assimilation ◽  
Model Parameters ◽  
Efficient Computation ◽  
Viscous Term ◽  
Highly Nonlinear ◽  
The Cost

Abstract. The problem of variational data assimilation (DA) for a nonlinear evolution model is formulated as an optimal control problem to find the initial condition, boundary conditions and/or model parameters. The input data contain observation and background errors, hence there is an error in the optimal solution. For mildly nonlinear dynamics, the covariance matrix of the optimal solution error can be approximated by the inverse Hessian of the cost function. For problems with strongly nonlinear dynamics, a new statistical method based on the computation of a sample of inverse Hessians is suggested. This method relies on the efficient computation of the inverse Hessian by means of iterative methods (Lanczos and quasi-Newton BFGS) with preconditioning. Numerical examples are presented for the model governed by the Burgers equation with a nonlinear viscous term.

scholarly journals A Second-Order Exact Ensemble Square Root Filter for Nonlinear Data Assimilation

2015 ◽  
Vol 143 (4) ◽  
pp. 1347-1367 ◽  
Author(s):  
Julian Tödter ◽  
Bodo Ahrens
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Second Order ◽  
Square Root ◽  
Computationally Efficient ◽  
Research Activities ◽  
Wide Range ◽  
Matrix Square Root ◽  
Non Gaussian ◽  
Ensemble Transform

Abstract The ensemble Kalman filter (EnKF) and its deterministic variants, mostly square root filters such as the ensemble transform Kalman filter (ETKF), represent a popular alternative to variational data assimilation schemes and are applied in a wide range of operational and research activities. Their forecast step employs an ensemble integration that fully respects the nonlinear nature of the analyzed system. In the analysis step, they implicitly assume the prior state and observation errors to be Gaussian. Consequently, in nonlinear systems, the analysis mean and covariance are biased, and these filters remain suboptimal. In contrast, the fully nonlinear, non-Gaussian particle filter (PF) only relies on Bayes’s theorem, which guarantees an exact asymptotic behavior, but because of the so-called curse of dimensionality it is exposed to weight collapse. Here, it is shown how to obtain a new analysis ensemble whose mean and covariance exactly match the Bayesian estimates. This is achieved by a deterministic matrix square root transformation of the forecast ensemble, and subsequently a suitable random rotation that significantly contributes to filter stability while preserving the required second-order statistics. The properties and performance of the proposed algorithm are further investigated via a set of experiments. They indicate that such a filter formulation can increase the analysis quality, even for relatively small ensemble sizes, compared to other ensemble filters in nonlinear, non-Gaussian scenarios. Localization enhances the potential applicability of this PF-inspired scheme in larger-dimensional systems. The proposed algorithm, which is fairly easy to implement and computationally efficient, is referred to as the nonlinear ensemble transform filter (NETF).

scholarly journals Assessment of a Nonlinear Ensemble Transform Filter for High-Dimensional Data Assimilation

2016 ◽  
Vol 144 (1) ◽  
pp. 409-427 ◽  
Author(s):  
Julian Tödter ◽  
Paul Kirchgessner ◽  
Lars Nerger ◽  
Bodo Ahrens
Keyword(s):  
Data Assimilation ◽  
Particle Filtering ◽  
General Circulation ◽  
Large Scale ◽  
Circulation Model ◽  
Second Order ◽  
High Dimensional ◽  
Deterministic System ◽  
Ensemble Size ◽  
Ensemble Transform

Abstract This work assesses the large-scale applicability of the recently proposed nonlinear ensemble transform filter (NETF) in data assimilation experiments with the NEMO ocean general circulation model. The new filter constitutes a second-order exact approximation to fully nonlinear particle filtering. Thus, it relaxes the Gaussian assumption contained in ensemble Kalman filters. The NETF applies an update step similar to the local ensemble transform Kalman filter (LETKF), which allows for efficient and simple implementation. Here, simulated observations are assimilated into a simplified ocean configuration that exhibits globally high-dimensional dynamics with a chaotic mesoscale flow. The model climatology is used to initialize an ensemble of 120 members. The number of observations in each local filter update is of the same order resulting from the use of a realistic oceanic observation scenario. Here, an importance sampling particle filter (PF) would require at least 106 members. Despite the relatively small ensemble size, the NETF remains stable and converges to the truth. In this setup, the NETF achieves at least the performance of the LETKF. However, it requires a longer spinup period because the algorithm only relies on the particle weights at the analysis time. These findings show that the NETF can successfully deal with a large-scale assimilation problem in which the local observation dimension is of the same order as the ensemble size. Thus, the second-order exact NETF does not suffer from the PF’s curse of dimensionality, even in a deterministic system.

scholarly journals Parameter Estimation of Partially Observed Turbulent Systems Using Conditional Gaussian Path-Wise Sampler

Computation ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. 91
Author(s):  
Ziheng Zhang ◽  
Nan Chen
Keyword(s):  
Parameter Estimation ◽  
Sampling Method ◽  
Data Augmentation ◽  
Iteration Algorithm ◽  
Model Parameters ◽  
Unknown Parameters ◽  
Partial Observations ◽  
Highly Nonlinear ◽  
Partially Observed ◽  
Non Gaussian

Parameter estimation of complex nonlinear turbulent dynamical systems using only partially observed time series is a challenging topic. The nonlinearity and partial observations often impede using closed analytic formulae to recover the model parameters. In this paper, an exact path-wise sampling method is developed, which is incorporated into a Bayesian Markov chain Monte Carlo (MCMC) algorithm in light of data augmentation to efficiently estimate the parameters in a rich class of nonlinear and non-Gaussian turbulent systems using partial observations. This path-wise sampling method exploits closed analytic formulae to sample the trajectories of the unobserved variables, which avoid the numerical errors in the general sampling approaches and significantly increase the overall parameter estimation efficiency. The unknown parameters and the missing trajectories are estimated in an alternating fashion in an adaptive MCMC iteration algorithm with rapid convergence. It is shown based on the noisy Lorenz 63 model and a stochastically coupled FitzHugh–Nagumo model that the new algorithm is very skillful in estimating the parameters in highly nonlinear turbulent models. The model with the estimated parameters succeeds in recovering the nonlinear and non-Gaussian features of the truth, including capturing the intermittency and extreme events, in both test examples.

scholarly journals Multi-mission satellite remote sensing data for improving land hydrological models via data assimilation

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
M. Khaki ◽  
H.-J. Hendricks Franssen ◽  
S. C. Han
Keyword(s):  
Remote Sensing ◽  
Parameter Estimation ◽  
Soil Moisture ◽  
Data Assimilation ◽  
Satellite Remote Sensing ◽  
Model Parameters ◽  
Parameter Uncertainties ◽  
Area Index ◽  
Mission Satellite ◽  
Model Predictions

Abstract Satellite remote sensing offers valuable tools to study Earth and hydrological processes and improve land surface models. This is essential to improve the quality of model predictions, which are affected by various factors such as erroneous input data, the uncertainty of model forcings, and parameter uncertainties. Abundant datasets from multi-mission satellite remote sensing during recent years have provided an opportunity to improve not only the model estimates but also model parameters through a parameter estimation process. This study utilises multiple datasets from satellite remote sensing including soil moisture from Soil Moisture and Ocean Salinity Mission and Advanced Microwave Scanning Radiometer Earth Observing System, terrestrial water storage from the Gravity Recovery And Climate Experiment, and leaf area index from Advanced Very-High-Resolution Radiometer to estimate model parameters. This is done using the recently proposed assimilation method, unsupervised weak constrained ensemble Kalman filter (UWCEnKF). UWCEnKF applies a dual scheme to separately update the state and parameters using two interactive EnKF filters followed by a water balance constraint enforcement. The performance of multivariate data assimilation is evaluated against various independent data over different time periods over two different basins including the Murray–Darling and Mississippi basins. Results indicate that simultaneous assimilation of multiple satellite products combined with parameter estimation strongly improves model predictions compared with single satellite products and/or state estimation alone. This improvement is achieved not only during the parameter estimation period ($$\sim $$ ∼  32% groundwater RMSE reduction and soil moisture correlation increase from $$\sim $$ ∼  0.66 to $$\sim $$ ∼  0.85) but also during the forecast period ($$\sim $$ ∼  14% groundwater RMSE reduction and soil moisture correlation increase from $$\sim $$ ∼  0.69 to $$\sim $$ ∼  0.78) due to the effective impacts of the approach on both state and parameters.

Sign in / Sign up

Export Citation Format

Share Document