A Probabilistic Framework for Agricultural Drought Forecasting Using the Ensemble Data Assimilation and Bayesian Multivariate Modeling

Abstract A new ensemble-based data assimilation method, named the maximum likelihood ensemble filter (MLEF), is presented. The analysis solution maximizes the likelihood of the posterior probability distribution, obtained by minimization of a cost function that depends on a general nonlinear observation operator. The MLEF belongs to the class of deterministic ensemble filters, since no perturbed observations are employed. As in variational and ensemble data assimilation methods, the cost function is derived using a Gaussian probability density function framework. Like other ensemble data assimilation algorithms, the MLEF produces an estimate of the analysis uncertainty (e.g., analysis error covariance). In addition to the common use of ensembles in calculation of the forecast error covariance, the ensembles in MLEF are exploited to efficiently calculate the Hessian preconditioning and the gradient of the cost function. A sufficient number of iterative minimization steps is 2–3, because of superior Hessian preconditioning. The MLEF method is well suited for use with highly nonlinear observation operators, for a small additional computational cost of minimization. The consistent treatment of nonlinear observation operators through optimization is an advantage of the MLEF over other ensemble data assimilation algorithms. The cost of MLEF is comparable to the cost of existing ensemble Kalman filter algorithms. The method is directly applicable to most complex forecast models and observation operators. In this paper, the MLEF method is applied to data assimilation with the one-dimensional Korteweg–de Vries–Burgers equation. The tested observation operator is quadratic, in order to make the assimilation problem more challenging. The results illustrate the stability of the MLEF performance, as well as the benefit of the cost function minimization. The improvement is noted in terms of the rms error, as well as the analysis error covariance. The statistics of innovation vectors (observation minus forecast) also indicate a stable performance of the MLEF algorithm. Additional experiments suggest the amplified benefit of targeted observations in ensemble data assimilation.

Download Full-text

An adaptive covariance relaxation method for ensemble data assimilation

Quarterly Journal of the Royal Meteorological Society ◽

10.1002/qj.2576 ◽

2015 ◽

Vol 141 (692) ◽

pp. 2898-2906 ◽

Cited By ~ 24

Author(s):

Yue Ying ◽

Fuqing Zhang

Keyword(s):

Data Assimilation ◽

Relaxation Method ◽

Ensemble Data Assimilation ◽

Ensemble Data

Download Full-text

Convection-Permitting Forecasts Initialized with Continuously Cycling Limited-Area 3DVAR, Ensemble Kalman Filter, and “Hybrid” Variational–Ensemble Data Assimilation Systems

Monthly Weather Review ◽

10.1175/mwr-d-13-00100.1 ◽

2014 ◽

Vol 142 (2) ◽

pp. 716-738 ◽

Cited By ~ 49

Author(s):

Craig S. Schwartz ◽

Zhiquan Liu

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Three Dimensional ◽

Skill Score ◽

Limited Area ◽

Computational Domain ◽

Ensemble Data Assimilation ◽

Ensemble Data ◽

Precipitation Characteristics ◽

High Precipitation

Abstract Analyses with 20-km horizontal grid spacing were produced from parallel continuously cycling three-dimensional variational (3DVAR), ensemble square root Kalman filter (EnSRF), and “hybrid” variational–ensemble data assimilation (DA) systems between 0000 UTC 6 May and 0000 UTC 21 June 2011 over a domain spanning the contiguous United States. Beginning 9 May, the 0000 UTC analyses initialized 36-h Weather Research and Forecasting Model (WRF) forecasts containing a large convection-permitting 4-km nest. These 4-km 3DVAR-, EnSRF-, and hybrid-initialized forecasts were compared to benchmark WRF forecasts initialized by interpolating 0000 UTC Global Forecast System (GFS) analyses onto the computational domain. While important differences regarding mean state characteristics of the 20-km DA systems were noted, verification efforts focused on the 4-km precipitation forecasts. The 3DVAR-, hybrid-, and EnSRF-initialized 4-km precipitation forecasts performed similarly regarding general precipitation characteristics, such as timing of the diurnal cycle, and all three forecast sets had high precipitation biases at heavier rainfall rates. However, meaningful differences emerged regarding precipitation placement as quantified by the fractions skill score. For most forecast hours, the hybrid-initialized 4-km precipitation forecasts were better than the EnSRF-, 3DVAR-, and GFS-initialized forecasts, and the improvement was often statistically significant at the 95th percentile. These results demonstrate the potential of limited-area continuously cycling hybrid DA configurations and suggest additional hybrid development is warranted.

Download Full-text

Southern High-Latitude Ensemble Data Assimilation in the Antarctic Mesoscale Prediction System

Monthly Weather Review ◽

10.1175/mwr3042.1 ◽

2005 ◽

Vol 133 (12) ◽

pp. 3431-3449 ◽

Cited By ~ 43

Author(s):

D. M. Barker

Keyword(s):

Data Assimilation ◽

Sampling Error ◽

Forecast Error ◽

Weather Prediction ◽

Forecast Errors ◽

Prediction System ◽

Spatial Covariance ◽

Ensemble Data Assimilation ◽

Ensemble Data ◽

The Antarctic

Abstract Ensemble data assimilation systems incorporate observations into numerical models via solution of the Kalman filter update equations, and estimates of forecast error covariances derived from ensembles of model integrations. In this paper, a particular algorithm, the ensemble square root filter (EnSRF), is tested in a limited-area, polar numerical weather prediction (NWP) model: the Antarctic Mesoscale Prediction System (AMPS). For application in the real-time AMPS, the number of model integrations that can be run to provide forecast error covariances is limited, resulting in an ensemble sampling error that degrades the analysis fit to observations. In this work, multivariate, climatologically plausible forecast error covariances are specified via averaged forecast difference statistics. Ensemble representations of the “true” forecast errors, created using randomized control variables of the fifth-generation Pennsylvania State University–National Center for Atmospheric Research (PSU–NCAR) Mesoscale Model (MM5) three-dimensional variational (3DVAR) data assimilation system, are then used to assess the dependence of sampling error on ensemble size, data density, and localization of covariances using simulated observation networks. Results highlight the detrimental impact of ensemble sampling error on the analysis increment structure of correlated, but unobserved fields—an issue not addressed by the spatial covariance localization techniques used to date. A 12-hourly cycling EnSRF/AMPS assimilation/forecast system is tested for a two-week period in December 2002 using real, conventional (surface, rawinsonde, satellite retrieval) observations. The dependence of forecast scores on methods used to maintain ensemble spread and the inclusion of perturbations to lateral boundary conditions are studied.

Download Full-text

A fault-tolerant HPC scheduler extension for large and operational ensemble data assimilation: Application to the Red Sea

Journal of Computational Science ◽

10.1016/j.jocs.2018.04.018 ◽

2018 ◽

Vol 27 ◽

pp. 46-56 ◽

Cited By ~ 1

Author(s):

Habib Toye ◽

Samuel Kortas ◽

Peng Zhan ◽

Ibrahim Hoteit

Keyword(s):

Data Assimilation ◽

Red Sea ◽

Fault Tolerant ◽

Ensemble Data Assimilation ◽

Ensemble Data

Download Full-text

Ensemble data assimilation in the presence of cloud

Computers & Fluids ◽

10.1016/j.compfluid.2011.01.033 ◽

2011 ◽

Vol 46 (1) ◽

pp. 493-497 ◽

Cited By ~ 1

Author(s):

S. Vetra-Carvalho ◽

S. Migliorini ◽

N.K. Nichols

Keyword(s):

Data Assimilation ◽

Ensemble Data Assimilation ◽

Ensemble Data

Download Full-text

Precipitation Ensemble Data Assimilation in NWP Models

Advances in Global Change Research - Satellite Precipitation Measurement ◽

10.1007/978-3-030-35798-6_25 ◽

2020 ◽

pp. 983-991

Author(s):

Takemasa Miyoshi ◽

Shunji Kotsuki ◽

Koji Terasaki ◽

Shigenori Otsuka ◽

Guo-Yuan Lien ◽

...

Keyword(s):

Data Assimilation ◽

Ensemble Data Assimilation ◽

Ensemble Data

Download Full-text

Technical note: Sequential ensemble data assimilation in convergent and divergent systems

10.5194/hess-2020-668 ◽

2021 ◽

Author(s):

Hannes Helmut Bauser ◽

Daniel Berg ◽

Kurt Roth

Keyword(s):

Data Assimilation ◽

Water Movement ◽

Measurement Data ◽

Technical Note ◽

Richards Equation ◽

Model Errors ◽

Parameter Uncertainties ◽

Ensemble Data Assimilation ◽

Ensemble Data ◽

Abstract Data

Abstract. Data assimilation methods are used throughout the geosciences to combine information from uncertain models and uncertain measurement data. However, the characteristics of geophysical systems differ and may be distinguished between divergent and convergent systems. In divergent systems initially nearby states will drift apart, while they will coalesce in convergent systems. This difference has implications for the application of sequential ensemble data assimilation methods. This study explores these implications on two exemplary systems: the divergent Lorenz-96 model and the convergent description of soil water movement by the Richards equation. The results show that sequential ensemble data assimilation methods require a sufficient divergent component. This makes the transfer of the methods from divergent to convergent systems challenging. We demonstrate through a set of case studies that it is imperative to represent model errors adequately and incorporate parameter uncertainties in ensemble data assimilation in convergent systems.

Download Full-text