scholarly journals A Nonlinear Rank Regression Method for Ensemble Kalman Filter Data Assimilation

2019 ◽  
Vol 147 (8) ◽  
pp. 2847-2860 ◽  
Author(s):  
Jeffrey L. Anderson
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
The State ◽  
Single Observation ◽  
Rank Regression ◽  
State Variable ◽  
Rank Histogram ◽  
Observation Space ◽  
The Impact ◽  
Standard Regression

Abstract It is possible to describe many variants of ensemble Kalman filters without loss of generality as the impact of a single observation on a single state variable. For most ensemble algorithms commonly applied to Earth system models, the computation of increments for the observation variable ensemble can be treated as a separate step from computing increments for the state variable ensemble. The state variable increments are normally computed from the observation increments by linear regression using the prior bivariate ensemble of the state and observation variable. Here, a new method that replaces the standard regression with a regression using the bivariate rank statistics is described. This rank regression is expected to be most effective when the relation between a state variable and an observation is nonlinear. The performance of standard versus rank regression is compared for both linear and nonlinear forward operators (also known as observation operators) using a low-order model. Rank regression in combination with a rank histogram filter in observation space produces better analyses than standard regression for cases with nonlinear forward operators and relatively large analysis error. Standard regression, in combination with either a rank histogram filter or an ensemble Kalman filter in observation space, produces the best results in other situations.

Evaluating Ensemble Kalman Filter Analyses of Severe Hailstorms on 8 May 2017 in Colorado: Effects of State Variable Updating and Multimoment Microphysics Schemes on State Variable Cross Covariances

2020 ◽  
Vol 148 (6) ◽  
pp. 2365-2389
Author(s):  
Jonathan Labriola ◽  
Nathan Snook ◽  
Youngsun Jung ◽  
Ming Xue
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
Forecast Error ◽  
Variable Cross ◽  
Background Error ◽  
Mixing Ratios ◽  
State Variable ◽  
Error Covariance ◽  
Thermodynamic Variables ◽  
The Impact

Abstract Ensemble Kalman filter (EnKF) analyses of the storms associated with the 8 May 2017 Colorado severe hail event using either the Milbrandt and Yau (MY) or the NSSL double-moment bulk microphysics scheme in the forecast model are evaluated. With each scheme, two experiments are conducted in which the reflectivity (Z) observations update in addition to dynamic and thermodynamic variables: 1) only the hydrometeor mixing ratios or 2) all microphysical variables. With fewer microphysical variables directly constrained by the Z observations, only updating hydrometeor mixing ratios causes the forecast error covariance structure to become unreliable, and results in larger errors in the analysis. Experiments that update all microphysical variables produce analyses with the lowest Z root-mean-square innovations; however, comparing the estimated hail size against hydrometeor classification algorithm output suggests that further constraint from observations is needed to more accurately estimate surface hail size. Ensemble correlation analyses are performed to determine the impact of hail growth assumptions in the MY and NSSL schemes on the forecast error covariance between microphysical and thermodynamic variables. In the MY scheme, Z is negatively correlated with updraft intensity because the strong updrafts produce abundant small hail aloft. The NSSL scheme predicts the growth of large hail aloft; consequently, Z is positively correlated with storm updraft intensity and hail state variables. Hail production processes are also shown to alter the background error covariance for liquid and frozen hydrometeor species. Results in this study suggest that EnKF analyses are sensitive to the choice of MP scheme (e.g., the treatment of hail growth processes).

scholarly journals Intercomparison of Variational Data Assimilation and the Ensemble Kalman Filter for Global Deterministic NWP. Part I: Description and Single-Observation Experiments

2010 ◽  
Vol 138 (5) ◽  
pp. 1550-1566 ◽  
Author(s):  
Mark Buehner ◽  
P. L. Houtekamer ◽  
Cecilien Charette ◽  
Herschel L. Mitchell ◽  
Bin He
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Spatial Covariance ◽  
4D Flow ◽  
Background Error ◽  
Single Observation ◽  
The Impact

Abstract An intercomparison of the Environment Canada variational and ensemble Kalman filter (EnKF) data assimilation systems is presented in the context of global deterministic NWP. In an EnKF experiment having the same spatial resolution as the inner loop in the four-dimensional variational data assimilation system (4D-Var), the mean of each analysis ensemble is used to initialize the higher-resolution deterministic forecasts. Five different variational data assimilation experiments are also conducted. These include both 4D-Var and 3D-Var (with first guess at appropriate time) experiments using either (i) prescribed background-error covariances similar to those used operationally, which are static in time and include horizontally homogeneous and isotropic correlations; or (ii) flow-dependent covariances computed from the EnKF background ensembles with spatial covariance localization applied. The fifth variational data assimilation experiment is a new approach called the Ensemble-4D-Var (En-4D-Var). This approach uses 4D flow-dependent background-error covariances estimated from EnKF ensembles to produce a 4D analysis without the need for tangent-linear or adjoint versions of the forecast model. In this first part of a two-part paper, results from a series of idealized assimilation experiments are presented. In these experiments, only a single observation or vertical profile of observations is assimilated to explore the impact of various fundamental differences among the EnKF and the various variational data assimilation approaches considered. In particular, differences in the application of covariance localization in the EnKF and variational approaches are shown to have a significant impact on the assimilation of satellite radiance observations. The results also demonstrate that 4D-Var and the EnKF can both produce similar 4D background-error covariances within a 6-h assimilation window. In the second part, results from medium-range deterministic forecasts for the study period of February 2007 are presented for the EnKF and the five variational data assimilation approaches considered.

scholarly journals Modeling the impact of drought on canopy carbon and water fluxes through parameter optimization using an ensemble Kalman filter

2009 ◽  
Vol 6 (4) ◽  
pp. 8279-8309 ◽  
Author(s):  
W. Ju ◽  
S. Wang ◽  
G. Yu ◽  
Y. Zhou ◽  
H. Wang
Keyword(s):  
Kalman Filter ◽  
Parameter Optimization ◽  
Ensemble Kalman Filter ◽  
Terrestrial Ecosystems ◽  
Model Parameters ◽  
Interannual Variations ◽  
Atmospheric Water ◽  
Model Parameterization ◽  
Coniferous Plantation ◽  
The Impact

Abstract. Soil and atmospheric water deficits have significant influences on CO2 and energy exchanges between the atmosphere and terrestrial ecosystems. Model parameterization significantly affects the ability of a model to simulate carbon, water, and energy fluxes. In this study, an ensemble Kalman filter (EnKF) and observations of gross primary productivity (GPP) and latent heat (LE) fluxes were used to optimize model parameters significantly affecting the calculation of these fluxes for a subtropical coniferous plantation in southeastern China. The optimized parameters include the maximum carboxylation rate (Vcmax), the Ball-Berry coefficient (m) and the coefficient determining the sensitivity of stomatal conductance to atmospheric water vapor deficit D0). Optimized Vcmax and m showed larger seasonal and interannual variations than D0. Seasonal variations of Vcmax and m are more pronounced than the interannual variations. Vcmax and m are associated with soil water content (SWC). During dry periods, SWC at the 20 cm depth can explain 61% and 64% of variations of Vcmax and m, respectively. EnKF parameter optimization improves the simulations of GPP, LE and sensible heat (SH), mainly during dry periods. After parameter optimization using EnKF, the variations of GPP, LE and SH explained by the model increased by 1% to 4% at half-hourly steps and by 3% to 5% at daily time steps. Efforts are needed to develop algorithms that can properly describe the variations of these parameters under different environmental conditions.

scholarly journals An Ensemble Kalman Filter for severe dust storm data assimilation over China

2008 ◽  
Vol 8 (11) ◽  
pp. 2975-2983 ◽  
Author(s):  
C. Lin ◽  
Z. Wang ◽  
J. Zhu
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Dust Storm ◽  
Ensemble Kalman Filter ◽  
Deposition Velocity ◽  
Dust Storms ◽  
Initial Time ◽  
Model Errors ◽  
The Impact ◽  
Severe Dust Storm

Abstract. An Ensemble Kalman Filter (EnKF) data assimilation system was developed for a regional dust transport model. This paper applied the EnKF method to investigate modeling of severe dust storm episodes occurring in March 2002 over China based on surface observations of dust concentrations to explore the impact of the EnKF data assimilation systems on forecast improvement. A series of sensitivity experiments using our system demonstrates the ability of the advanced EnKF assimilation method using surface observed PM10 in North China to correct initial conditions, which leads to improved forecasts of dust storms. However, large errors in the forecast may arise from model errors (uncertainties in meteorological fields, dust emissions, dry deposition velocity, etc.). This result illustrates that the EnKF requires identification and correction model errors during the assimilation procedure in order to significantly improve forecasts. Results also show that the EnKF should use a large inflation parameter to obtain better model performance and forecast potential. Furthermore, the ensemble perturbations generated at the initial time should include enough ensemble spreads to represent the background error after several assimilation cycles.

scholarly journals Covariance Inflation in the Ensemble Kalman Filter: A Residual Nudging Perspective and Some Implications

2013 ◽  
Vol 141 (10) ◽  
pp. 3360-3368 ◽  
Author(s):  
Xiaodong Luo ◽  
Ibrahim Hoteit
Keyword(s):  
Kalman Filter ◽  
Numerical Experiment ◽  
Ensemble Kalman Filter ◽  
Sufficient Conditions ◽  
The State ◽  
State Estimate ◽  
Inflation Factor

Abstract This article examines the influence of covariance inflation on the distance between the measured observation and the simulated (or predicted) observation with respect to the state estimate. In order for the aforementioned distance to be bounded in a certain interval, some sufficient conditions are derived, indicating that the covariance inflation factor should be bounded in a certain interval, and that the inflation bounds are related to the maximum and minimum eigenvalues of certain matrices. Implications of these analytic results are discussed, and a numerical experiment is presented to verify the validity of the analysis conducted.

Ensemble Kalman Filter Assimilation of HIWRAP Observations of Hurricane Karl (2010) from the Unmanned Global Hawk Aircraft

2014 ◽  
Vol 142 (12) ◽  
pp. 4559-4580 ◽  
Author(s):  
Jason A. Sippel ◽  
Fuqing Zhang ◽  
Yonghui Weng ◽  
Lin Tian ◽  
Gerald M. Heymsfield ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
Positive Impact ◽  
Simulated Data ◽  
Real Data ◽  
Horizontal Wind ◽  
Doppler Velocity ◽  
Flight Duration ◽  
The Impact ◽  
Global Hawk

Abstract This study utilizes an ensemble Kalman filter (EnKF) to assess the impact of assimilating observations of Hurricane Karl from the High-Altitude Imaging Wind and Rain Airborne Profiler (HIWRAP). HIWRAP is a new Doppler radar on board the NASA Global Hawk unmanned airborne system, which has the benefit of a 24–26-h flight duration, or about 2–3 times that of a conventional aircraft. The first HIWRAP observations were taken during NASA’s Genesis and Rapid Intensification Processes (GRIP) experiment in 2010. Observations considered here are Doppler velocity (Vr) and Doppler-derived velocity–azimuth display (VAD) wind profiles (VWPs). Karl is the only hurricane to date for which HIWRAP data are available. Assimilation of either Vr or VWPs has a significant positive impact on the EnKF analyses and forecasts of Hurricane Karl. Analyses are able to accurately estimate Karl’s observed location, maximum intensity, size, precipitation distribution, and vertical structure. In addition, forecasts initialized from the EnKF analyses are much more accurate than a forecast without assimilation. The forecasts initialized from VWP-assimilating analyses perform slightly better than those initialized from Vr-assimilating analyses, and the latter are less accurate than EnKF-initialized forecasts from a recent proof-of-concept study with simulated data. Likely causes for this discrepancy include the quality and coverage of the HIWRAP data collected from Karl and the presence of model error in this real-data situation. The advantages of assimilating VWP data likely include the ability to simultaneously constrain both components of the horizontal wind and to circumvent reliance upon vertical velocity error covariance.

scholarly journals Including Uncertainties of Sea Surface Temperature in an Ensemble Kalman Filter: A Case Study of Typhoon Sinlaku (2008)

2012 ◽  
Vol 27 (6) ◽  
pp. 1586-1597 ◽  
Author(s):  
Masaru Kunii ◽  
Takemasa Miyoshi
Keyword(s):  
Kalman Filter ◽  
Surface Temperature ◽  
Sea Surface Temperature ◽  
Ensemble Kalman Filter ◽  
Forecast Error ◽  
Ocean Model ◽  
Sea Surface ◽  
Ensemble Spread ◽  
Error Covariance ◽  
The Impact

Abstract Sea surface temperature (SST) plays an important role in tropical cyclone (TC) life cycle evolution, but often the uncertainties in SST estimates are not considered in the ensemble Kalman filter (EnKF). The lack of uncertainties in SST generally results in the lack of ensemble spread in the atmospheric states near the sea surface, particularly for temperature and moisture. In this study, the uncertainties of SST are included by adding ensemble perturbations to the SST field, and the impact of the SST perturbations is investigated using the local ensemble transform Kalman filter (LETKF) with the Weather Research and Forecasting Model (WRF) in the case of Typhoon Sinlaku (2008). In addition to the experiment with the perturbed SST, another experiment with manually inflated ensemble perturbations near the sea surface is performed for comparison. The results indicate that the SST perturbations within EnKF generally improve analyses and their subsequent forecasts, although manually inflating the ensemble spread instead of perturbing SST does not help. Investigations of the ensemble-based forecast error covariance indicate larger scales for low-level temperature and moisture from the SST perturbations, although manual inflation of ensemble spread does not produce such structural effects on the forecast error covariance. This study suggests the importance of considering SST perturbations within ensemble-based data assimilation and promotes further studies with more sophisticated methods of perturbing SST fields such as using a fully coupled atmosphere–ocean model.

scholarly journals Sensitivity tests for an ensemble Kalman filter for aerosol assimilation

2010 ◽  
Vol 10 (3) ◽  
pp. 5947-5997
Author(s):  
N. A. J. Schutgens ◽  
T. Miyoshi ◽  
T. Takemura ◽  
T. Nakajima
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
Patch Size ◽  
Initial Conditions ◽  
Grid Cell ◽  
Ensemble Size ◽  
Sensitivity Tests ◽  
Local Patch ◽  
The Impact ◽  
Assimilation System

Abstract. We present sensitivity tests for a global aerosol assimilation system utilizing AERONET observations of AOT (aerosol optical thickness) and AAE (aerosol Ångström exponent). The assimilation system employs an ensemble Kalman filter which requires optimization of three numerical parameters: ensemble size nens, local patch size npatch and inflation factor ρ. In addition, experiments are performed to test the impact of various implementations of the system. For instance, we use a different prescription of the emission ensemble or a different combination of observations. The various experiments are compared against one-another and against independent AERONET andMODIS/Aqua observations. The assimilation leads to significant improvements in modelled AOT and AAE fields. Moreover remaining errors are mostly random while they are mostly systematic for an experiment without assimilation. In addition, these results do not depend much on our parameter or design choices. It appears that the value of the local patch size has by far the biggest impact on the assimilation, which has sufficiently converged for an ensemble size of nens=20. Assimilating AOT and AAE is clearly preferential to assimilating AOT at two different wavelengths. In contrast, initial conditions or a description of aerosol beyond two modes (coarse and fine) have only little effect. We also discuss the use of the ensemble spread as an error estimate of the analysed AOT and AAE fields. We show that a very common prescription of the emission ensemble (independent random modification in each grid cell) can have trouble generating sufficient spread in the forecast ensemble.

scholarly journals Penalized ensemble Kalman filters for high dimensional non-linear systems

PLoS ONE ◽  
2021 ◽  
Vol 16 (3) ◽  
pp. e0248046
Author(s):  
Elizabeth Hou ◽  
Earl Lawrence ◽  
Alfred O. Hero
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
Estimation Error ◽  
Covariance Structure ◽  
Full Rank ◽  
The State ◽  
High Dimensional ◽  
Ensemble Kalman Filters ◽  
Non Linear ◽  
Non Linear System

The ensemble Kalman filter (EnKF) is a data assimilation technique that uses an ensemble of models, updated with data, to track the time evolution of a usually non-linear system. It does so by using an empirical approximation to the well-known Kalman filter. However, its performance can suffer when the ensemble size is smaller than the state space, as is often necessary for computationally burdensome models. This scenario means that the empirical estimate of the state covariance is not full rank and possibly quite noisy. To solve this problem in this high dimensional regime, we propose a computationally fast and easy to implement algorithm called the penalized ensemble Kalman filter (PEnKF). Under certain conditions, it can be theoretically proven that the PEnKF will be accurate (the estimation error will converge to zero) despite having fewer ensemble members than state dimensions. Further, as contrasted to localization methods, the proposed approach learns the covariance structure associated with the dynamical system. These theoretical results are supported with simulations of several non-linear and high dimensional systems.

Sign in / Sign up

Export Citation Format

Share Document