Heterogeneous Convective-Scale Background Error Covariances with the Inclusion of Hydrometeor Variables

Convective-scale models used in NWP nowadays include detailed realistic parameterization for the representation of cloud and precipitation processes. Yet they still lack advanced data assimilation schemes able to efficiently use observations to initialize hydrometeor fields. This challenging task may benefit from a better understanding of the statistical structure of background errors in precipitating areas for both traditional and hydrometeor variables, which is the goal of this study. A special binning has been devised to compute separate background error covariance matrices for precipitating and nonprecipitating areas. This binning is based on bidimensional geographical masks defined by the vertical averaged rain content of the background error perturbations. The sample for computing the covariances is taken from an ensemble of short range forecasts run at 3-km resolution for the prediction of two specific cases of convective storms over the United States. The covariance matrices and associated diagnostics are built on the control variable transform formulation typical of variational data assimilation. The comparison especially highlights the strong coupling of specific humidity, cloud, and rain content with divergence. Shorter horizontal correlations have been obtained in precipitating areas. Vertical correlations mostly reflect the cloud vertical extension due to the convective processes. The statistics for hydrometeor variables show physically meaningful autocovariances and statistical couplings with other variables. Issues for data assimilation of radar reflectivity or more generally of observations linked to cloud and rain content with this kind of background error matrix formulation are thereon briefly discussed.

Download Full-text

Interchannel Error Correlation Associated with AIRS Radiance Observations: Inference and Impact in Data Assimilation

Journal of Applied Meteorology and Climatology ◽

10.1175/jam2496.1 ◽

2007 ◽

Vol 46 (6) ◽

pp. 714-725 ◽

Cited By ~ 32

Author(s):

Louis Garand ◽

Sylvain Heilliette ◽

Mark Buehner

Keyword(s):

Data Assimilation ◽

Meteorological Data ◽

Specific Humidity ◽

Observation Error ◽

Carbon Dioxide Absorption ◽

Background Error ◽

Error Matrix ◽

Vapor Sensing ◽

Error Correlation ◽

The Impact

Abstract The interchannel observation error correlation (IOEC) associated with radiance observations is currently assumed to be zero in meteorological data assimilation systems. This assumption may lead to suboptimal analyses. Here, the IOEC is inferred for the Atmospheric Infrared Radiance Sounder (AIRS) hyperspectral radiance observations using a subset of 123 channels covering the spectral range of 4.1–15.3 μm. Observed minus calculated radiances are computed for a 1-week period using a 6-h forecast as atmospheric background state. A well-established technique is used to separate the observation and background error components for each individual channel and each channel pair. The large number of collocations combined with the 40-km horizontal spacing between AIRS fields of view allows robust results to be obtained. The resulting background errors are in good agreement with those inferred from the background error matrix used operationally in data assimilation at the Meteorological Service of Canada. The IOEC is in general high among the water vapor–sensing channels in the 6.2–7.2-μm region and among surface-sensitive channels. In contrast, it is negligible for channels within the main carbon dioxide absorption band (13.2–15.4 μm). The impact of incorporating the IOEC is evaluated from 1D variational retrievals at 381 clear-sky oceanic locations. Temperature increments differ on average by 0.25 K, and ln(q) increments by 0.10, where q is specific humidity. Without IOEC, the weight given to the observations appears to be too high; the assimilation attempts to fit the observations nearly perfectly. The IOEC better constrains the variational assimilation process, and the rate of convergence is systematically faster by a factor of 2.

Download Full-text

A GSI-Based Coupled EnSRF–En3DVar Hybrid Data Assimilation System for the Operational Rapid Refresh Model: Tests at a Reduced Resolution

Monthly Weather Review ◽

10.1175/mwr-d-13-00242.1 ◽

2014 ◽

Vol 142 (10) ◽

pp. 3756-3780 ◽

Cited By ~ 40

Author(s):

Yujie Pan ◽

Kefeng Zhu ◽

Ming Xue ◽

Xuguang Wang ◽

Ming Hu ◽

...

Keyword(s):

Data Assimilation ◽

Hybrid System ◽

Control Variable ◽

Three Dimensional ◽

Horizontal Resolution ◽

Precipitation Forecast ◽

Background Error ◽

Hybrid Data Assimilation ◽

Noticeable Improvement ◽

Forecasting System

Abstract A coupled ensemble square root filter–three-dimensional ensemble-variational hybrid (EnSRF–En3DVar) data assimilation (DA) system is developed for the operational Rapid Refresh (RAP) forecasting system. The En3DVar hybrid system employs the extended control variable method, and is built on the NCEP operational gridpoint statistical interpolation (GSI) three-dimensional variational data assimilation (3DVar) framework. It is coupled with an EnSRF system for RAP, which provides ensemble perturbations. Recursive filters (RF) are used to localize ensemble covariance in both horizontal and vertical within the En3DVar. The coupled En3DVar hybrid system is evaluated with 3-h cycles over a 9-day period with active convection. All conventional observations used by operational RAP are included. The En3DVar hybrid system is run at ⅓ of the operational RAP horizontal resolution or about 40-km grid spacing, and its performance is compared to parallel GSI 3DVar and EnSRF runs using the same datasets and resolution. Short-term forecasts initialized from the 3-hourly analyses are verified against sounding and surface observations. When using equally weighted static and ensemble background error covariances and 40 ensemble members, the En3DVar hybrid system outperforms the corresponding GSI 3DVar and EnSRF. When the recursive filter coefficients are tuned to achieve a similar height-dependent localization as in the EnSRF, the En3DVar results using pure ensemble covariance are close to EnSRF. Two-way coupling between EnSRF and En3DVar did not produce noticeable improvement over one-way coupling. Downscaled precipitation forecast skill on the 13-km RAP grid from the En3DVar hybrid is better than those from GSI 3DVar analyses.

Download Full-text

A Comparison of Multiscale GSI-Based EnKF and 3DVar Data Assimilation Using Radar and Conventional Observations for Midlatitude Convective-Scale Precipitation Forecasts

Monthly Weather Review ◽

10.1175/mwr-d-14-00345.1 ◽

2015 ◽

Vol 143 (8) ◽

pp. 3087-3108 ◽

Cited By ~ 59

Author(s):

Aaron Johnson ◽

Xuguang Wang ◽

Jacob R. Carley ◽

Louis J. Wicker ◽

Christopher Karstens

Keyword(s):

Data Assimilation ◽

Multiscale Analysis ◽

Multiple Scales ◽

Three Dimensional ◽

Forecast Period ◽

Background Error ◽

Cold Pools ◽

Error Covariance ◽

Convective Scale

Abstract A GSI-based data assimilation (DA) system, including three-dimensional variational assimilation (3DVar) and ensemble Kalman filter (EnKF), is extended to the multiscale assimilation of both meso- and synoptic-scale observation networks and convective-scale radar reflectivity and velocity observations. EnKF and 3DVar are systematically compared in this multiscale context to better understand the impacts of differences between the DA techniques on the analyses at multiple scales and the subsequent convective-scale precipitation forecasts. Averaged over 10 diverse cases, 8-h precipitation forecasts initialized using GSI-based EnKF are more skillful than those using GSI-based 3DVar, both with and without storm-scale radar DA. The advantage from radar DA persists for ~5 h using EnKF, but only ~1 h using 3DVar. A case study of an upscale growing MCS is also examined. The better EnKF-initialized forecast is attributed to more accurate analyses of both the mesoscale environment and the storm-scale features. The mesoscale location and structure of a warm front is more accurately analyzed using EnKF than 3DVar. Furthermore, storms in the EnKF multiscale analysis are maintained during the subsequent forecast period. However, storms in the 3DVar multiscale analysis are not maintained and generate excessive cold pools. Therefore, while the EnKF forecast with radar DA remains better than the forecast without radar DA throughout the forecast period, the 3DVar forecast quality is degraded by radar DA after the first hour. Diagnostics revealed that the inferior analysis at mesoscales and storm scales for the 3DVar is primarily attributed to the lack of flow dependence and cross-variable correlation, respectively, in the 3DVar static background error covariance.

Download Full-text

Relationships among Four-Dimensional Hybrid Ensemble–Variational Data Assimilation Algorithms with Full and Approximate Ensemble Covariance Localization

Monthly Weather Review ◽

10.1175/mwr-d-15-0203.1 ◽

2016 ◽

Vol 144 (2) ◽

pp. 591-606 ◽

Cited By ~ 10

Author(s):

Chengsi Liu ◽

Ming Xue

Keyword(s):

Data Assimilation ◽

Linear Model ◽

Control Variable ◽

Adjoint Model ◽

Background Error ◽

Background Error Covariance ◽

Error Covariance ◽

Covariance Term ◽

Observation Space ◽

Static Background

Abstract Ensemble–variational data assimilation algorithms that can incorporate the time dimension (four-dimensional or 4D) and combine static and ensemble-derived background error covariances (hybrid) are formulated in general forms based on the extended control variable and the observation-space-perturbation approaches. The properties and relationships of these algorithms and their approximated formulations are discussed. The main algorithms discussed include the following: 1) the standard ensemble 4DVar (En4DVar) algorithm incorporating ensemble-derived background error covariance through the extended control variable approach, 2) the 4DEnVar neglecting the time propagation of the extended control variable (4DEnVar-NPC), 3) the 4D ensemble–variational algorithm based on observation space perturbation (4DEnVar), and 4) the 4DEnVar with no propagation of covariance localization (4DEnVar-NPL). Without the static background error covariance term, none of the algorithms requires the adjoint model except for En4DVar. Costly applications of the tangent linear model to localized ensemble perturbations can be avoided by making the NPC and NPL approximations. It is proven that En4DVar and 4DEnVar are mathematically equivalent, while 4DEnVar-NPC and 4DEnVar-NPL are mathematically equivalent. Such equivalences are also demonstrated by single-observation assimilation experiments with a 1D linear advection model. The effects of the non-flow-following or stationary localization approximations are also examined through the experiments. All of the above algorithms can include the static background error covariance term to establish a hybrid formulation. When the static term is included, all algorithms will require a tangent linear model and an adjoint model. The first guess at appropriate time (FGAT) approximation is proposed to avoid the tangent linear and adjoint models. Computational costs of the algorithms are also discussed.

Download Full-text

Construction of non-diagonal background error covariance matrices for global chemical data assimilation

Geoscientific Model Development Discussions ◽

10.5194/gmdd-3-1783-2010 ◽

2010 ◽

Vol 3 (4) ◽

pp. 1783-1827 ◽

Cited By ~ 2

Author(s):

K. Singh ◽

M. Jardak ◽

A. Sandu ◽

K. Bowman ◽

M. Lee ◽

...

Keyword(s):

Data Assimilation ◽

Computational Procedure ◽

Covariance Matrices ◽

Chemical Data ◽

Spatial Correlations ◽

Transport Models ◽

Background Error ◽

Spatial Error ◽

Background Error Covariance ◽

Error Covariance

Abstract. Chemical data assimilation attempts to optimally use noisy observations along with imperfect model predictions to produce a better estimate of the chemical state of the atmosphere. It is widely accepted that a key ingredient for successful data assimilation is a realistic estimation of the background error distribution. Particularly important is the specification of the background error covariance matrix, which contains information about the magnitude of the background errors and about their correlations. Most models currently use diagonal background covariance matrices. As models evolve toward finer resolutions, the diagonal background covariance matrices become increasingly inaccurate, since they captures less of the spatial error correlations. This paper discusses an efficient computational procedure for constructing non-diagonal background error covariance matrices which account for the spatial correlations of errors. The benefits of using the non-diagonal covariance matrices for variational data assimilation with chemical transport models are illustrated.

Download Full-text

Biogeochemical Oceanographic Data Assimilation: Dimensionality Reduced Kalman Filter For Mediterranean Sea Forecasting

10.14293/p2199-8442.1.sop-math.vgditb.v1 ◽

2018 ◽

Author(s):

Simone K. Spada ◽

Gianpiero Cossarini ◽

Stefano Salon ◽

Stefano Maset

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Mediterranean Sea ◽

Computational Efficiency ◽

Transport Model ◽

Computational Cost ◽

Principal Component ◽

Covariance Matrices ◽

Background Error ◽

Biological Variables

Data assimilation is a key element to improve the performance of biogeochemical ocean/marine forecasting systems. Handling the very big dimension of the state vector of the system (often of the order of 10 6 ) remains an issue, also considering the computational efficiency of operational systems. Indeed, simple product operations involving the covariance matrices are too heavy to be computed for operational forecasting purposes. Various attempts have been made in literature to reduce the complexity of this task, often adding strong hypotheses to simplify the problem and decrease the computational cost. The MedBFM model system, which is responsible for monitoring and forecasting the biogeochemical state of the Mediterranean Sea within the European Copernicus Marine Services (see http://marine.copernicus.eu/ ) assimilates surface chlorophyll data through a 3D Variational algorithm, that decomposes the background error covariance matrix into sequential operators to reduce complexity. In this work, we developed a novel Kalman Filter for the MedBFM system. The novel Kalman Filter scheme starts from a SEIK approach but benefits from advanced Principal Component Analysis to reduce the dimension of covariance matrices and improve the computational efficiency. We compared the standard SEIK filter and the new Kalman filter implementations in a one dimensional transport model with 2 biological variables in terms of root mean square distance. In the vast majority of the experiments, the new Kalman filter had better performances.

Download Full-text

The “ABC-DA system” (v1.4): a variational data assimilation system for convective scale assimilation research with a study of the impact of a balance constraint

10.5194/gmd-2019-318 ◽

2020 ◽

Author(s):

Ross Noel Bannister

Keyword(s):

Data Assimilation ◽

Covariance Matrix ◽

Variational Data Assimilation ◽

Background Error ◽

Error Covariance Matrix ◽

Background Error Covariance ◽

Error Covariance ◽

Convective Scale ◽

Data Assimilation System ◽

Assimilation System

Abstract. Following the development of the simplified atmospheric convective-scale "toy" model (the ABC model, named after its three key parameters: the pure gravity wave frequency, A, the controller of the acoustic wave speed, B, and the constant of proportionality between pressure and density perturbations, C), this paper introduces its associated variational data assimilation system, ABC-DA. The purpose of ABC-DA is to permit quick and efficient research into data assimilation methods suitable for convective scale systems. The system can also be used as an aid to teach and demonstrate data assimilation principles. ABC-DA is flexible, configurable and is efficient enough to be run on a personal computer. The system can run a number of assimilation methods (currently 3DVar and 3DFGAT have been implemented), with user configurable observation networks. Observation operators for direct observations and wind speeds are part of the system, although these can be expanded relatively easily. A key feature of any data assimilation system is how it specifies the background error covariance matrix. ABC-DA uses a control variable transform method to allow this to be done efficiently. This version of ABC-DA mirrors many operational configurations, by modelling multivariate error covariances with uncorrelated control parameters, and spatial error covariances with special uncorrelated spatial patterns separately for each parameter. The software developed (amongst other things) does model runs, calibration tasks associated with the background error covariance matrix, testing and diagnostic tasks, single data assimilation runs, multi-cycle assimilation/forecast experiments, and has associated visualisation software. As a demonstration, the system is used to tackle a scientific question concerning the role of geostrophic balance (GB) to model background error covariances between mass and wind fields. This question arises because, although GB is a very useful mechanism that is successfully exploited in larger scale assimilation systems, its use is questionable at convective scales due to the typically larger Rossby numbers where GB is not so relevant. A series of identical twin experiments is done in cycled assimilation configurations. One experiment exploits GB to represent mass-wind covariances in a mirror of an operational set-up (with use of an additional vertical regression (VR) step, as used operationally). This experiment performs badly where assimilation error accumulates over time. Two further experiments are done: one that does not use GB, and another that does but without the VR step. Turning off GB impairs the performance, and turning off VR improves the performance in general. It is concluded that there is scope to further improve the way that the background error covariance matrices are calibrated, with some directions discussed.

Download Full-text

An Overlooked Issue of Variational Data Assimilation

Monthly Weather Review ◽

10.1175/mwr-d-14-00404.1 ◽

2015 ◽

Vol 143 (10) ◽

pp. 3925-3930 ◽

Cited By ~ 7

Author(s):

Benjamin Ménétrier ◽

Thomas Auligné

Keyword(s):

Data Assimilation ◽

Cost Function ◽

Control Variable ◽

Variational Data Assimilation ◽

Inner Product ◽

Background Error ◽

Error Covariance ◽

Definition Of ◽

Linear Unbiased Estimate ◽

The Cost

Abstract The control variable transform (CVT) is a keystone of variational data assimilation. In publications using such a technique, the background term of the transformed cost function is defined as a canonical inner product of the transformed control variable with itself. However, it is shown in this paper that this practical definition of the cost function is not correct if the CVT uses a square root of the background error covariance matrix that is not square. Fortunately, it is then shown that there is a manifold of the control space for which this flaw has no impact, and that most minimizers used in practice precisely work in this manifold. It is also shown that both correct and practical transformed cost functions have the same minimum. This explains more rigorously why the CVT is working in practice. The case of a singular is finally detailed, showing that the practical cost function still reaches the best linear unbiased estimate (BLUE).

Download Full-text

Assessment of the data assimilation framework for the Rapid Refresh Forecast System v0.1 and impacts on forecasts of convective storms

10.5194/gmd-2021-289 ◽

2021 ◽

Author(s):

Ivette H. Banos ◽

Will D. Mayfield ◽

Guoqing Ge ◽

Luiz F. Sapucci ◽

Jacob R. Carley ◽

...

Keyword(s):

Data Assimilation ◽

Current Data ◽

Dew Point ◽

Squall Line ◽

Forecast System ◽

Background Error ◽

Convective Storms ◽

Error Covariance ◽

Convective Scale ◽

Environmental Prediction

Abstract. The Rapid Refresh Forecast System (RRFS) is currently under development and aims to replace the National Centers for Environmental Prediction (NCEP) operational suite of regional and convective scale modeling systems in the next upgrade. In order to achieve skillful forecasts comparable to the current operational suite, each component of the RRFS needs to be configured through exhaustive testing and evaluation. The current data assimilation component uses the Gridpoint Statistical Interpolation (GSI) system. In this study, various data assimilation algorithms and configurations in GSI are assessed for their impacts on RRFS analyses and forecasts of a squall line over Oklahoma on 4 May 2020. Results show that a baseline RRFS run without data assimilation is able to represent the observed convection, but with stronger cells and large location errors. With data assimilation, these errors are reduced, especially in the 4 and 6 h forecasts using 75 % of the ensemble background error covariance (BEC) and with the supersaturation removal function activated in GSI. Decreasing the vertical ensemble localization radius in the first 10 layers of the hybrid analysis results in overall less skillful forecasts. Convection and precipitation are overforecast in most forecast hours when using planetary boundary layer pseudo-observations, but the root mean square error and bias of the 2 h forecast of 2 m dew point temperature are reduced by 1.6 K during the afternoon hours. Lighter hourly accumulated precipitation is predicted better when using 100 % ensemble BEC in the first 4 h forecast, but heavier hourly accumulated precipitation is better predicted with 75 % ensemble BEC. Our results provide insight into current capabilities of the RRFS data assimilation system and identify configurations that should be considered as candidates for the first version of RRFS.

Download Full-text

Inhomogeneous Background Error Modeling for WRF-Var Using the NMC Method

Journal of Applied Meteorology and Climatology ◽

10.1175/jamc-d-13-0281.1 ◽

2014 ◽

Vol 53 (10) ◽

pp. 2287-2309 ◽

Cited By ~ 26

Author(s):

Hongli Wang ◽

Xiang-Yu Huang ◽

Juanzhen Sun ◽

Dongmei Xu ◽

Man Zhang ◽

...

Keyword(s):

Data Assimilation ◽

Forecast Error ◽

Control Variable ◽

Variational Data Assimilation ◽

Error Modeling ◽

Error Statistics ◽

Background Error ◽

New Approach ◽

Data Assimilation System ◽

Assimilation System

AbstractBackground error modeling plays a key role in a variational data assimilation system. The National Meteorological Center (NMC) method has been widely used in variational data assimilation systems to generate a forecast error ensemble from which the climatological background error covariance can be modeled. In this paper, the characteristics of the background error modeling via the NMC method are investigated for the variational data assimilation system of the Weather Research and Forecasting (WRF-Var) Model. The background error statistics are extracted from short-term 3-km-resolution forecasts in June, July, and August 2012 over a limited-area domain. It is found 1) that background error variances vary from month to month and also have a feature of diurnal variations in the low-level atmosphere and 2) that u- and υ-wind variances are underestimated and their autocorrelation length scales are overestimated when the default control variable option in WRF-Var is used. A new approach of control variable transform (CVT) is proposed to model the background error statistics based on the NMC method. The new approach is capable of extracting inhomogeneous and anisotropic climatological information from the forecast error ensemble obtained via the NMC method. Single observation assimilation experiments show that the proposed method not only has the merit of incorporating geographically dependent covariance information, but also is able to produce a multivariate analysis. The results from the data assimilaton and forecast study of a real convective case show that the use of the new CVT improves synoptic weather system and precipitation forecasts for up to 12 h.

Download Full-text