Assimilation of GOSAT Methane in the Hemispheric CMAQ; Part II: Results Using Optimal Error Statistics

We applied the parametric variance Kalman filter (PvKF) data assimilation designed in Part I of this two-part paper to GOSAT methane observations with the hemispheric version of CMAQ to obtain the methane field (i.e., optimized analysis) with its error variance. Although the Kalman filter computes error covariances, the optimality depends on how these covariances reflect the true error statistics. To achieve more accurate representation, we optimize the global variance parameters, including correlation length scales and observation errors, based on a cross-validation cost function. The model and the initial error are then estimated according to the normalized variance matching diagnostic, also to maintain a stable analysis error variance over time. The assimilation results in April 2010 are validated against independent surface and aircraft observations. The statistics of the comparison of the model and analysis show a meaningful improvement against all four types of available observations. Having the advantage of continuous assimilation, we showed that the analysis also aims at pursuing the temporal variation of independent measurements, as opposed to the model. Finally, the performance of the PvKF assimilation in capturing the spatial structure of bias and uncertainty reduction across the Northern Hemisphere is examined, indicating the capability of analysis in addressing those biases originated, whether from inaccurate emissions or modelling error.

Download Full-text

Interpretation of Adaptive Observing Guidance for Atlantic Tropical Cyclones

Monthly Weather Review ◽

10.1175/2007mwr2027.1 ◽

2007 ◽

Vol 135 (12) ◽

pp. 4006-4029 ◽

Cited By ~ 26

Author(s):

C. A. Reynolds ◽

M. S. Peng ◽

S. J. Majumdar ◽

S. D. Aberson ◽

C. H. Bishop ◽

...

Keyword(s):

Data Assimilation ◽

Total Energy ◽

Tropical Cyclones ◽

Error Variance ◽

Naval Research ◽

Error Statistics ◽

Background Error ◽

Analysis Error ◽

The Impact ◽

Perturbation Growth

Abstract Adaptive observing guidance products for Atlantic tropical cyclones are compared using composite techniques that allow one to quantitatively examine differences in the spatial structures of the guidance maps and relate these differences to the constraints and approximations of the respective techniques. The guidance maps are produced using the ensemble transform Kalman filter (ETKF) based on ensembles from the National Centers for Environmental Prediction and the European Centre for Medium-Range Weather Forecasts (ECMWF), and total-energy singular vectors (TESVs) produced by ECMWF and the Naval Research Laboratory. Systematic structural differences in the guidance products are linked to the fact that TESVs consider the dynamics of perturbation growth only, while the ETKF combines information on perturbation evolution with error statistics from an ensemble-based data assimilation scheme. The impact of constraining the SVs using different estimates of analysis error variance instead of a total-energy norm, in effect bringing the two methods closer together, is also assessed. When the targets are close to the storm, the TESV products are a maximum in an annulus around the storm, whereas the ETKF products are a maximum at the storm location itself. When the targets are remote from the storm, the TESVs almost always indicate targets northwest of the storm, whereas the ETKF targets are more scattered relative to the storm location and often occur over the northern North Atlantic. The ETKF guidance often coincides with locations in which the ensemble-based analysis error variance is large. As the TESV method is not designed to consider spatial differences in the likely analysis errors, it will produce targets over well-observed regions, such as the continental United States. Constraining the SV calculation using analysis error variance values from an operational 3D variational data assimilation system (with stationary, quasi-isotropic background error statistics) results in a modest modulation of the target areas away from the well-observed regions, and a modest reduction of perturbation growth. Constraining the SVs using the ETKF estimate of analysis error variance produces SV targets similar to ETKF targets and results in a significant reduction in perturbation growth, due to the highly localized nature of the analysis error variance estimates. These results illustrate the strong sensitivity of SVs to the norm (and to the analysis error variance estimate used to define it) and confirm that discrepancies between target areas computed using different methods reflect the mathematical and physical differences between the methods themselves.

Download Full-text

Assimilation with an Ensemble Kalman Filter of Synthetic Radial Wind Data in Anisotropic Turbulence: Perfect Model Experiments

Monthly Weather Review ◽

10.1175/mwr3081.1 ◽

2006 ◽

Vol 134 (2) ◽

pp. 618-637 ◽

Cited By ~ 4

Author(s):

Martin Charron ◽

P. L. Houtekamer ◽

Peter Bartello

Keyword(s):

Kalman Filter ◽

Vertical Velocity ◽

Ensemble Kalman Filter ◽

Grid Point ◽

Synthetic Data ◽

Wind Data ◽

Error Statistics ◽

Boussinesq Model ◽

Analysis Error ◽

The Impact

Abstract The ensemble Kalman filter (EnKF) developed at the Meteorological Research Branch of Canada is used in the context of synthetic radial wind data assimilation at the mesoscale. A dry Boussinesq model with periodic boundary conditions is employed to provide a control run, as well as two ensembles of first guesses. Synthetic data, which are interpolated from the control run, are assimilated and simulate Doppler radar wind measurements. Nine “radars” with a range of 120 km are placed evenly on the horizontal 1000 km × 1000 km domain. These radars measure the radial wind with assumed Gaussian error statistics at each grid point within their range provided that there is sufficient upward motion (a proxy for precipitation). These data of radial winds are assimilated every 30 min and the assimilation period extends over 4 days. Results show that the EnKF technique with 2 × 50 members performed well in terms of reducing the analysis error for horizontal winds and temperature (even though temperature is not an observed variable) over a period of 4 days. However the analyzed vertical velocity shows an initial degradation. During the first 2 days of the assimilation period, the analysis error of the vertical velocity is greater when assimilating radar observations than when scoring forecasts initialized at t = 0 without assimilating any data. The type of assimilated data as well as the localization of the impact of the observations is thought to be the cause of this degradation of the analyzed vertical velocity. External gravity modes are present in the increments when localization is performed. This degradation can be eliminated by filtering the external gravity modes of the analysis increments. A similar set of experiments is realized in which the model dissipation coefficient is reduced by a factor of 10. This shows the level of sensitivity of the results to the kinetic energy power spectrum, and that the quality of the analyzed vertical wind is worse when dissipation is small.

Download Full-text

Evaluation of Analysis by Cross-Validation. Part I: Using Verification Metrics

10.20944/preprints201801.0217.v1 ◽

2018 ◽

Author(s):

Richard Menard ◽

Martin Deshaies-Jacques

Keyword(s):

Air Quality ◽

Cross Validation ◽

Error Variance ◽

Quality Analysis ◽

Second Moment ◽

Error Margin ◽

Analysis Scheme ◽

Analysis Error ◽

First Moment

We examine how observations can be used to evaluate an air quality analysis by verifying against passive observations (i.e. cross-validation) that are not used to create the analysis and we compare these verifications to those made against the same set of (active) observations that were used to generate the analysis. The results show that both active and passive observations can be used to evaluate of first moment metrics (e.g. bias) but only passive observations are useful to evaluate second moment metrics such as variance of observed-minus-analysis and correlation between observations and analysis. We derive a set of diagnostics based on passive observation–minus-analysis residuals and we show that the true analysis error variance can be estimated, without relying on any statistical optimality assumption. This diagnostic is used to obtain near optimal analyses that are then used to evaluate the analysis error using several different methods. We compare the estimates according to the method of Hollingsworth Lonnberg, Desroziers, a diagnostic we introduce, and the perceived analysis error computed from the analysis scheme, to conclude that as long as the analysis is optimal, all estimates agrees within a certain error margin. The analysis error variance at passive observation sites is also obtained.

Download Full-text

Correlation-Cutoff Method for Covariance Localization in Strongly Coupled Data Assimilation

Monthly Weather Review ◽

10.1175/mwr-d-17-0365.1 ◽

2018 ◽

Vol 146 (9) ◽

pp. 2881-2889 ◽

Cited By ~ 5

Author(s):

Takuma Yoshida ◽

Eugenia Kalnay

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Coupled Model ◽

Error Variance ◽

State Variables ◽

Background Error ◽

Strongly Coupled ◽

Error Correlation ◽

Analysis Error ◽

Coupled Data Assimilation

Abstract Strongly coupled data assimilation (SCDA), where observations of one component of a coupled model are allowed to directly impact the analysis of other components, sometimes fails to improve the analysis accuracy with an ensemble Kalman filter (EnKF) as compared with weakly coupled data assimilation (WCDA). It is well known that an observation’s area of influence should be localized in EnKFs since the assimilation of distant observations often degrades the analysis because of spurious correlations. This study derives a method to estimate the reduction of the analysis error variance by using estimates of the cross covariances between the background errors of the state variables in an idealized situation. It is shown that the reduction of analysis error variance is proportional to the squared background error correlation between the analyzed and observed variables. From this, the authors propose an offline method to systematically select which observations should be assimilated into which model state variable by cutting off the assimilation of observations when the squared background error correlation between the observed and analyzed variables is small. The proposed method is tested with the local ensemble transform Kalman filter (LETKF) and a nine-variable coupled model, in which three Lorenz models with different time scales are coupled with each other. The covariance localization with the correlation-cutoff method achieves an analysis more accurate than either the full SCDA or the WCDA methods, especially with smaller ensemble sizes.

Download Full-text

Evaluation of Analysis by Cross-Validation. Part II: Diagnostic and Optimization of Analysis Error Covariance

10.20944/preprints201711.0047.v1 ◽

2017 ◽

Author(s):

Richard Ménard ◽

Martin Deshaies-Jacques

Keyword(s):

Air Quality ◽

General Theory ◽

Cross Validation ◽

Error Variance ◽

Geometric Interpretation ◽

Quality Analysis ◽

Error Margin ◽

Error Covariance ◽

Analysis Scheme ◽

Analysis Error

We present a general theory of estimation of analysis error covariances based on cross-validation as well as a geometric interpretation of the method. In particular we use the variance of passive observation–minus-analysis residuals and show that the true analysis error variance can be estimated, without relying on the optimality assumption. This approach is used to obtain near optimal analyses that are then used to evaluate the air quality analysis error using several different methods at active and passive observation sites. We compare the estimates according to the method of Hollingsworth-Lönnberg, Desroziers et al., a new diagnostic we developed, and the perceived analysis error computed from the analysis scheme, to conclude that, as long as the analysis is near optimal, all estimates agree within a certain error margin.

Download Full-text

Assimilation of GOSAT Methane in the Hemispheric CMAQ; Part I: Design of the Assimilation System

Remote Sensing ◽

10.3390/rs14020371 ◽

2022 ◽

Vol 14 (2) ◽

pp. 371

Author(s):

Sina Voshtani ◽

Richard Ménard ◽

Thomas W. Walker ◽

Amir Hakami

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Error Variance ◽

Companion Paper ◽

Advective Flow ◽

Vertical Error ◽

Analysis Error ◽

Data Assimilation System ◽

Assimilation System

We present a parametric Kalman filter data assimilation system using GOSAT methane observations within the hemispheric CMAQ model. The assimilation system produces forecasts and analyses of concentrations and explicitly computes its evolving error variance while remaining computationally competitive with other data assimilation schemes such as 4-dimensional variational (4D-Var) and ensemble Kalman filter (EnKF). The error variance in this system is advected using the native advection scheme of the CMAQ model and updated at each analysis while the error correlations are kept fixed. We discuss extensions to the CMAQ model to include methane transport and emissions (both anthropogenic and natural) and perform a bias correction for the GOSAT observations. The results using synthetic observations show that the analysis error and analysis increments follow the advective flow while conserving the information content (i.e., total variance). We also demonstrate that the vertical error correlation contributes to the inference of variables down to the surface. In a companion paper, we use this assimilation system to obtain optimal assimilation of GOSAT observations.

Download Full-text

Evaluation of Surface Analyses and Forecasts with a Multiscale Ensemble Kalman Filter in Regions of Complex Terrain

Monthly Weather Review ◽

10.1175/2010mwr3612.1 ◽

2011 ◽

Vol 139 (6) ◽

pp. 2008-2024 ◽

Cited By ~ 24

Author(s):

Brian C. Ancell ◽

Clifford F. Mass ◽

Gregory J. Hakim

Keyword(s):

Kalman Filter ◽

High Resolution ◽

Data Assimilation ◽

Ensemble Kalman Filter ◽

Complex Terrain ◽

Error Variance ◽

Small Scale ◽

Observation Error ◽

Grid Spacing ◽

Background Error

Abstract Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30–50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%–30%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested.

Download Full-text