Adjoint-Based Observation Impact Estimation with Direct Verification Using Forward Calculation

2018 ◽  
Vol 146 (9) ◽  
pp. 2837-2858 ◽  
Author(s):  
Toshiyuki Ishibashi
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Weather Prediction ◽  
Adjoint Operator ◽  
Estimation Method ◽  
Forecast Model ◽  
Japan Meteorological Agency ◽  
Essential Information ◽  
Observation Impact ◽  
Forward Calculation

Abstract The adjoint-based observation impact estimation method has been providing essential information to improve data assimilation systems (DASs) in numerical weather prediction (NWP). This paper has two purposes. The first is to verify the four approximations used in the method: iterative construction of the Kalman gain adjoint operator, the tangent linear (TL) approximation of a forecast model, ignoring cross terms of observation impacts, and approximations for incremental DASs. The second is to add new information to our knowledge of observation impacts. For the verification of the adjoint-based method, we use the TL-based observation impact estimation method that can calculate the same quantity as the adjoint-based method without adjoint calculations. Results of these verifications and observation impact estimations performed on the global NWP system of the Japan Meteorological Agency are as follows. First, observation impacts calculated using the adjoint-based method agree well with those from the TL-based method, with the correlation coefficient between them exceeding 0.97. Second, estimated observation impacts are consistent with previous studies in many aspects. There are also system-dependent properties, such as relatively small impacts from GPS radio occultation data above 13 km. Furthermore, new aspects of observation impacts are found: 1) the probability density function of the observation impact agrees well with the scalar theory when giving the experimental value of the observation and the forecast error standard deviations; 2) later observations in the data assimilation window have larger positive impacts; and 3) impacts of AMSU-A in the Southern Hemisphere are more than double those in the Northern Hemisphere.

A New Approach for Estimating the Observation Impact in Ensemble–Variational Data Assimilation

2018 ◽  
Vol 146 (2) ◽  
pp. 447-465 ◽  
Author(s):  
Mark Buehner ◽  
Ping Du ◽  
Joël Bédard
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Forecast Model ◽  
Variational Data Assimilation ◽  
Observation Error ◽  
New Approach ◽  
Regional Prediction ◽  
Observation Impact ◽  
Adjoint Approach ◽  
The Impact

Abstract Two types of approaches are commonly used for estimating the impact of arbitrary subsets of observations on short-range forecast error. The first was developed for variational data assimilation systems and requires the adjoint of the forecast model. Comparable approaches were developed for use with the ensemble Kalman filter and rely on ensembles of forecasts. In this study, a new approach for computing observation impact is proposed for ensemble–variational data assimilation (EnVar). Like standard adjoint approaches, the adjoint of the data assimilation procedure is implemented through the iterative minimization of a modified cost function. However, like ensemble approaches, the adjoint of the forecast step is obtained by using an ensemble of forecasts. Numerical experiments were performed to compare the new approach with the standard adjoint approach in the context of operational deterministic NWP. Generally similar results are obtained with both approaches, especially when the new approach uses covariance localization that is horizontally advected between analysis and forecast times. However, large differences in estimated impacts are obtained for some surface observations. Vertical propagation of the observation impact is noticeably restricted with the new approach because of vertical covariance localization. The new approach is used to evaluate changes in observation impact as a result of the use of interchannel observation error correlations for radiance observations. The estimated observation impact in similarly configured global and regional prediction systems is also compared. Overall, the new approach should provide useful estimates of observation impact for data assimilation systems based on EnVar when an adjoint model is not available.

scholarly journals Southern High-Latitude Ensemble Data Assimilation in the Antarctic Mesoscale Prediction System

2005 ◽  
Vol 133 (12) ◽  
pp. 3431-3449 ◽  
Author(s):  
D. M. Barker
Keyword(s):  
Data Assimilation ◽  
Sampling Error ◽  
Forecast Error ◽  
Weather Prediction ◽  
Forecast Errors ◽  
Prediction System ◽  
Spatial Covariance ◽  
Ensemble Data Assimilation ◽  
Ensemble Data ◽  
The Antarctic

Abstract Ensemble data assimilation systems incorporate observations into numerical models via solution of the Kalman filter update equations, and estimates of forecast error covariances derived from ensembles of model integrations. In this paper, a particular algorithm, the ensemble square root filter (EnSRF), is tested in a limited-area, polar numerical weather prediction (NWP) model: the Antarctic Mesoscale Prediction System (AMPS). For application in the real-time AMPS, the number of model integrations that can be run to provide forecast error covariances is limited, resulting in an ensemble sampling error that degrades the analysis fit to observations. In this work, multivariate, climatologically plausible forecast error covariances are specified via averaged forecast difference statistics. Ensemble representations of the “true” forecast errors, created using randomized control variables of the fifth-generation Pennsylvania State University–National Center for Atmospheric Research (PSU–NCAR) Mesoscale Model (MM5) three-dimensional variational (3DVAR) data assimilation system, are then used to assess the dependence of sampling error on ensemble size, data density, and localization of covariances using simulated observation networks. Results highlight the detrimental impact of ensemble sampling error on the analysis increment structure of correlated, but unobserved fields—an issue not addressed by the spatial covariance localization techniques used to date. A 12-hourly cycling EnSRF/AMPS assimilation/forecast system is tested for a two-week period in December 2002 using real, conventional (surface, rawinsonde, satellite retrieval) observations. The dependence of forecast scores on methods used to maintain ensemble spread and the inclusion of perturbations to lateral boundary conditions are studied.

scholarly journals Assimilation of MODIS Dark Target and Deep Blue observations in the dust aerosol component of NMMB-MONARCH version 1.0

2017 ◽  
Vol 10 (3) ◽  
pp. 1107-1129 ◽  
Author(s):  
Enza Di Tomaso ◽  
Nick A. J. Schutgens ◽  
Oriol Jorba ◽  
Carlos Pérez García-Pando
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Mineral Dust ◽  
Positive Impact ◽  
Forecast Error ◽  
Weather Prediction ◽  
Prediction System ◽  
Deep Blue ◽  
Modis Aod ◽  
Sahel Region

Abstract. A data assimilation capability has been built for the NMMB-MONARCH chemical weather prediction system, with a focus on mineral dust, a prominent type of aerosol. An ensemble-based Kalman filter technique (namely the local ensemble transform Kalman filter – LETKF) has been utilized to optimally combine model background and satellite retrievals. Our implementation of the ensemble is based on known uncertainties in the physical parametrizations of the dust emission scheme. Experiments showed that MODIS AOD retrievals using the Dark Target algorithm can help NMMB-MONARCH to better characterize atmospheric dust. This is particularly true for the analysis of the dust outflow in the Sahel region and over the African Atlantic coast. The assimilation of MODIS AOD retrievals based on the Deep Blue algorithm has a further positive impact in the analysis downwind from the strongest dust sources of the Sahara and in the Arabian Peninsula. An analysis-initialized forecast performs better (lower forecast error and higher correlation with observations) than a standard forecast, with the exception of underestimating dust in the long-range Atlantic transport and degradation of the temporal evolution of dust in some regions after day 1. Particularly relevant is the improved forecast over the Sahara throughout the forecast range thanks to the assimilation of Deep Blue retrievals over areas not easily covered by other observational datasets. The present study on mineral dust is a first step towards data assimilation with a complete aerosol prediction system that includes multiple aerosol species.

scholarly journals Existence of multiple scales in uncertainty of numerical weather prediction

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Hyo-Jong Song
Keyword(s):  
Numerical Weather Prediction ◽  
Short Range ◽  
Multiple Scales ◽  
Forecast Error ◽  
Weather Prediction ◽  
Initial Condition ◽  
Forecast Uncertainty ◽  
Essential Information ◽  
Numerical Weather ◽  
Sample Correlation

Abstract Numerical weather prediction provides essential information of societal influence. Advances in the initial condition estimation have led to the improvement of the prediction skill. The process to produce the better initial condition (analysis) with the combination of short-range forecast and observation over the globe requires information about uncertainty of the forecast results to decide how much observation is reflected to the analysis and how far the observation information should be propagated. Forecast ensemble represents the error of the short-range forecast at the instance. The influence of observation propagating along with forecast ensemble correlation needs to be restricted by localized correlation function because of less reliability of sample correlation. So far, solitary radius of influence is usually used since there has not been an understanding about the realism of multiple scales in the forecast uncertainty. In this study, it is explicitly shown that multiple scales exist in short-range forecast error and any single-scale localization approach could not resolve this situation. A combination of Gaussian correlation functions of various scales is designed, which more weighs observation itself near the data point and makes ensemble perturbation, far from the observation position, more participate in decision of the analysis. Its outstanding performance supports the existence of multi-scale correlation in forecast uncertainty.

scholarly journals 2018 Atmospheric Motion Vector (AMV) Intercomparison Study

2019 ◽  
Vol 11 (19) ◽  
pp. 2240 ◽  
Author(s):  
David Santek ◽  
Richard Dworak ◽  
Sharon Nebuda ◽  
Steve Wanzong ◽  
Régis Borde ◽  
...  
Keyword(s):  
Weather Prediction ◽  
Motion Vector ◽  
Estimation Method ◽  
Japan Meteorological Agency ◽  
Radiative Transfer Model ◽  
Transfer Model ◽  
European Organization ◽  
Specific Configuration ◽  
Atmospheric Motion ◽  
Height Assignment

Atmospheric Motion Vectors (AMVs) calculated by six different institutions (Brazil Center for Weather Prediction and Climate Studies/CPTEC/INPE, European Organization for the Exploitation of Meteorological Satellites/EUMETSAT, Japan Meteorological Agency/JMA, Korea Meteorological Administration/KMA, Unites States National Oceanic and Atmospheric Administration/NOAA, and the Satellite Application Facility on Support to Nowcasting and Very short range forecasting/NWCSAF) with JMA’s Himawari-8 satellite data and other common input data are here compared. The comparison is based on two different AMV input datasets, calculated with two different image triplets for 21 July 2016, and the use of a prescribed and a specific configuration. The main results of the study are summarized as follows: (1) the differences in the AMV datasets depend very much on the ‘AMV height assignment’ used and much less on the use of a prescribed or specific configuration; (2) the use of the ‘Common Quality Indicator (CQI)’ has a quantified skill in filtering collocated AMVs for an improved statistical agreement between centers; (3) Among the six AMV operational algorithms verified by this AMV Intercomparison, JMA AMV algorithm has the best overall performance considering all validation metrics, mainly due to its new height assignment method: ‘Optimal estimation method considering the observed infrared radiances, the vertical profile of the Numerical Weather Prediction wind, and the estimated brightness temperature using a radiative transfer model’.

scholarly journals Some Experimental Lessons on Digital Filtering in the ALADIN-France 3DVAR Based on Near-Ground Examination

2011 ◽  
Vol 139 (3) ◽  
pp. 774-785 ◽  
Author(s):  
Claude Fischer ◽  
Ludovic Auger
Keyword(s):  
Data Assimilation ◽  
Digital Filter ◽  
Three Dimensional ◽  
Weather Forecast ◽  
Forecast Model ◽  
Horizontal Wind ◽  
Screen Level ◽  
Incremental Formulation ◽  
Weather Forecast Model ◽  
Observation Impact

Abstract This paper deals with the characteristics and effects of digital filter initialization, as implemented in the operational three-dimensional variational data assimilation (3DVAR) system of the Aire Limitée Adaptation Dynamique Développement International (ALADIN)-France regional weather forecast model. First, a series of findings on the properties of the initialization of the model are discussed. Examples of initial spinup linked with inertia–gravity wave occurrence are shown, and the major sources for their generation are listed. These experimental results are compared with past and present experiences concerning the use and need for digital filter initialization. Furthermore, the impacts of switching to an incremental formulation of the filter in data assimilation mode are demonstrated. Second, the effects of the filter formulation on the results of an observation impact study are illustrated. The latter consists of implementing screen-level, 10-m horizontal wind information into the ALADIN 3DVAR analysis. There can, indeed, be some delicate interference between observation impact evaluation and the effects of filtering, at least on short-term forecasts. The paper is concluded with some general considerations on the experimental evaluation of spinup and the link between the assimilation system design and model state filtering.

scholarly journals Linear Filtering of Sample Covariances for Ensemble-Based Data Assimilation. Part I: Optimality Criteria and Application to Variance Filtering and Covariance Localization

2015 ◽  
Vol 143 (5) ◽  
pp. 1622-1643 ◽  
Author(s):  
Benjamin Ménétrier ◽  
Thibaut Montmerle ◽  
Yann Michel ◽  
Loïk Berre
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Weather Prediction ◽  
Computational Cost ◽  
Analytical Framework ◽  
Companion Paper ◽  
Optimality Criteria ◽  
Linear Filtering ◽  
Sampling Errors ◽  
Output Only

Abstract In data assimilation (DA) schemes for numerical weather prediction (NWP) systems, the estimation of forecast error covariances is a key point to get some flow dependency. As shown in previous studies, ensemble data assimilation methods are the most accurate for this task. However, their huge computational cost raises a strong limitation to the ensemble size. Consequently, covariances estimated with small ensembles are affected by random sampling errors. The aim of this study is to develop a theory of covariance filtering in order to remove most of the sampling noise while keeping the signal of interest and then to use it in the DA scheme of a real NWP system. This first part of a two-part study presents the theoretical aspects of such criteria for optimal filtering based on the merging of the theories of optimal linear filtering and of sample centered moments estimation. Its strength relies on the use of sample estimated quantities and filter output only. These criteria pave the way for new algorithms and interesting applications for NWP. Two of them are detailed here: spatial filtering of variances and covariance localization. Results obtained in an idealized 1D analytical framework are shown for illustration. Applications on real forecast error covariances deduced from ensembles at convective scale are discussed in a companion paper.

scholarly journals Diagnosing Summertime Mesoscale Vertical Motion: Implications for Atmospheric Data Assimilation

2007 ◽  
Vol 135 (6) ◽  
pp. 2076-2094 ◽  
Author(s):  
Christian Pagé ◽  
Luc Fillion ◽  
Peter Zwack
Keyword(s):  
Data Assimilation ◽  
Weather Prediction ◽  
Control Variable ◽  
Vertical Motion ◽  
Forecast Model ◽  
Horizontal Resolution ◽  
Limited Area ◽  
Significant Information ◽  
Special Procedure ◽  
Atmospheric Data

Abstract Balance omega equations have recently been used to try to improve the characterization of balance in variational data assimilation schemes for numerical weather prediction (NWP). Results from Fisher and Fillion et al. indicate that a quasigeostrophic omega equation can be used adequately in the definition of the control variable to represent synoptic-scale balanced vertical motion. For high-resolution limited-area data assimilation and forecasting (1–10-km horizontal resolution), such a diagnostic equation for vertical motion needs to be revisited. Using a state-of-the-art NWP forecast model at 2.5-km horizontal resolution, these issues are examined. Starting from a complete diagnostic partial differential equation for omega, the rhs forcing terms were computed from model-generated fields. These include the streamfunction, temperature, and physical time tendencies of temperature in gridpoint space. To accurately compute one term of second-order importance (i.e., the ageostrophic vorticity tendency forcing term), a special procedure was used. With this procedure it is shown that Charney’s balance equation brings significant information in order to deduce the geostrophic time tendency term. Under these conditions, results show that for phenomena of length scales of 15–100 km over convective regions, a diagnostic equation can capture the major part of the model-generated vertical motion. The limitations of the digital filter initialization approach when used as in Fillion et al. with a cutoff period reduced to 1 h are also illustrated. The potential usefulness of this study for mesoscale atmospheric data assimilation is briefly discussed.

scholarly journals Comparing Two Approaches for Assessing Observation Impact

2013 ◽  
Vol 141 (5) ◽  
pp. 1484-1505 ◽  
Author(s):  
Ricardo Todling
Keyword(s):  
Data Assimilation ◽  
State Space ◽  
Adjoint Operator ◽  
Additional Advantage ◽  
Earth Observing System ◽  
Natural Choice ◽  
Observation Impact ◽  
Observation Space ◽  
The Impact ◽  
Space Approach

Abstract Langland and Baker introduced an approach to assess the impact of observations on the forecasts. In that approach, a state-space aspect of the forecast is defined and a procedure is derived ultimately relating changes in the aspect with changes in the observing system. Some features of the state-space approach are to be noted: the typical choice of forecast aspect is rather subjective and leads to incomplete assessment of the observing system, it requires availability of a verification state that is in practice correlated with the forecast, and it involves the adjoint operator of the entire data assimilation system and is thus constrained by the validity of this operator. This article revisits the topic of observation impacts from the perspective of estimation theory. An observation-space metric is used to allow inferring observation impact on the forecasts without the limitations just mentioned. Using differences of observation-minus-forecast residuals obtained from consecutive forecasts leads to the following advantages: (i) it suggests a rather natural choice of forecast aspect that directly links to the data assimilation procedure, (ii) it avoids introducing undesirable correlations in the forecast aspect since verification is done against the observations, and (iii) it does not involve linearization and use of adjoints. The observation-space approach has the additional advantage of being nearly cost free and very simple to implement. In its simplest form it reduces to evaluating the statistics of observation-minus-background and observation-minus-analysis residuals with traditional methods. Illustrations comparing the approaches are given using the NASA Goddard Earth Observing System.

scholarly journals Forecast Sensitivity Observation Impact in the 4DVAR and Hybrid-4DVAR Data Assimilation Systems

2019 ◽  
Vol 36 (8) ◽  
pp. 1563-1575 ◽  
Author(s):  
Sung-Min Kim ◽  
Hyun Mee Kim
Keyword(s):  
Data Assimilation ◽  
East Asia ◽  
Forecast Error ◽  
Integration Time ◽  
Ensemble Forecasts ◽  
Background Error ◽  
Error Covariance ◽  
Atmospheric Motion ◽  
Observation Impact ◽  
Atmospheric Motion Vectors

AbstractIn this study, the observation impacts on 24-h forecast error reduction (FER), based on the adjoint method in the four-dimensional variational (4DVAR) data assimilation (DA) and hybrid-4DVAR DA systems coupled with the Unified Model, were evaluated from 0000 UTC 5 August to 1800 UTC 26 August 2014. The nonlinear FER in hybrid-4DVAR was 12.2% greater than that in 4DVAR due to the use of flow-dependent background error covariance (BEC), which was a weighted combination of the static BEC and the ensemble BEC based on ensemble forecasts. In hybrid-4DVAR, the observation impacts (i.e., the approximated nonlinear FER) for most observation types increase compared to those in 4DVAR. The increased observation impact from using hybrid-4DVAR instead of 4DVAR changes depending on the analysis time and regions. To calculate the ensemble BEC in hybrid-4DVAR, analyses at 0600 and 1800 UTC (0000 and 1200 UTC) used 3-h (9-h) ensemble forecasts. Greater observation impact was obtained when 3-h ensemble forecasts were used for the ensemble BEC at 0600 and 1800 UTC, than with 9-h ensemble forecasts at 0000 and 1200 UTC. Different from other observations, the atmospheric motion vectors (AMVs) deduced from geostationary satellite are more frequently observed in the same area. When the ensemble forecasts with longer integration times were used for the ensemble BEC in hybrid-4DVAR, the observation impact of the AMVs decreased the most in East Asia. This implies that the observation impact of AMVs in East Asia shows the highest sensitivity to the integration time of the ensemble members used for deducing the flow-dependent BEC in hybrid-4DVAR.

Sign in / Sign up

Export Citation Format

Share Document