scholarly journals Linear Filtering of Sample Covariances for Ensemble-Based Data Assimilation. Part I: Optimality Criteria and Application to Variance Filtering and Covariance Localization

2015 ◽  
Vol 143 (5) ◽  
pp. 1622-1643 ◽  
Author(s):  
Benjamin Ménétrier ◽  
Thibaut Montmerle ◽  
Yann Michel ◽  
Loïk Berre
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Weather Prediction ◽  
Computational Cost ◽  
Analytical Framework ◽  
Companion Paper ◽  
Optimality Criteria ◽  
Linear Filtering ◽  
Sampling Errors ◽  
Output Only

Abstract In data assimilation (DA) schemes for numerical weather prediction (NWP) systems, the estimation of forecast error covariances is a key point to get some flow dependency. As shown in previous studies, ensemble data assimilation methods are the most accurate for this task. However, their huge computational cost raises a strong limitation to the ensemble size. Consequently, covariances estimated with small ensembles are affected by random sampling errors. The aim of this study is to develop a theory of covariance filtering in order to remove most of the sampling noise while keeping the signal of interest and then to use it in the DA scheme of a real NWP system. This first part of a two-part study presents the theoretical aspects of such criteria for optimal filtering based on the merging of the theories of optimal linear filtering and of sample centered moments estimation. Its strength relies on the use of sample estimated quantities and filter output only. These criteria pave the way for new algorithms and interesting applications for NWP. Two of them are detailed here: spatial filtering of variances and covariance localization. Results obtained in an idealized 1D analytical framework are shown for illustration. Applications on real forecast error covariances deduced from ensembles at convective scale are discussed in a companion paper.

scholarly journals Linear Filtering of Sample Covariances for Ensemble-Based Data Assimilation. Part II: Application to a Convective-Scale NWP Model

2015 ◽  
Vol 143 (5) ◽  
pp. 1644-1664 ◽  
Author(s):  
Benjamin Ménétrier ◽  
Thibaut Montmerle ◽  
Yann Michel ◽  
Loïk Berre
Keyword(s):  
Data Assimilation ◽  
Weather Prediction ◽  
Computational Cost ◽  
Localization Length ◽  
Optimality Criteria ◽  
Global Scale ◽  
Linear Filtering ◽  
Background Error ◽  
Full Generality ◽  
Convective Scale

Abstract In Part I of this two-part study, a new theory for optimal linear filtering of covariances sampled from an ensemble of forecasts was detailed. This method, especially designed for data assimilation (DA) schemes in numerical weather prediction (NWP) systems, has the advantage of using optimality criteria that involve sample estimated quantities and filter output only. In this second part, the theory is tested with real background error covariances computed using a large ensemble data assimilation (EDA) at the convective scale coupled with a large EDA at the global scale, based respectively on the Applications of Research to Operations at Mesoscale (AROME) and ARPEGE operational NWP systems. Background error variances estimated with a subset of this ensemble are filtered and evaluated against values obtained with the remaining members, which are considered as an independent reference. Algorithms presented in Part I show relevant results, with the homogeneous filtering being quasi optimal. Heterogeneous filtering is also successfully tested with different local criteria, yet at a higher computational cost, showing the full generality of the method. As a second application, horizontal and vertical localization functions are diagnosed from the ensemble, providing pertinent localization length scales that consistently depend on the number of members, on the meteorological variables, and on the vertical levels.

scholarly journals Southern High-Latitude Ensemble Data Assimilation in the Antarctic Mesoscale Prediction System

2005 ◽  
Vol 133 (12) ◽  
pp. 3431-3449 ◽  
Author(s):  
D. M. Barker
Keyword(s):  
Data Assimilation ◽  
Sampling Error ◽  
Forecast Error ◽  
Weather Prediction ◽  
Forecast Errors ◽  
Prediction System ◽  
Spatial Covariance ◽  
Ensemble Data Assimilation ◽  
Ensemble Data ◽  
The Antarctic

Abstract Ensemble data assimilation systems incorporate observations into numerical models via solution of the Kalman filter update equations, and estimates of forecast error covariances derived from ensembles of model integrations. In this paper, a particular algorithm, the ensemble square root filter (EnSRF), is tested in a limited-area, polar numerical weather prediction (NWP) model: the Antarctic Mesoscale Prediction System (AMPS). For application in the real-time AMPS, the number of model integrations that can be run to provide forecast error covariances is limited, resulting in an ensemble sampling error that degrades the analysis fit to observations. In this work, multivariate, climatologically plausible forecast error covariances are specified via averaged forecast difference statistics. Ensemble representations of the “true” forecast errors, created using randomized control variables of the fifth-generation Pennsylvania State University–National Center for Atmospheric Research (PSU–NCAR) Mesoscale Model (MM5) three-dimensional variational (3DVAR) data assimilation system, are then used to assess the dependence of sampling error on ensemble size, data density, and localization of covariances using simulated observation networks. Results highlight the detrimental impact of ensemble sampling error on the analysis increment structure of correlated, but unobserved fields—an issue not addressed by the spatial covariance localization techniques used to date. A 12-hourly cycling EnSRF/AMPS assimilation/forecast system is tested for a two-week period in December 2002 using real, conventional (surface, rawinsonde, satellite retrieval) observations. The dependence of forecast scores on methods used to maintain ensemble spread and the inclusion of perturbations to lateral boundary conditions are studied.

scholarly journals Assimilation of MODIS Dark Target and Deep Blue observations in the dust aerosol component of NMMB-MONARCH version 1.0

2017 ◽  
Vol 10 (3) ◽  
pp. 1107-1129 ◽  
Author(s):  
Enza Di Tomaso ◽  
Nick A. J. Schutgens ◽  
Oriol Jorba ◽  
Carlos Pérez García-Pando
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Mineral Dust ◽  
Positive Impact ◽  
Forecast Error ◽  
Weather Prediction ◽  
Prediction System ◽  
Deep Blue ◽  
Modis Aod ◽  
Sahel Region

Abstract. A data assimilation capability has been built for the NMMB-MONARCH chemical weather prediction system, with a focus on mineral dust, a prominent type of aerosol. An ensemble-based Kalman filter technique (namely the local ensemble transform Kalman filter – LETKF) has been utilized to optimally combine model background and satellite retrievals. Our implementation of the ensemble is based on known uncertainties in the physical parametrizations of the dust emission scheme. Experiments showed that MODIS AOD retrievals using the Dark Target algorithm can help NMMB-MONARCH to better characterize atmospheric dust. This is particularly true for the analysis of the dust outflow in the Sahel region and over the African Atlantic coast. The assimilation of MODIS AOD retrievals based on the Deep Blue algorithm has a further positive impact in the analysis downwind from the strongest dust sources of the Sahara and in the Arabian Peninsula. An analysis-initialized forecast performs better (lower forecast error and higher correlation with observations) than a standard forecast, with the exception of underestimating dust in the long-range Atlantic transport and degradation of the temporal evolution of dust in some regions after day 1. Particularly relevant is the improved forecast over the Sahara throughout the forecast range thanks to the assimilation of Deep Blue retrievals over areas not easily covered by other observational datasets. The present study on mineral dust is a first step towards data assimilation with a complete aerosol prediction system that includes multiple aerosol species.

scholarly journals Some Aspects of Sensitivity Analysis in Variational Data Assimilation for Coupled Dynamical Systems

2015 ◽  
Vol 2015 ◽  
pp. 1-22 ◽  
Author(s):  
Sergei Soldatenko ◽  
Peter Steinle ◽  
Chris Tingwell ◽  
Denis Chichkine
Keyword(s):  
Sensitivity Analysis ◽  
Dynamical Systems ◽  
Data Assimilation ◽  
Weather Prediction ◽  
Computational Cost ◽  
Variational Data Assimilation ◽  
Practical Implementation ◽  
Parameter Uncertainties ◽  
Sensitivity Functions ◽  
Sensitivity Analysis Method

Variational data assimilation (VDA) remains one of the key issues arising in many fields of geosciences including the numerical weather prediction. While the theory of VDA is well established, there are a number of issues with practical implementation that require additional consideration and study. However, the exploration of VDA requires considerable computational resources. For simple enough low-order models, the computational cost is minor and therefore models of this class are used as simple test instruments to emulate more complex systems. In this paper, the sensitivity with respect to variations in the parameters of one of the main components of VDA, the nonlinear forecasting model, is considered. For chaotic atmospheric dynamics, conventional methods of sensitivity analysis provide uninformative results since the envelopes of sensitivity functions grow with time and sensitivity functions themselves demonstrate the oscillating behaviour. The use of sensitivity analysis method, developed on the basis of the theory of shadowing pseudoorbits in dynamical systems, allows us to calculate sensitivity functions correctly. Sensitivity estimates for a simple coupled dynamical system are calculated and presented in the paper. To estimate the influence of model parameter uncertainties on the forecast, the relative error in the energy norm is applied.

scholarly journals Methods of investigating forecast error sensitivity to ensemble size in a limited-area convection-permitting ensemble

2017 ◽  
Author(s):  
Ross Noel Bannister ◽  
Stefano Migliorini ◽  
Alison Clare Rudd ◽  
Laura Hart Baker
Keyword(s):  
Data Assimilation ◽  
Weather Forecasting ◽  
Sampling Error ◽  
Forecast Error ◽  
Significant Degree ◽  
Limited Area ◽  
Ensemble Size ◽  
Sampling Errors ◽  
Convective Scale

Abstract. Ensemble-based predictions are increasingly used as an aid to weather forecasting and to data assimilation, where the aim is to capture the range of possible outcomes consistent with the underlying uncertainties. Constraints on computing resources mean that ensembles have a relatively small size, which can lead to an incomplete range of possible outcomes, and to inherent sampling errors. This paper discusses how an existing ensemble can be relatively easily increased in size, it develops a range of standard and extended diagnostics to help determine whether a given ensemble is large enough to be useful for forecasting and data assimilation purposes, and it applies the diagnostics to a convective-scale case study for illustration. Diagnostics include the effect of ensemble size on various aspects of rainfall forecasts, kinetic energy spectra, and (co)-variance statistics in the spatial and spectral domains. The work here extends the Met Office's 24 ensemble members to 93. It is found that the extra members do develop a significant degree of linear independence, they increase the ensemble spread (although with caveats to do with non-Gaussianity), they reduce sampling error in many statistical quantities (namely variances, correlations, and length-scales), and improve the effective spatial resolution of the ensemble. The extra members though do not improve the probabilistic rain rate forecasts. It is assumed that the 93-member ensemble approximates the error-free statistics, which is a practical assumption, but the data suggests that this number of members is ultimately not enough to justify this assumption, and therefore more ensembles are likely required for such convective-scale systems to further reduce sampling errors, especially for ensemble data assimilation purposes.

scholarly journals Regionally Enhanced Global (REG) 4D-Var

2018 ◽  
Vol 146 (12) ◽  
pp. 4015-4038
Author(s):  
Michael A. Herrera ◽  
Istvan Szunyogh ◽  
Adam Brainard ◽  
David D. Kuhl ◽  
Karl Hoppel ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Prediction Models ◽  
Initial Conditions ◽  
Weather Prediction ◽  
Computational Cost ◽  
Limited Area ◽  
Atmospheric Flow ◽  
Forecast Performance ◽  
Limited Area Model ◽  
Area Model

Abstract A regionally enhanced global (REG) data assimilation (DA) method is proposed. The technique blends high-resolution model information from a single or multiple limited-area model domains with global model and observational information to create a regionally enhanced analysis of the global atmospheric state. This single analysis provides initial conditions for both the global and limited-area model forecasts. The potential benefits of the approach for operational data assimilation are (i) reduced development cost, (ii) reduced overall computational cost, (iii) improved limited-area forecast performance from the use of global information about the atmospheric flow, and (iv) improved global forecast performance from the use of more accurate model information in the limited-area domains. The method is tested by an implementation on the U.S. Navy’s four-dimensional variational global data assimilation system and global and limited-area numerical weather prediction models. The results of the monthlong forecast experiments suggest that the REG DA approach has the potential to deliver the desired benefits.

Adjoint-Based Observation Impact Estimation with Direct Verification Using Forward Calculation

2018 ◽  
Vol 146 (9) ◽  
pp. 2837-2858 ◽  
Author(s):  
Toshiyuki Ishibashi
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Weather Prediction ◽  
Adjoint Operator ◽  
Estimation Method ◽  
Forecast Model ◽  
Japan Meteorological Agency ◽  
Essential Information ◽  
Observation Impact ◽  
Forward Calculation

Abstract The adjoint-based observation impact estimation method has been providing essential information to improve data assimilation systems (DASs) in numerical weather prediction (NWP). This paper has two purposes. The first is to verify the four approximations used in the method: iterative construction of the Kalman gain adjoint operator, the tangent linear (TL) approximation of a forecast model, ignoring cross terms of observation impacts, and approximations for incremental DASs. The second is to add new information to our knowledge of observation impacts. For the verification of the adjoint-based method, we use the TL-based observation impact estimation method that can calculate the same quantity as the adjoint-based method without adjoint calculations. Results of these verifications and observation impact estimations performed on the global NWP system of the Japan Meteorological Agency are as follows. First, observation impacts calculated using the adjoint-based method agree well with those from the TL-based method, with the correlation coefficient between them exceeding 0.97. Second, estimated observation impacts are consistent with previous studies in many aspects. There are also system-dependent properties, such as relatively small impacts from GPS radio occultation data above 13 km. Furthermore, new aspects of observation impacts are found: 1) the probability density function of the observation impact agrees well with the scalar theory when giving the experimental value of the observation and the forecast error standard deviations; 2) later observations in the data assimilation window have larger positive impacts; and 3) impacts of AMSU-A in the Southern Hemisphere are more than double those in the Northern Hemisphere.

scholarly journals Some Observing System Simulation Experiments with a Hybrid 3DEnVAR System for Storm-Scale Radar Data Assimilation

2014 ◽  
Vol 142 (9) ◽  
pp. 3326-3346 ◽  
Author(s):  
Jidong Gao ◽  
David J. Stensrud
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Control Variable ◽  
Simulated Data ◽  
Variational Data Assimilation ◽  
Sampling Errors ◽  
Advanced Regional Prediction System ◽  
Sensitivity Experiments ◽  
Variable Approach ◽  
Regional Prediction

A hybrid three-dimensional ensemble–variational data assimilation (3DEnVAR) algorithm is developed based on the 3D variational data assimilation (3DVAR) and ensemble Kalman filter (EnKF) programs with the Advanced Regional Prediction System (ARPS). The method uses the extended control variable approach to combine the static and ensemble-derived flow-dependent forecast error covariances. The method is applied to the assimilation of simulated data from two radars for a supercell storm. Some sensitivity experiments are performed to answer questions about how flow-dependent covariance estimated from the forecast ensemble can be best used in the hybrid 3DEnVAR scheme. When the ensemble size is relatively small (with 5 or 10 ensemble members), it is found that experiments with a weaker weighting value for the ensemble covariance leads to better analysis results. Even when severe sampling errors exist, introducing ensemble-estimated covariances into the variational method still benefits the analysis. For reasonably large ensemble sizes (50–100 members), a stronger relative weighting (>0.8) for the ensemble covariance leads to better analyses from the hybrid 3DEnVAR. In addition, the sensitivity experiments also indicate that the best results are obtained when the number of the augmented control variables is a function of three spatial dimensions and ensemble members, and is the same for all analysis variables.

scholarly journals Prediction of Lake-Effect Snow Using Convection-Allowing Ensemble Forecasts and Regional Data Assimilation

2017 ◽  
Vol 32 (5) ◽  
pp. 1727-1744 ◽  
Author(s):  
Seth Saslo ◽  
Steven J. Greybush
Keyword(s):  
Data Assimilation ◽  
Great Lakes ◽  
Forecast Error ◽  
Weather Prediction ◽  
The United States ◽  
Ensemble Prediction ◽  
Great Lakes Region ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Lake Effect

Abstract Lake-effect snow (LES) is a cold-season mesoscale convective phenomenon that can lead to significant snowfall rates and accumulations in the Great Lakes region of the United States. While limited-area numerical weather prediction models have shown skill in prediction of warm-season convective storms, forecasting the sharp nature of LES precipitation timing, intensity, and location is difficult because of model error and initial and boundary condition uncertainties. Ensemble forecasting can incorporate and quantify some sources of forecast error, but ensemble design must be considered. This study examines the relative contributions of forecast uncertainties to LES forecast error using a regional convection-allowing data assimilation and ensemble prediction system. Ensembles are developed using various methods of perturbations to simulate a long-lived and high-precipitation LES event in December 2013, and forecast performance is evaluated using observations including those from the Ontario Winter Lake-Effect Systems (OWLeS) campaign. Model lateral boundary conditions corresponding to weather conditions beyond the Great Lakes region play an influential role in LES precipitation forecasts and their uncertainty, as evidenced by ensemble spread, particularly at lead times beyond one day. A strong forecast dependence on regional initial conditions was shown using data assimilation. This sensitivity impacts the timing and intensity of predicted precipitation, as well as band location and orientation assessed with an object-based verification approach, giving insight into the time scales of practical predictability of LES. Overall, an assimilation-cycling convection-allowing ensemble prediction system could improve future lake-effect snow precipitation forecasts and analyses and can help quantify and understand sources of forecast uncertainty.

scholarly journals First Application of the Local Ensemble Tangent Linear Model (LETLM) to a Realistic Model of the Global Atmosphere

2018 ◽  
Vol 146 (7) ◽  
pp. 2247-2270 ◽  
Author(s):  
Sergey Frolov ◽  
Douglas R. Allen ◽  
Craig H. Bishop ◽  
Rolf Langland ◽  
Karl W. Hoppel ◽  
...  
Keyword(s):  
Boundary Layer ◽  
Data Assimilation ◽  
Linear Model ◽  
Weather Prediction ◽  
Computational Cost ◽  
Maximum Size ◽  
Realistic Model ◽  
Sensitivity Studies ◽  
The Difference ◽  
The Cost

Abstract The local ensemble tangent linear model (LETLM) provides an alternative method for creating the tangent linear model (TLM) and adjoint of a nonlinear model that promises to be easier to maintain and more computationally scalable than earlier methods. In this paper, we compare the ability of the LETLM to predict the difference between two nonlinear trajectories of the Navy’s global weather prediction model at low resolution (2.5° at the equator) with that of the TLM currently used in the Navy’s four-dimensional variational (4DVar) data assimilation scheme. When compared to the pair of nonlinear trajectories, the traditional TLM and the LETLM have improved skill relative to persistence everywhere in the atmosphere, except for temperature in the planetary boundary layer. In addition, the LETLM was, on average, more accurate than the traditional TLM (error reductions of about 20% in the troposphere and 10% overall). Sensitivity studies showed that the LETLM was most sensitive to the number of ensemble members, with the performance gradually improving with increased ensemble size up to the maximum size attempted (400). Inclusion of physics in the LETLM ensemble leads to a significantly improved representation of the boundary layer winds (error reductions of up to 50%), in addition to improved winds and temperature in the free troposphere and in the upper stratosphere/lower mesosphere. The computational cost of the LETLM was dominated by the cost of ensemble propagation. However, the LETLM can be precomputed before the 4DVar data assimilation algorithm is executed, leading to a significant computational advantage.

Sign in / Sign up

Export Citation Format

Share Document