Four-dimensional ensemble-variational data assimilation for global deterministic weather prediction

Abstract. The goal of this study is to evaluate a version of the ensemble-variational data assimilation approach (EnVar) for possible replacement of 4D-Var at Environment Canada for global deterministic weather prediction. This implementation of EnVar relies on 4-D ensemble covariances, obtained from an ensemble Kalman filter, that are combined in a vertically dependent weighted average with simple static covariances. Verification results are presented from a set of data assimilation experiments over two separate 6-week periods that used assimilated observations and model configuration very similar to the currently operational system. To help interpret the comparison of EnVar versus 4D-Var, additional experiments using 3D-Var and a version of EnVar with only 3-D ensemble covariances are also evaluated. To improve the rate of convergence for all approaches evaluated (including EnVar), an estimate of the cost function Hessian generated by the quasi-Newton minimization algorithm is cycled from one analysis to the next. Analyses from EnVar (with 4-D ensemble covariances) nearly always produce improved, and never degraded, forecasts when compared with 3D-Var. Comparisons with 4D-Var show that forecasts from EnVar analyses have either similar or better scores in the troposphere of the tropics and the winter extra-tropical region. However, in the summer extra-tropical region the medium-range forecasts from EnVar have either similar or worse scores than 4D-Var in the troposphere. In contrast, the 6 h forecasts from EnVar are significantly better than 4D-Var relative to radiosonde observations for both periods and in all regions. The use of 4-D versus 3-D ensemble covariances only results in small improvements in forecast quality. By contrast, the improvements from using 4D-Var versus 3D-Var are much larger. Measurement of the fit of the background and analyzed states to the observations suggests that EnVar and 4D-Var can both make better use of observations distributed over time than 3D-Var. In summary, the results from this study suggest that the EnVar approach is a viable alternative to 4D-Var, especially when the simplicity and computational efficiency of EnVar are considered. Additional research is required to understand the seasonal dependence of the difference in forecast quality between EnVar and 4D-Var in the extra-tropics.

Download Full-text

Multiplicative and Additive Incremental Variational Data Assimilation for Mixed Lognormal–Gaussian Errors

Monthly Weather Review ◽

10.1175/mwr-d-13-00136.1 ◽

2014 ◽

Vol 142 (7) ◽

pp. 2521-2544 ◽

Cited By ~ 18

Author(s):

Steven J. Fletcher ◽

Andrew S. Jones

Keyword(s):

Data Assimilation ◽

Lognormal Distribution ◽

Weather Prediction ◽

Variational Data Assimilation ◽

Incremental Formulation ◽

Incremental Scheme ◽

Observational Errors ◽

Mixed Approach ◽

The Cost ◽

Gaussian Space

Abstract An advance that made Gaussian-based three- and four-dimensional variational data assimilation (3D- and 4DVAR, respectively) operationally viable for numerical weather prediction was the introduction of the incremental formulation. This reduces the computational costs of the variational methods by searching for a small increment to a background state whose evolution is approximately linear. In this paper, incremental formulations for 3D- and 4DVAR with lognormal and mixed lognormal–Gaussian-distributed background and observation errors are presented. As the lognormal distribution has geometric properties, a geometric version for the tangent linear model (TLM) is proven that enables the linearization of the observational component of the cost functions with respect to a geometric increment. This is combined with the additive TLM for the mixed distribution–based cost function. Results using the mixed incremental scheme with the Lorenz’63 model are presented for different observational error variances, observation set sizes, and assimilation window lengths. It is shown that for sparse accurate observations the scheme has a relative error of ±0.5% for an assimilation window of 100 time steps. This improves to ±0.3% with more frequent observations. The distributions of the analysis errors are presented that appear to approximate a lognormal distribution with a mode at 1, which, given that the background and observational errors are unbiased in Gaussian space, shows that the scheme is approximating a mode and not a median. The mixed approach is also compared against a Gaussian-only incremental scheme where it is shown that as the z-component observational errors become more lognormal, the mixed approach appears to be more accurate than the Gaussian approach.

Download Full-text

First Application of the Local Ensemble Tangent Linear Model (LETLM) to a Realistic Model of the Global Atmosphere

Monthly Weather Review ◽

10.1175/mwr-d-17-0315.1 ◽

2018 ◽

Vol 146 (7) ◽

pp. 2247-2270 ◽

Cited By ~ 4

Author(s):

Sergey Frolov ◽

Douglas R. Allen ◽

Craig H. Bishop ◽

Rolf Langland ◽

Karl W. Hoppel ◽

...

Keyword(s):

Boundary Layer ◽

Data Assimilation ◽

Linear Model ◽

Weather Prediction ◽

Computational Cost ◽

Maximum Size ◽

Realistic Model ◽

Sensitivity Studies ◽

The Difference ◽

The Cost

Abstract The local ensemble tangent linear model (LETLM) provides an alternative method for creating the tangent linear model (TLM) and adjoint of a nonlinear model that promises to be easier to maintain and more computationally scalable than earlier methods. In this paper, we compare the ability of the LETLM to predict the difference between two nonlinear trajectories of the Navy’s global weather prediction model at low resolution (2.5° at the equator) with that of the TLM currently used in the Navy’s four-dimensional variational (4DVar) data assimilation scheme. When compared to the pair of nonlinear trajectories, the traditional TLM and the LETLM have improved skill relative to persistence everywhere in the atmosphere, except for temperature in the planetary boundary layer. In addition, the LETLM was, on average, more accurate than the traditional TLM (error reductions of about 20% in the troposphere and 10% overall). Sensitivity studies showed that the LETLM was most sensitive to the number of ensemble members, with the performance gradually improving with increased ensemble size up to the maximum size attempted (400). Inclusion of physics in the LETLM ensemble leads to a significantly improved representation of the boundary layer winds (error reductions of up to 50%), in addition to improved winds and temperature in the free troposphere and in the upper stratosphere/lower mesosphere. The computational cost of the LETLM was dominated by the cost of ensemble propagation. However, the LETLM can be precomputed before the 4DVar data assimilation algorithm is executed, leading to a significant computational advantage.

Download Full-text

Cloud and Precipitation Parameterizations in Modeling and Variational Data Assimilation: A Review

Journal of the Atmospheric Sciences ◽

10.1175/2006jas2030.1 ◽

2007 ◽

Vol 64 (11) ◽

pp. 3766-3784 ◽

Cited By ~ 32

Author(s):

Philippe Lopez

Keyword(s):

Data Assimilation ◽

General Circulation ◽

Large Scale ◽

Weather Prediction ◽

General Circulation Models ◽

Variational Data Assimilation ◽

Current Status ◽

General Question ◽

Observation Error ◽

Moist Processes

Abstract This paper first reviews the current status, issues, and limitations of the parameterizations of atmospheric large-scale and convective moist processes that are used in numerical weather prediction and climate general circulation models. Both large-scale (resolved) and convective (subgrid scale) moist processes are dealt with. Then, the general question of the inclusion of diabatic processes in variational data assimilation systems is addressed. The focus is put on linearity and resolution issues, the specification of model and observation error statistics, the formulation of the control vector, and the problems specific to the assimilation of observations directly affected by clouds and precipitation.

Download Full-text

Quasi-static ensemble variational data assimilation: a theoretical and numerical study with the iterative ensemble Kalman smoother

Nonlinear Processes in Geophysics ◽

10.5194/npg-25-315-2018 ◽

2018 ◽

Vol 25 (2) ◽

pp. 315-334 ◽

Cited By ~ 2

Author(s):

Anthony Fillion ◽

Marc Bocquet ◽

Serge Gratton

Keyword(s):

Data Assimilation ◽

Cost Function ◽

Global Minimum ◽

Numerical Study ◽

Variational Data Assimilation ◽

Kalman Smoother ◽

Local Extrema ◽

Starting Point ◽

The Cost ◽

Temporal Extent

Abstract. The analysis in nonlinear variational data assimilation is the solution of a non-quadratic minimization. Thus, the analysis efficiency relies on its ability to locate a global minimum of the cost function. If this minimization uses a Gauss–Newton (GN) method, it is critical for the starting point to be in the attraction basin of a global minimum. Otherwise the method may converge to a local extremum, which degrades the analysis. With chaotic models, the number of local extrema often increases with the temporal extent of the data assimilation window, making the former condition harder to satisfy. This is unfortunate because the assimilation performance also increases with this temporal extent. However, a quasi-static (QS) minimization may overcome these local extrema. It accomplishes this by gradually injecting the observations in the cost function. This method was introduced by Pires et al. (1996) in a 4D-Var context. We generalize this approach to four-dimensional strong-constraint nonlinear ensemble variational (EnVar) methods, which are based on both a nonlinear variational analysis and the propagation of dynamical error statistics via an ensemble. This forces one to consider the cost function minimizations in the broader context of cycled data assimilation algorithms. We adapt this QS approach to the iterative ensemble Kalman smoother (IEnKS), an exemplar of nonlinear deterministic four-dimensional EnVar methods. Using low-order models, we quantify the positive impact of the QS approach on the IEnKS, especially for long data assimilation windows. We also examine the computational cost of QS implementations and suggest cheaper algorithms.

Download Full-text

Scale-Dependent Representation of the Information Content of Observations in the Global Ensemble Kalman Filter Data Assimilation

Monthly Weather Review ◽

10.1175/mwr-d-15-0401.1 ◽

2016 ◽

Vol 144 (8) ◽

pp. 2927-2945

Author(s):

Nedjeljka Žagar ◽

Jeffrey Anderson ◽

Nancy Collins ◽

Timothy Hoar ◽

Kevin Raeder ◽

...

Keyword(s):

Kalman Filter ◽

Data Assimilation ◽

Information Content ◽

Ensemble Kalman Filter ◽

Large Scale ◽

Variance Reduction ◽

Weather Prediction ◽

Current Data ◽

Model Framework ◽

The Tropics

Abstract Global data assimilation systems for numerical weather prediction (NWP) are characterized by significant uncertainties in tropical analysis fields. Furthermore, the largest spread of global ensemble forecasts in the short range on all scales is in the tropics. The presented results suggest that these properties hold even in the perfect-model framework and the ensemble Kalman filter data assimilation with a globally homogeneous network of wind and temperature profiles. The reasons for this are discussed by using the modal analysis, which provides information about the scale dependency of analysis and forecast uncertainties and information about the efficiency of data assimilation to reduce the prior uncertainties in the balanced and inertio-gravity dynamics. The scale-dependent representation of variance reduction of the prior ensemble by the data assimilation shows that the peak efficiency of data assimilation is on the synoptic scales in the midlatitudes that are associated with quasigeostrophic dynamics. In contrast, the variance associated with the inertia–gravity modes is less successfully reduced on all scales. A smaller information content of observations on planetary scales with respect to the synoptic scales is discussed in relation to the large-scale tropical uncertainties that current data assimilation methodologies do not address successfully. In addition, it is shown that a smaller reduction of the large-scale uncertainties in the prior state for NWP in the tropics than in the midlatitudes is influenced by the applied radius for the covariance localization.

Download Full-text

Cycling the Representer Algorithm for Variational Data Assimilation with the Lorenz Attractor

Monthly Weather Review ◽

10.1175/mwr3281.1 ◽

2007 ◽

Vol 135 (2) ◽

pp. 373-386 ◽

Cited By ~ 9

Author(s):

H. E. Ngodock ◽

S. R. Smith ◽

G. A. Jacobs

Keyword(s):

Data Assimilation ◽

Variational Data Assimilation ◽

Lorenz Attractor ◽

Strong Constraint ◽

Strongly Nonlinear ◽

Time Period ◽

Representer Method ◽

The Cost ◽

Solution Accuracy ◽

Perturbation Magnitude

Abstract Realistic dynamic systems are often strongly nonlinear, particularly those for the ocean and atmosphere. Applying variational data assimilation to these systems requires a tangent linearization of the nonlinear dynamics about a background state for the cost function minimization. The tangent linearization may be accurate for limited time scales. Here it is proposed that linearized assimilation systems may be accurate if the assimilation time period is less than the tangent linear accuracy time limit. In this paper, the cycling representer method is used to test this assumption with the Lorenz attractor. The outer loops usually required to accommodate the linear assimilation for a nonlinear problem may be dropped beyond the early cycles once the solution (and forecast used as the background in the tangent linearization) is sufficiently accurate. The combination of cycling the representer method and limiting the number of outer loops significantly lowers the cost of the overall assimilation problem. In addition, this study shows that weak constraint assimilation corrects tangent linear model inaccuracies and allows extension of the limited assimilation period. Hence, the weak constraint outperforms the strong constraint method. Assimilated solution accuracy at the first cycle end is computed as a function of the initial condition error, model parameter perturbation magnitude, and outer loops. Results indicate that at least five outer loops are needed to achieve solution accuracy in the first cycle for the selected error range. In addition, this study clearly shows that one outer loop in the first cycle does not preclude accuracy convergence in future cycles.

Download Full-text

Mapping of Snow Depth by Blending Satellite and In-Situ Data Using Two-Dimensional Optimal Interpolation—Application to AMSR2

Remote Sensing ◽

10.3390/rs11243049 ◽

2019 ◽

Vol 11 (24) ◽

pp. 3049 ◽

Cited By ~ 1

Author(s):

Cezar Kongoli ◽

Jeffrey Key ◽

Thomas M. Smith

Keyword(s):

North America ◽

Snow Depth ◽

Weather Prediction ◽

Weighted Average ◽

Optimal Interpolation ◽

Horizontal Distance ◽

Two Dimensional ◽

In Situ Data ◽

The Difference

The development of a snow depth product over North America is investigated by applying two-dimensional optimal interpolation to passive microwave satellite-derived and in-situ measured snow depth. At each snow-covered satellite footprint, the technique computes a snow depth increment as the weighted average of data increments, and updates the satellite-derived snow depth accordingly. Data increments are computed as the difference between the in-situ-measured and satellite snow depth at station locations surrounding the satellite footprint. Calculation of optimal weights is based on spatial lag autocorrelation of snow depth increments, modelled as functions of horizontal distance and elevation difference between pairs of observations. The technique is applied to Advanced Microwave Scanning Radiometer 2 (AMSR2) snow depth and in-situ snow depth obtained from the Global Historical Climatology Network. The results over North America during January–February 2017 indicate that the technique greatly enhances the performance of the satellite estimates, especially over mountain terrain, albeit with an accuracy inferior to that over low elevation areas. Moreover, the technique generates more accurate output compared to that from NOAA’s Global Forecast System, with implications for improving the utilization of satellite data in snow assessments and numerical weather prediction.

Download Full-text

Optimal solution error covariance in highly nonlinear problems of variational data assimilation

Nonlinear Processes in Geophysics ◽

10.5194/npg-19-177-2012 ◽

2012 ◽

Vol 19 (2) ◽

pp. 177-184 ◽

Cited By ~ 8

Author(s):

V. Shutyaev ◽

I. Gejadze ◽

G. J. M. Copeland ◽

F.-X. Le Dimet

Keyword(s):

Nonlinear Dynamics ◽

Data Assimilation ◽

Optimal Solution ◽

Nonlinear Problems ◽

Variational Data Assimilation ◽

Model Parameters ◽

Efficient Computation ◽

Viscous Term ◽

Highly Nonlinear ◽

The Cost

Abstract. The problem of variational data assimilation (DA) for a nonlinear evolution model is formulated as an optimal control problem to find the initial condition, boundary conditions and/or model parameters. The input data contain observation and background errors, hence there is an error in the optimal solution. For mildly nonlinear dynamics, the covariance matrix of the optimal solution error can be approximated by the inverse Hessian of the cost function. For problems with strongly nonlinear dynamics, a new statistical method based on the computation of a sample of inverse Hessians is suggested. This method relies on the efficient computation of the inverse Hessian by means of iterative methods (Lanczos and quasi-Newton BFGS) with preconditioning. Numerical examples are presented for the model governed by the Burgers equation with a nonlinear viscous term.

Download Full-text

Some Aspects of Sensitivity Analysis in Variational Data Assimilation for Coupled Dynamical Systems

Advances in Meteorology ◽

10.1155/2015/753031 ◽

2015 ◽

Vol 2015 ◽

pp. 1-22 ◽

Cited By ~ 4

Author(s):

Sergei Soldatenko ◽

Peter Steinle ◽

Chris Tingwell ◽

Denis Chichkine

Keyword(s):

Sensitivity Analysis ◽

Dynamical Systems ◽

Data Assimilation ◽

Weather Prediction ◽

Computational Cost ◽

Variational Data Assimilation ◽

Practical Implementation ◽

Parameter Uncertainties ◽

Sensitivity Functions ◽

Sensitivity Analysis Method

Variational data assimilation (VDA) remains one of the key issues arising in many fields of geosciences including the numerical weather prediction. While the theory of VDA is well established, there are a number of issues with practical implementation that require additional consideration and study. However, the exploration of VDA requires considerable computational resources. For simple enough low-order models, the computational cost is minor and therefore models of this class are used as simple test instruments to emulate more complex systems. In this paper, the sensitivity with respect to variations in the parameters of one of the main components of VDA, the nonlinear forecasting model, is considered. For chaotic atmospheric dynamics, conventional methods of sensitivity analysis provide uninformative results since the envelopes of sensitivity functions grow with time and sensitivity functions themselves demonstrate the oscillating behaviour. The use of sensitivity analysis method, developed on the basis of the theory of shadowing pseudoorbits in dynamical systems, allows us to calculate sensitivity functions correctly. Sensitivity estimates for a simple coupled dynamical system are calculated and presented in the paper. To estimate the influence of model parameter uncertainties on the forecast, the relative error in the energy norm is applied.

Download Full-text