scholarly journals A Study of Rain Forecast Error Structure Based on Radar Observations over a Continental-Scale Spatial Domain

2016 ◽  
Vol 144 (8) ◽  
pp. 2871-2887
Author(s):  
Majid Fekri ◽  
M. K. Yau
Keyword(s):  
Forecast Error ◽  
Error Variance ◽  
Precipitation Forecast ◽  
Local Error ◽  
Error Statistics ◽  
Continental Scale ◽  
Error Covariance ◽  
Hourly Rainfall ◽  
Dependent Error

Abstract This study examines the univariate error covariances of hourly rainfall accumulations using two different NWP models and a mosaic of radar reflectivity over a continental-scale domain. The study focuses on two main areas. The focus of the first part of the paper is on the ensemble-based and the innovation-based error variance and correlation estimations. An ensemble of forecasts and a set of observations provide the basis for estimating the errors in two different ways. The results indicate that both ensemble- and innovation-based methods lead to comparable variance estimations, while the local error correlation estimates have larger differences due to the sensitivity of calculations to the gradient of the variance field. The second part of the paper uses innovations for identifying the errors. The focus of this part is on a prognostic method for estimating the error statistics from the background based on the Bayesian inference technique. The case study shows that the predictive model produces a similar result regarding the magnitude and the dispersion of variance in comparison with the innovation and ensemble-based variances. This study represents a step toward estimating local error variances and local error correlations to construct a nonhomogeneous and precipitation-dependent error covariance matrix of rainfall. These results will be used in a future paper in the design of a 2D-VAR Assimilation Method for Blending Extrapolated Radars (AMBER) with NWP precipitation forecast to form a precipitation nowcasting model.

How is the Balance of a Forecast Ensemble Affected by Adaptive and Nonadaptive Localization Schemes?

2015 ◽  
Vol 143 (9) ◽  
pp. 3680-3699 ◽  
Author(s):  
Ross N. Bannister
Keyword(s):  
Forecast Error ◽  
Covariance Matrices ◽  
Error Statistics ◽  
Error Covariance ◽  
Localization Scheme ◽  
Adaptive Schemes ◽  
Convective Scale ◽  
Free Parameters ◽  
Hydrostatic Balance ◽  
First Time

Abstract This paper investigates the effect on balance of a number of Schur product–type localization schemes that have been designed with the primary function of reducing spurious far-field correlations in forecast error statistics. The localization schemes studied comprise a nonadaptive scheme (where the moderation matrix is decomposed in a spectral basis), and two adaptive schemes: a simplified version of Smoothed Ensemble Correlations Raised to a Power (SENCORP) and Ensemble Correlations Raised to a Power (ECO-RAP). The paper shows, the author believes for the first time, how the degree of balance (geostrophic and hydrostatic) implied by the error covariance matrices localized by these schemes can be diagnosed. Here it is considered that an effective localization scheme is one that reduces spurious correlations adequately, but also minimizes disruption of balance (where the “correct” degree of balance or imbalance is assumed to be possessed by the unlocalized ensemble). By varying free parameters that describe each scheme (e.g., the degree of truncation in the schemes that use the spectral basis, the “order” of each scheme, and the degree of ensemble smoothing), it is found that a particular configuration of the ECO-RAP scheme is best suited to the convective-scale system studied. According to the diagnostics this ECO-RAP configuration still weakens geostrophic and hydrostatic balance, but overall this is less so than for other schemes.

scholarly journals Efficient Adaptive Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to the Control of Ocean Mesoscale Signals

2010 ◽  
Vol 138 (3) ◽  
pp. 932-950 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Error Estimates ◽  
Forecast Error ◽  
Observation Error ◽  
Square Root ◽  
Error Statistics ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Optimal Estimates

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

scholarly journals Regional Ensemble Forecasts Using the Ensemble Transform Technique

2009 ◽  
Vol 137 (1) ◽  
pp. 288-298 ◽  
Author(s):  
Craig H. Bishop ◽  
Teddy R. Holt ◽  
Jason Nachamkin ◽  
Sue Chen ◽  
Justin G. McLay ◽  
...  
Keyword(s):  
Forecast Error ◽  
Initial Perturbation ◽  
Error Variance ◽  
Precipitation Forecast ◽  
Initial Time ◽  
Naval Research ◽  
Ensemble Forecasts ◽  
Breeding Technique ◽  
Forecast Error Variance ◽  
Ensemble Transform

Abstract A computationally inexpensive ensemble transform (ET) method for generating high-resolution initial perturbations for regional ensemble forecasts is introduced. The method provides initial perturbations that (i) have an initial variance consistent with the best available estimates of initial condition error variance, (ii) are dynamically conditioned by a process similar to that used in the breeding technique, (iii) add to zero at the initial time, (iv) are quasi-orthogonal and equally likely, and (v) partially respect mesoscale balance constraints by ensuring that each initial perturbation is a linear sum of forecast perturbations from the preceding forecast. The technique is tested using estimates of analysis error variance from the Naval Research Laboratory (NRL) Atmospheric Variational Data Assimilation System (NAVDAS) and the Navy’s regional Coupled Ocean–Atmosphere Mesoscale Prediction System (COAMPS) over a 3-week period during the summer of 2005. Lateral boundary conditions are provided by a global ET ensemble. The tests show that the ET regional ensemble has a skillful mean and a useful spread–skill relationship in mass, momentum, and precipitation variables. Diagnostics indicate that ensemble variance was close to, but probably a little less than, the forecast error variance for wind and temperature variables, while precipitation ensemble variance was significantly smaller than precipitation forecast error variance.

Vertical Covariance Localization for Satellite Radiances in Ensemble Kalman Filters

2010 ◽  
Vol 138 (1) ◽  
pp. 282-290 ◽  
Author(s):  
William F. Campbell ◽  
Craig H. Bishop ◽  
Daniel Hodyss
Keyword(s):  
Kalman Filters ◽  
Forecast Error ◽  
Model Space ◽  
Error Variance ◽  
Observation Error ◽  
Ensemble Kalman Filters ◽  
Weighting Functions ◽  
Error Covariance ◽  
Space Localization ◽  
Satellite Channels

Abstract A widely used observation space covariance localization method is shown to adversely affect satellite radiance assimilation in ensemble Kalman filters (EnKFs) when compared to model space covariance localization. The two principal problems are that distance and location are not well defined for integrated measurements, and that neighboring satellite channels typically have broad, overlapping weighting functions, which produce true, nonzero correlations that localization in radiance space can incorrectly eliminate. The limitations of the method are illustrated in a 1D conceptual model, consisting of three vertical levels and a two-channel satellite instrument. A more realistic 1D model is subsequently tested, using the 30 vertical levels from the Navy Operational Global Atmospheric Prediction System (NOGAPS), the Advanced Microwave Sounding Unit A (AMSU-A) weighting functions for channels 6–11, and the observation error variance and forecast error covariance from the NRL Atmospheric Variational Data Assimilation System (NAVDAS). Analyses from EnKFs using radiance space localization are compared with analyses from raw EnKFs, EnKFs using model space localization, and the optimal analyses using the NAVDAS forecast error covariance as a proxy for the true forecast error covariance. As measured by mean analysis error variance reduction, radiance space localization is inferior to model space localization for every ensemble size and meaningful observation error variance tested. Furthermore, given as many satellite channels as vertical levels, radiance space localization cannot recover the true temperature state with perfect observations, whereas model space localization can.

scholarly journals Four-Dimensional Data Assimilation and Balanced Dynamics

2006 ◽  
Vol 63 (7) ◽  
pp. 1840-1858 ◽  
Author(s):  
Lisa J. Neef ◽  
Saroja M. Polavarapu ◽  
Theodore G. Shepherd
Keyword(s):  
Data Assimilation ◽  
Degrees Of Freedom ◽  
Forecast Error ◽  
Wave Mode ◽  
Slow Manifold ◽  
Error Statistics ◽  
Ensemble Kalman Filters ◽  
Error Covariance ◽  
Balanced Dynamics ◽  
Four Dimensional Data Assimilation

Abstract The problem of spurious excitation of gravity waves in the context of four-dimensional data assimilation is investigated using a simple model of balanced dynamics. The model admits a chaotic vortical mode coupled to a comparatively fast gravity wave mode, and can be initialized such that the model evolves on a so-called slow manifold, where the fast motion is suppressed. Identical twin assimilation experiments are performed, comparing the extended and ensemble Kalman filters (EKF and EnKF, respectively). The EKF uses a tangent linear model (TLM) to estimate the evolution of forecast error statistics in time, whereas the EnKF uses the statistics of an ensemble of nonlinear model integrations. Specifically, the case is examined where the true state is balanced, but observation errors project onto all degrees of freedom, including the fast modes. It is shown that the EKF and EnKF will assimilate observations in a balanced way only if certain assumptions hold, and that, outside of ideal cases (i.e., with very frequent observations), dynamical balance can easily be lost in the assimilation. For the EKF, the repeated adjustment of the covariances by the assimilation of observations can easily unbalance the TLM, and destroy the assumptions on which balanced assimilation rests. It is shown that an important factor is the choice of initial forecast error covariance matrix. A balance-constrained EKF is described and compared to the standard EKF, and shown to offer significant improvement for observation frequencies where balance in the standard EKF is lost. The EnKF is advantageous in that balance in the error covariances relies only on a balanced forecast ensemble, and that the analysis step is an ensemble-mean operation. Numerical experiments show that the EnKF may be preferable to the EKF in terms of balance, though its validity is limited by ensemble size. It is also found that overobserving can lead to a more unbalanced forecast ensemble and thus to an unbalanced analysis.

scholarly journals On the Sensitivity Equations of Four-Dimensional Variational (4D-Var) Data Assimilation

2008 ◽  
Vol 136 (8) ◽  
pp. 3050-3065 ◽  
Author(s):  
Dacian N. Daescu
Keyword(s):  
Data Assimilation ◽  
Forecast Error ◽  
Computational Effort ◽  
Water Model ◽  
Error Statistics ◽  
Background Error ◽  
Practical Applications ◽  
Error Covariance ◽  
Sensitivity Equations ◽  
The Impact

Abstract The equations of the forecast sensitivity to observations and to the background estimate in a four-dimensional variational data assimilation system (4D-Var DAS) are derived from the first-order optimality condition in unconstrained minimization. Estimation of the impact of uncertainties in the specification of the error statistics is considered by evaluating the sensitivity to the observation and background error covariance matrices. The information provided by the error covariance sensitivity analysis is used to identify the input components for which improved estimates of the statistical properties of the errors are of most benefit to the analysis and forecast. A close relationship is established between the sensitivities within each input pair data/error covariance such that once the observation and background sensitivities are available the evaluation of the sensitivity to the specification of the corresponding error statistics requires little additional computational effort. The relevance of the 4D-Var sensitivity equations to assess the data impact in practical applications is discussed. Computational issues are addressed and idealized 4D-Var experiments are set up with a finite-volume shallow-water model to illustrate the theoretical concepts. Time-dependent observation sensitivity and potential applications to improve the model forecast are presented. Guidance provided by the sensitivity fields is used to adjust a 4D-Var DAS to achieve forecast error reduction through assimilation of supplementary data and through an accurate specification of a few of the background error variances.

scholarly journals Hidden Error Variance Theory. Part I: Exposition and Analytic Model

2013 ◽  
Vol 141 (5) ◽  
pp. 1454-1468 ◽  
Author(s):  
Craig H. Bishop ◽  
Elizabeth A. Satterfield
Keyword(s):  
Posterior Distribution ◽  
Forecast Error ◽  
Error Variance ◽  
Empirical Estimation ◽  
Empirical Observation ◽  
Inverse Gamma ◽  
Error Covariance ◽  
Error Distributions ◽  
Forecast Error Variance ◽  
Covariance Models

Abstract A conundrum of predictability research is that while the prediction of flow-dependent error distributions is one of its main foci, chaos fundamentally hides flow-dependent forecast error distributions from empirical observation. Empirical estimation of such error distributions requires a large sample of error realizations given the same flow-dependent conditions. However, chaotic elements of the flow and the observing network make it impossible to collect a large enough conditioned error sample to empirically define such distributions and their variance. Such conditional variances are “hidden.” Here, an exposition of the problem is developed from an ensemble Kalman filter data assimilation system applied to a 10-variable nonlinear chaotic model and 25 000 replicate models. The 25 000 replicates reveal the error variances that would otherwise be hidden. It is found that the inverse-gamma distribution accurately approximates the posterior distribution of conditional error variances given an imperfect ensemble variance and provides a reasonable approximation to the prior climatological distribution of conditional error variances. A new analytical model shows how the properties of a likelihood distribution of ensemble variances given a true conditional error variance determine the posterior distribution of error variances given an ensemble variance. The analytically generated distributions are shown to satisfactorily fit empirically determined distributions. The theoretical analysis yields a rigorous interpretation and justification of hybrid error variance models that linearly combine static and flow-dependent estimates of forecast error variance; in doing so, it also helps justify and inform hybrid error covariance models.

Effects of Accounting-Method Choices on Subjective Performance-Measure Weighting Decisions: Experimental Evidence on Precision and Error Covariance

2005 ◽  
Vol 80 (4) ◽  
pp. 1163-1192 ◽  
Author(s):  
Ranjani Krishnan ◽  
Joan L. Luft ◽  
Michael D. Shields
Keyword(s):  
Performance Measures ◽  
Error Variance ◽  
Spillover Effects ◽  
Performance Measure ◽  
Incentive System ◽  
Experimental Setting ◽  
Error Covariance ◽  
Psychology Theory ◽  
Optimal Weighting ◽  
Similarity And Difference

Performance-measure weights for incentive compensation are often determined subjectively. Determining these weights is a cognitively difficult task, and archival research shows that observed performance-measure weights are only partially consistent with the predictions of agency theory. Ittner et al. (2003) have concluded that psychology theory can help to explain such inconsistencies. In an experimental setting based on Feltham and Xie (1994), we use psychology theories of reasoning to predict distinctive patterns of similarity and difference between optimal and actual subjective performance-measure weights. The following predictions are supported. First, in contrast to a number of prior studies, most individuals' decisions are significantly influenced by the performance measures' error variance (precision) and error covariance. Second, directional errors in the use of these measurement attributes are relatively frequent, resulting in a mean underreaction to an accounting change that alters performance measurement error. Third, individuals seem insufficiently aware that a change in the accounting for one measure has spillover effects on the optimal weighting of the other measure in a two-measure incentive system. In consequence, they make performance-measure weighting decisions that are likely to result in misallocations of agent effort.

scholarly journals Ensemble model for estimating continental-scale patterns of human movement: a case study of Australia

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Karen McCulloch ◽  
Nick Golding ◽  
Jodie McVernon ◽  
Sarah Goodwin ◽  
Martin Tomko
Keyword(s):  
Spatial Scales ◽  
Predictive Ability ◽  
Movement Patterns ◽  
Human Movement ◽  
Continental Scale ◽  
Movement Characteristics ◽  
Human Movements ◽  
Movement Models ◽  
Epidemiology Data

AbstractUnderstanding human movement patterns at local, national and international scales is critical in a range of fields, including transportation, logistics and epidemiology. Data on human movement is increasingly available, and when combined with statistical models, enables predictions of movement patterns across broad regions. Movement characteristics, however, strongly depend on the scale and type of movement captured for a given study. The models that have so far been proposed for human movement are best suited to specific spatial scales and types of movement. Selecting both the scale of data collection, and the appropriate model for the data remains a key challenge in predicting human movements. We used two different data sources on human movement in Australia, at different spatial scales, to train a range of statistical movement models and evaluate their ability to predict movement patterns for each data type and scale. Whilst the five commonly-used movement models we evaluated varied markedly between datasets in their predictive ability, we show that an ensemble modelling approach that combines the predictions of these models consistently outperformed all individual models against hold-out data.

Sign in / Sign up

Export Citation Format

Share Document