scholarly journals Assessing Goodness of Fit for Verifying Probabilistic Forecasts

Forecasting ◽  
2021 ◽  
Vol 3 (4) ◽  
pp. 763-773
Author(s):  
Tae-Ho Kang ◽  
Ashish Sharma ◽  
Lucy Marshall
Keyword(s):  
Goodness Of Fit ◽  
Integral Transformation ◽  
Southern Oscillation ◽  
Probability Distributions ◽  
Ensemble Prediction ◽  
Probabilistic Prediction ◽  
Test Statistic ◽  
Chi Square ◽  
Predicted Probability ◽  
Probabilistic Forecasts

The verification of probabilistic forecasts in hydro-climatology is integral to their development, use, and adoption. We propose here a means of utilizing goodness of fit measures for verifying the reliability of probabilistic forecasts. The difficulty in measuring the goodness of fit for a probabilistic prediction or forecast is that predicted probability distributions for a target variable are not stationary in time, meaning one observation alone exists to quantify goodness of fit for each prediction issued. Therefore, we suggest an additional dissociation that can dissociate target information from the other time variant part—the target to be verified in this study is the alignment of observations to the predicted probability distribution. For this dissociation, the probability integral transformation is used. To measure the goodness of fit for the predicted probability distributions, this study uses the root mean squared deviation metric. If the observations after the dissociation can be assumed to be independent, the mean square deviation metric becomes a chi-square test statistic, which enables statistically testing the hypothesis regarding whether the observations are from the same population as the predicted probability distributions. An illustration of our proposed rationale is provided using the multi-model ensemble prediction for El Niño–Southern Oscillation.

Ensemble Construction and Verification of the Probabilistic ENSO Prediction in the LDEO5 Model

2010 ◽  
Vol 23 (20) ◽  
pp. 5476-5497 ◽  
Author(s):  
Yanjie Cheng ◽  
Youmin Tang ◽  
Peter Jackson ◽  
Dake Chen ◽  
Ziwang Deng
Keyword(s):  
High Frequency ◽  
Singular Vector ◽  
Southern Oscillation ◽  
Initial Perturbation ◽  
Probabilistic Prediction ◽  
Construction Methods ◽  
Sea Surface Temperature Anomaly ◽  
Probabilistic Forecasts ◽  
Construction Strategy ◽  
Construction Strategies

Abstract El Niño–Southern Oscillation (ENSO) retrospective ensemble-based probabilistic predictions were performed for the period of 1856–2003 using the Lamont-Doherty Earth Observatory, version 5 (LDEO5), model. To obtain more reliable and skillful ENSO probabilistic predictions, first, four ensemble construction strategies were investigated: (i) the optimal initial perturbation with singular vector of sea surface temperature anomaly (SSTA), (ii) the realistic high-frequency anomalous winds, (iii) the stochastic optimal pattern of anomalous winds, and (iv) a combination of the first and the third strategy. Second, verifications were conducted to examine the reliability and resolution of the probabilistic forecasts provided by the four methods. Results suggest that reliability of ENSO probabilistic forecast is more sensitive to the choice of ensemble construction strategy than the resolution, and a reliable and skillful ENSO probabilistic prediction system may not necessarily have the best deterministic prediction skills. Among these ensemble construction methods, the fourth strategy produces the most reliable and skillful ENSO probabilistic prediction, benefiting from the joint contributions of the stochastic optimal winds and the singular vector of SSTA. In particular, the stochastic optimal winds play an important role in improving the ENSO probabilistic predictability for the LDEO5 model.

scholarly journals Estimating the Best-Fitted Probability Distribution for Monthly Maximum Temperature at the Sylhet Station in Bangladesh

2021 ◽  
Vol 2 (2) ◽  
pp. 60-67
Author(s):  
Rashidul Hasan Rashidul Hasan
Keyword(s):  
Probability Distribution ◽  
Goodness Of Fit ◽  
Probability Distributions ◽  
Probability Model ◽  
Temperature Data ◽  
Maximum Temperature ◽  
Chi Square ◽  
Goodness Of Fit Tests ◽  
Anderson Darling ◽  
Log Normal

The estimation of a suitable probability model depends mainly on the features of available temperature data at a particular place. As a result, existing probability distributions must be evaluated to establish an appropriate probability model that can deliver precise temperature estimation. The study intended to estimate the best-fitted probability model for the monthly maximum temperature at the Sylhet station in Bangladesh from January 2002 to December 2012 using several statistical analyses. Ten continuous probability distributions such as Exponential, Gamma, Log-Gamma, Beta, Normal, Log-Normal, Erlang, Power Function, Rayleigh, and Weibull distributions were fitted for these tasks using the maximum likelihood technique. To determine the model’s fit to the temperature data, several goodness-of-fit tests were applied, including the Kolmogorov-Smirnov test, Anderson-Darling test, and Chi-square test. The Beta distribution is found to be the best-fitted probability distribution based on the largest overall score derived from three specified goodness-of-fit tests for the monthly maximum temperature data at the Sylhet station.

scholarly journals A Modified Chi-square Type Test Statistic for the Double Burr X Model with Applications to Right Censored Medical and Reliability Data

2021 ◽  
pp. 615-623
Author(s):  
Khaoula Aidi ◽  
Nadeem Shafique Butt ◽  
Mir Masoom Ali ◽  
Mohamed Ibrahim ◽  
Haitham M. Yousof ◽  
...  
Keyword(s):  
Censored Data ◽  
Goodness Of Fit ◽  
Estimation Method ◽  
Likelihood Estimation ◽  
Data Sets ◽  
Test Statistic ◽  
Goodness Of Fit Test ◽  
Chi Square ◽  
Right Censored Data ◽  
The Right

A new modified version of the Bagdonavičius-Nikulin goodness-of-fit test statistic is presented for validity for the right censor case under the double Burr type X distribution. The maximum likelihood estimation method in censored data case is used and applied. Simulations via the algorithm of Barzilai-Borwein is performed for assessing the right censored estimation method. Another simulation study is presented for testing the null hypothesis under the modified version of the Bagdonavičius and Nikulin goodness-of-fit statistical test. Four right censored data sets are analyzed under the new modified test statistic for checking the distributional validation.

scholarly journals Generation of Scenarios from Calibrated Ensemble Forecasts with a Dual-Ensemble Copula-Coupling Approach

2016 ◽  
Vol 144 (12) ◽  
pp. 4737-4750 ◽  
Author(s):  
Zied Ben Bouallègue ◽  
Tobias Heppelmann ◽  
Susanne E. Theis ◽  
Pierre Pinson
Keyword(s):  
Probability Distributions ◽  
Forecast Error ◽  
Computational Cost ◽  
Joint Probability ◽  
Ensemble Prediction ◽  
Small Scale ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Probabilistic Forecasts ◽  
Joint Probability Distributions

Abstract Probabilistic forecasts in the form of ensembles of scenarios are required for complex decision-making processes. Ensemble forecasting systems provide such products but the spatiotemporal structures of the forecast uncertainty is lost when statistical calibration of the ensemble forecasts is applied for each lead time and location independently. Nonparametric approaches allow the reconstruction of spatiotemporal joint probability distributions at a small computational cost. For example, the ensemble copula coupling (ECC) method rebuilds the multivariate aspect of the forecast from the original ensemble forecasts. Based on the assumption of error stationarity, parametric methods aim to fully describe the forecast dependence structures. In this study, the concept of ECC is combined with past data statistics in order to account for the autocorrelation of the forecast error. The new approach, called d-ECC, is applied to wind forecasts from the high-resolution Consortium for Small-Scale Modeling (COSMO) ensemble prediction system (EPS) run operationally at the German Weather Service (COSMO-DE-EPS). Scenarios generated by ECC and d-ECC are compared and assessed in the form of time series by means of multivariate verification tools and within a product-oriented framework. Verification results over a 3-month period show that the innovative method d-ECC performs as well as or even outperforms ECC in all investigated aspects.

TESTING INDEPENDENCE IN TIME SERIES VIA UNIVERSAL DISTRIBUTIONS OF PERMUTATIONS AND WORDS

2005 ◽  
Vol 15 (05) ◽  
pp. 1757-1765 ◽  
Author(s):  
CAMILLO CAMMAROTA ◽  
ENRICO ROGORA
Keyword(s):  
Time Series ◽  
General Framework ◽  
Goodness Of Fit ◽  
Probability Distributions ◽  
Recursive Algorithm ◽  
Chi Square ◽  
Independent Sequence ◽  
Empirical Distributions ◽  
Testing Independence ◽  
Analysis Of Time Series

We study probability distributions of permutations and binary words, which arise in symbolic analysis of time series and their differences. Under the assumptions that the series is stationary and independent we show that these probability distributions are universal and we derive a recursive algorithm for computing the distribution of binary words. This provides a general framework for performing chi square tests of goodness of fit of empirical distributions versus universal ones. We apply these methods to analyze heartbeat time series; in particular, we measure the extent to which atrial fibrillation can be modeled as an independent sequence.

scholarly journals Intercomparison of estimators of extreme value family of distributions for rainfall frequency analysis

MAUSAM ◽  
2022 ◽  
Vol 73 (1) ◽  
pp. 59-70
Author(s):  
N. VIVEKANANDAN
Keyword(s):  
Goodness Of Fit ◽  
Probability Distributions ◽  
Extreme Value ◽  
Qualitative Assessment ◽  
Likelihood Method ◽  
Extreme Value Analysis ◽  
Chi Square ◽  
Maximum Rainfall ◽  
Value Analysis ◽  
Family Of Distributions

Estimation of rainfall for a given return period is of utmost importance for planning and design of minor and major hydraulic structures. This can be achieved through Extreme Value Analysis (EVA) of rainfall by fitting Extreme Value family of Distributions (EVD) such as Generalized Extreme Value, Extreme Value Type-1, Extreme Value Type-2 and Generalized Pareto to the series of observed Annual 1-Day Maximum Rainfall (AMR) data. Based on the intended applications and the variate under consideration, Method of Moments (MoM), Maximum Likelihood Method (MLM) and L-Moments (LMO) are used for determination of parameters of probability distributions. The adequacy of fitting EVD to the AMR series was evaluated by quantitative assessment using Goodness-of-Fit (viz., Chi-square and Kolmogorov-Smirnov) and diagnostic test (viz., D-index) tests and qualitative assessment by the fitted curves of the estimated rainfall. The paper presents a study on intercomparison of EVD (using MoM, MLM and LMO) adopted in EVA of rainfall with illustrative example and the results obtained thereof. 

scholarly journals A Modified Chi-square Type Test for Distributional Validity with Applications to Right Censored Reliability and Medical Data

2021 ◽  
pp. 1113-1121 ◽  
Author(s):  
Haitham M. Yousof ◽  
Abdullah H. Al-nefaie ◽  
Khaoula Aidi ◽  
M. Masoom Ali ◽  
Mohamed ibrahim Mohamed
Keyword(s):  
Censored Data ◽  
Goodness Of Fit ◽  
Data Sets ◽  
Test Statistic ◽  
Goodness Of Fit Test ◽  
Chi Square ◽  
Right Censored Data ◽  
Type Test ◽  
The Right ◽  
New Distribution

In this paper, a modified Chi-square goodness-of-fit test called the modified Bagdonavičius-Nikulin goodness-of-fit test statistic is investigated and the applied for distributional validation under the right censored case. The new modified goodness-of-fit test is presented and applied for the right censored data sets. The algorithm of the censored Barzilai-Borwein is employed via a comprehensive simulation study for assessing validity of the new test. The modified Bagdonavičius-Nikulin test is applied to four real and right censored data sets. A new distribution is compared with many other competitive distributions under the new modified Bagdonavičius-Nikulin goodness-of-fit test statistic.

Goodness-of-Fit Tests for Probability Distributions and Spectral Distributions

1995 ◽  
Vol 9 (1) ◽  
pp. 27-37 ◽  
Author(s):  
T. W. Anderson
Keyword(s):  
Least Squares ◽  
Goodness Of Fit ◽  
Probability Distributions ◽  
Close Friend ◽  
Columbia University ◽  
Chi Square ◽  
Goodness Of Fit Tests ◽  
Spectral Distributions ◽  
One Year ◽  
Promising Student

In the fall of 1948 in my course on Least Squares in the Department of Mathematical Statistics at Columbia University (and in the spring in Correlation and Chi-Square), I was particularly impressed by one of the students— Gerald J. Lieberman. I was disappointed that this promising student left Columbia after one year, but it was not long until our paths met again. It is a pleasure to dedicate this paper to my colleague and close friend!

scholarly journals Dynamical, Statistical–Dynamical, and Multimodel Ensemble Forecasts of Australian Spring Season Rainfall

2011 ◽  
Vol 139 (3) ◽  
pp. 958-975 ◽  
Author(s):  
Eun-Pa Lim ◽  
Harry H. Hendon ◽  
David L. T. Anderson ◽  
Andrew Charles ◽  
Oscar Alves
Keyword(s):  
El Niño ◽  
El Nino ◽  
Southern Oscillation ◽  
Prediction Method ◽  
Rainfall Anomaly ◽  
Ensemble Prediction ◽  
Spring Season ◽  
Multimodel Ensemble ◽  
Austral Spring ◽  
Probabilistic Forecasts

Abstract The prediction skill of the Australian Bureau of Meteorology dynamical seasonal forecast model Predictive Ocean Atmosphere Model for Australia (POAMA) is assessed for probabilistic forecasts of spring season rainfall in Australia and the feasibility of increasing forecast skill through statistical postprocessing is examined. Two statistical postprocessing techniques are explored: calibrating POAMA prediction of rainfall anomaly against observations and using dynamically predicted mean sea level pressure to infer regional rainfall anomaly over Australia (referred to as “bridging”). A “homogeneous” multimodel ensemble prediction method (HMME) is also introduced that consists of the combination of POAMA’s direct prediction of rainfall anomaly together with the two statistically postprocessed predictions. Using hindcasts for the period 1981–2006, the direct forecasts from POAMA exhibit skill relative to a climatological forecast over broad areas of eastern and southern Australia, where El Niño and the Indian Ocean dipole (whose behavior POAMA can skillfully predict at short lead times) are known to exert a strong influence in austral spring. The calibrated and bridged forecasts, while potentially offering improvement over the direct forecasts because of POAMA’s ability to predict the main drivers of springtime rainfall (e.g., El Niño and the Southern Oscillation), show only limited areas of improvement, mainly because strict cross-validation limits the ability to capitalize on relatively modest predictive signals with short record lengths. However, when POAMA and the two statistical–dynamical rainfall forecasts are combined in the HMME, higher deterministic and probabilistic skill is achieved over any of the single models, which suggests the HMME is another useful method to calibrate dynamical model forecasts.

Sign in / Sign up

Export Citation Format

Share Document