scholarly journals Development and Analysis of a Probabilistic Forecasting Game for Meteorology Students

2012 ◽  
Vol 93 (12) ◽  
pp. 1833-1843
Author(s):  
Steven G. Decker
Keyword(s):  
Undergraduate Students ◽  
Operating Characteristic ◽  
Weather Forecasting ◽  
Student Feedback ◽  
Brier Score ◽  
Probabilistic Forecasting ◽  
Relative Operating Characteristic ◽  
Rutgers University ◽  
Forecast Interval ◽  
Temperature Errors

Calls for moving from a deterministic to a probabilistic view of weather forecasting have become increasingly urgent over recent decades, yet the primary national forecasting competition and many in-class forecasting games are wholly deterministic in nature. To counter these conflicting trends, a long-running forecasting game at Rutgers University has recently been modified to become probabilistic in nature. Students forecast high- and low-temperature intervals and probabilities of precipitation for two locations: one fixed at the Rutgers cooperative observing station, the other chosen for each forecast window to maximize difficulty. Precipitation errors are tabulated with a Brier score, while temperature errors contain a sharpness component dependent on the width of the forecast interval and an interval miss component dependent on the degree to which the verification falls within the interval. The inclusion of a probabilistic forecasting game allows for the creation of a substantial database of forecasts that can be analyzed using standard probabilistic approaches, such as reliability diagrams, relative operating characteristic curves, and histograms. Discussions of probabilistic forecast quality can be quite abstract for undergraduate students, but the use of a forecast database that students themselves help construct motivates these discussions and helps students make connections between their forecast process, their standing in class rankings, and the verification diagrams they use. Student feedback on the probabilistic game is also discussed.

scholarly journals Análisis, modelamiento y simulación espacial del cambio de cobertura del suelo, entre las áreas naturales y las de origen antrópico en la provincia de Napo (Ecuador), para el período 1990-2020

2014 ◽  
Author(s):  
◽  
Jorge Hurtado Pidal
Keyword(s):  
Operating Characteristic ◽  
Relative Operating Characteristic ◽  
Del Rio

Se recopiló una base de datos geográfica, con cartografía básica y temática, sobre la provincia de Napo (Ecuador), en la que se destacan los mapas de cobertura del suelo de los años 2002 y 2008. Como primer producto se elaboró un mapa de cobertura del suelo del año 1990 a partir de imágenes del sensor TM, (Landsat 4 y 5). Posteriormente se realizó un modelo de probabilidad de presencia de coberturas de tipo antrópico, usando la técnica de regresión logística multivariada; se evaluó el modelo con la curva ROC (Relative Operating Characteristic) y se determinó un alto poder de predicción en el modelo (AUC 0.89), distinguiendo además, que la distancia a centros poblados y a vías de comunicación son las variables más influyentes para la presencia de coberturas antrópicas. Se utilizó el mapa resultante del modelo de probabilidad como entrada en un modelo de transición de coberturas que combina Autómatas Celulares y Cadenas de Markov, entre otros aspectos, simulando un mapa de tipo de coberturas (natural o antrópico) para el año 2008. Se evaluó este mapa simulado, comparándolo con uno de referencia, a partir de índices kappa, y se obtuvo un porcentaje de concordancia general de 93%, lo cual es un buen indicador. Una vez que se ha contado con un modelo que permitía hacer simulaciones con el grado de confianza necesario, se realizaron simulaciones para los años 2015 y 2020. En estos escenarios de tipo de cobertura, se puede ver una clara presión hacia los bosques de la rivera del Rio Napo en un futuro, y también en aquellos cercanos a los principales centros poblados como Tena especialmente. Sin embargo, las áreas protegidas muestran un estado de conservación “natural” en las simulaciones, y esto se debe a sus condiciones de inaccesibilidad, en cuanto a falta de infraestructura vial, y a sus condiciones ambientales especiales. Por último, se verificó que la tasa de deforestación (cambio de natural hacia antrópico) en el período 1990-2008 fue de 4661 ha/año y en el período 2008-2020 sería de 3550 ha/año, indicando que la tendencia en el tiempo muestra en el mejor de los casos una disminución o por lo menos una estabilización de los procesos de deforestación.

scholarly journals Designing Multimodel Applications with Surrogate Forecast Systems

2020 ◽  
Vol 148 (6) ◽  
pp. 2233-2249
Author(s):  
Leonard A. Smith ◽  
Hailiang Du ◽  
Sarah Higgins
Keyword(s):  
Bayesian Model Averaging ◽  
Weather Forecasting ◽  
Model Averaging ◽  
Probabilistic Forecasting ◽  
Single Model ◽  
Forecast System ◽  
Mathematical System ◽  
Probabilistic Forecasts ◽  
Perfect Model ◽  
Seasonal Weather

Abstract Probabilistic forecasting is common in a wide variety of fields including geoscience, social science, and finance. It is sometimes the case that one has multiple probability forecasts for the same target. How is the information in these multiple nonlinear forecast systems best “combined”? Assuming stationarity, in the limit of a very large forecast–outcome archive, each model-based probability density function can be weighted to form a “multimodel forecast” that will, in expectation, provide at least as much information as the most informative single model forecast system. If one of the forecast systems yields a probability distribution that reflects the distribution from which the outcome will be drawn, Bayesian model averaging will identify this forecast system as the preferred system in the limit as the number of forecast–outcome pairs goes to infinity. In many applications, like those of seasonal weather forecasting, data are precious; the archive is often limited to fewer than 26 entries. In addition, no perfect model is in hand. It is shown that in this case forming a single “multimodel probabilistic forecast” can be expected to prove misleading. These issues are investigated in the surrogate model (here a forecast system) regime, where using probabilistic forecasts of a simple mathematical system allows many limiting behaviors of forecast systems to be quantified and compared with those under more realistic conditions.

scholarly journals Increasing the horizontal resolution of ensemble forecasts at CMC

2003 ◽  
Vol 10 (6) ◽  
pp. 463-468 ◽  
Author(s):  
G. Pellerin ◽  
L. Lefaivre ◽  
P. Houtekamer ◽  
C. Girard
Keyword(s):  
Operating Characteristic ◽  
Horizontal Resolution ◽  
Ensemble Prediction ◽  
Ensemble Size ◽  
Ensemble Forecasts ◽  
Relative Operating Characteristic ◽  
Sea Level Pressure ◽  
Ensemble Prediction System ◽  
Probability Of Precipitation ◽  
Model Approach

Abstract. Ensemble forecasts are run operationally since February 1998 at the Canadian Meteorological Centre, with outputs up to ten days. The ensemble size was increased from eight to sixteen members in August 1999. The method of producing the perturbed analyses consists of running independent assimilation cycles that use perturbed sets of observations and are driven by eight different models, mainly different in their physical parameterizations. Perturbed analyses are doubled by taking opposite pairs. A multi-model approach is then used to obtain the forecasts. The ensemble output has been used to generate several products. In view of increasing computing facilities, the ensemble prediction system horizontal resolution was increased to TL149 in June 2001. Heights at 500 hPa and mean sea-level pressure maps are regularly used. Charts of precipitation with the probability of precipitation being above various thresholds are also produced at each run. The probabilistic forecast of the 24-h accumulated precipitation has shown skill as demonstrated by the relative operating characteristic (ROC). Verifications of the ensemble forecasts will be presented.

scholarly journals An Extended Procedure for Implementing the Relative Operating Characteristic Graphical Method

2006 ◽  
Vol 45 (9) ◽  
pp. 1215-1223 ◽  
Author(s):  
Fredrick H. M. Semazzi ◽  
Roberto J. Mera
Keyword(s):  
Operating Characteristic ◽  
Graphical Method ◽  
General Procedure ◽  
Economic Value ◽  
Skill Score ◽  
Prediction System ◽  
End User ◽  
Relative Operating Characteristic ◽  
Typical Situation ◽  
Pass Through

Abstract The functional relationship between the relative operating characteristic (ROC) and the economic value (EV) graphical methods have been exploited to develop a hybrid procedure called the extended ROC (EROC) method. The EROC retains the appealing simplicity of the traditional ROC method and the ability of the EV method to provide evaluation of the performance of an ensemble climate prediction system (EPS) for a hypothetical end user defined by the cost–loss ratio (μ = C/L). An inequality defining the lower and upper theoretical bounds of μ has been derived. Outside these limits, the EPS yields no added benefits for end user μ relative to the use of climatological persistence as an alternative prediction system. In the traditional ROC graphical method, the ROC skill (ROCS) is often expressed in terms of the area between the ROC graph and the diagonal baseline passing through the origin with slope m = 1. Thus, ROCS = 2A − 1, where A is the area under the ROC graph. In the proposed EROC approach, a more general procedure is recommended based on the construction of user-specific baselines that do not necessarily pass through the origin and, in general, have slope m ≠ 1. The skill of a particular EPS computed from the EROC method is proportional to the corresponding estimated value based on the EV graphical method. Therefore, the EROC geometry conveys the same basic information as the EV method. The Semazzi–Mera skill score (SMSS) is proposed as a convenient and compact way of expressing the combined verification based on the ROC and EV methods. The ROCS estimate is a special case of the SMSS. The near-horizontal trail-like geometry sometimes exhibited by EV graphs is also examined. It is shown to occur when either the hit-rate or false-alarm term dominates in the formula for EV, unlike the more typical situation in which both terms are comparable in magnitude.

scholarly journals Comparisons of QPFs Derived from Single- and Multicore Convection-Allowing Ensembles

2019 ◽  
Vol 34 (6) ◽  
pp. 1955-1964
Author(s):  
Adam J. Clark
Keyword(s):  
Operating Characteristic ◽  
Characteristic Curve ◽  
Skill Score ◽  
Ensemble Forecast ◽  
Grid Spacing ◽  
Forecast System ◽  
Relative Operating Characteristic ◽  
Core System ◽  
Testing And Evaluation ◽  
Better Than

Abstract This study compares ensemble precipitation forecasts from 10-member, 3-km grid-spacing, CONUS domain single- and multicore ensembles that were a part of the 2016 Community Leveraged Unified Ensemble (CLUE) that was run for the 2016 NOAA Hazardous Weather Testbed Spring Forecasting Experiment. The main results are that a 10-member ARW ensemble was significantly more skillful than a 10-member NMMB ensemble, and a 10-member MIX ensemble (5 ARW and 5 NMMB members) performed about the same as the 10-member ARW ensemble. Skill was measured by area under the relative operating characteristic curve (AUC) and fractions skill score (FSS). Rank histograms in the ARW ensemble were flatter than the NMMB ensemble indicating that the envelope of ensemble members better encompassed observations (i.e., better reliability) in the ARW. Rank histograms in the MIX ensemble were similar to the ARW ensemble. In the context of NOAA’s plans for a Unified Forecast System featuring a CAM ensemble with a single core, the results are positive and indicate that it should be possible to develop a single-core system that performs as well as or better than the current operational CAM ensemble, which is known as the High-Resolution Ensemble Forecast System (HREF). However, as new modeling applications are developed and incremental changes that move HREF toward a single-core system are made possible, more thorough testing and evaluation should be conducted.

Diagnostic power of lecithin/sphingomyelin ratio and fluorescence polarization assays for respiratory distress syndrome compared by relative operating characteristic curves

1994 ◽  
Vol 40 (4) ◽  
pp. 541-545 ◽  
Author(s):  
T M Bender ◽  
L R Stone ◽  
J S Amenta
Keyword(s):  
Respiratory Distress Syndrome ◽  
Respiratory Distress ◽  
Fluorescence Polarization ◽  
Laboratory Tests ◽  
Operating Characteristic ◽  
Distress Syndrome ◽  
Roc Curves ◽  
Relative Operating Characteristic ◽  
Diagnostic Power ◽  
S Curve

Abstract Amniotic fluids from 328 patients were analyzed for lecithin/sphingomyelin (L/S) ratio and surfactant/albumin (S/A) ratio by fluorescence polarization. Of this group, 61 neonates showed respiratory distress syndrome (RDS) on delivery within 3 days of testing. We compared the power of the L/S and S/A in diagnosing pulmonary maturity, using relative operating characteristic (ROC) curves. The area defined by the ROC curve of the S/A test exceeded the area defined by the L/S curve, but this difference was not statistically significant. The diagnostic power of the S/A test appears to be at least equal to that of the standard L/S test. A review of five cases of RDS in which laboratory tests had suggested maturity showed that neither the L/S nor the S/A could satisfactorily resolve the problem of false interpretations of maturity, particularly in mothers with diabetes mellitus who underwent cesarean section.

A study of the validity of the scaled version of the General Health Questionnaire in paralysed spinally injured out-patients

1993 ◽  
Vol 23 (2) ◽  
pp. 497-504 ◽  
Author(s):  
T. C. Griffiths ◽  
D. H. Myers ◽  
A. W. Talbot
Keyword(s):  
Confidence Interval ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
General Health Questionnaire ◽  
Misclassification Rate ◽  
Health Questionnaire ◽  
The West ◽  
Interview Schedule ◽  
Relative Operating Characteristic ◽  
West Midlands

SynopsisThis study validates the GHQ-28 on a sample of paralysed, spinally injured outpatients under the care of the West Midlands Spinal Injuries Unit. The validity of the GHQ-28 was 0·83 (95% confidence interval: 0·70 to 0·93) using the Clinical Interview Schedule (CIS) as the validating criterion. The effectiveness of the GHQ-28 as a screening instrument using CIS 11/12 as case criterion, and thus a prevalence rate of 0·18, was judged by relative operating characteristic (ROC) analysis. The area under the ROC curve was 0·91±0·03. “Optimum’ discrimination occurred near GHQ 3/4 (0011 scoring scheme) giving a sensitivity of 0·81, a specificity of 0·82 and a misclassification rate of 18%.

Sign in / Sign up

Export Citation Format

Share Document