scholarly journals Exploring the Usefulness of Downscaling Free Forecasts from the Warn-on-Forecast System

2021 ◽  
Keyword(s):  
Forecast Model ◽  
Probability Of Detection ◽  
Forecast System ◽  
Local Maxima ◽  
Critical Success ◽  
Object Based ◽  
Verification Methods ◽  
Horizontal Grid ◽  
Critical Success Index ◽  
Eleven Case

Abstract The National Severe Storms Lab (NSSL) Warn-on-Forecast System (WoFS) is an experimental real-time rapidly-updating convection-allowing ensemble that provides probabilistic short-term thunderstorm forecasts. This study evaluates the impacts of reducing the forecast model horizontal grid spacing Δx from 3 km to 1.5 km on the WoFS deterministic and probabilistic forecast skill, using eleven case days selected from the 2020 NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (SFE). Verification methods include (i) subjective forecaster impressions; (ii) a deterministic object-based technique that identifies forecast reflectivity and rotation track storm objects as contiguous local maxima in the composite reflectivity and updraft helicity fields, respectively, and matches them to observed storm objects; and (iii) a recently developed algorithm that matches observed mesocyclones to mesocyclone probability swath objects constructed from the full ensemble of rotation track objects. Reducing Δx fails to systematically improve deterministic skill in forecasting reflectivity object occurrence, as measured by critical success index (CSIDET), a metric that incorporates both probability of detection (PODDET) and false alarm ratio (FARDET). However, compared to the Δx = 3 km configuration, the Δx = 1.5 km WoFS shows improved mid-level mesocyclone detection, as evidenced by its statistically significant (i) higher CSIDET for deterministic mid-level rotation track objects and (ii) higher normalized area under the performance diagram curve (NAUPDC) score for probability swath objects. Comparison between Δx = 3 km and Δx = 1.5 km reflectivity object properties reveals that the latter have 30% stronger mean updraft speeds, 17% stronger median 80-m winds, 67% larger median hail diameter, and 28% higher median near-storm-maximum 0-3 km storm-relative helicity.

scholarly journals APPLICATION OF A SPATIAL VERIFICATION METHOD TO GFS PRECIPITATION FORECASTS

2014 ◽  
Vol 32 (3) ◽  
pp. 561
Author(s):  
Fabiani Denise Bender ◽  
Rita Yuri Ynoue
Keyword(s):  
Weather Forecast ◽  
Sao Paulo ◽  
Probability Of Detection ◽  
Precipitation Forecast ◽  
São Paulo ◽  
Verification Method ◽  
Critical Success ◽  
Object Based ◽  
Centroid Distance ◽  
Critical Success Index

BSTRACT. This study aims to describe a spatial analysis of precipitation field with the MODE tool, which consists in comparing features converted from griddedforecast and observed precipitation values. This evaluation was performed daily from April 2010 to March 2011, for the 36-h GFS precipitation forecast started at00 UTC over the state of São Paulo and neighborhood. Besides traditional verification measures, such as accuracy (A), critical success index (CSI), bias (BIAS),probability of detection (POD), and false alarm ratio (FAR); new verification measures are proposed, such as area ratio (AR), centroid distance (CD) and 50th and 90thpercentiles ratio of intensity (PR50 and PR90). Better performance was attained during the rainy season. Part of the errors in the simulations was due to overestimationof the forecasted intensity and precipitation areas.Keywords: object-based verification, weather forecast, precipitation, MODE, São Paulo. RESUMO. Este estudo tem como objetivo descrever uma análise espacial do campo de precipitação com a ferramenta MODE, que consiste em converter valores deprecipitação de grade do campo previsto e observado em objetos, que posteriormente serão comparados entre si. A avaliação é realizada diariamente sobre o estadode São Paulo e vizinhança, para o período de abril de 2010 a março de 2011, para as simulações do modelo GFS iniciadas às 00 UTC, na integração de 36 horas. Além da verificação através de índices tradicionais, como probabilidade de acerto (PA), índice crítico de sucesso (ICS), viés (VIÉS), probabilidade de detecção (PD)e razão de falso alarme (RFA), novos índices de avaliação são propostos, como razão de área (RA), distância do centroide (DC) e razão dos percentis 50 e 90 deintensidade (RP50 e RP90). O melhor desempenho ocorreu para a estação chuvosa. Parte dos erros nas simulações foi devido à superestimativa da intensidade e da área de abrangência dos eventos de precipitação em relação ao observado.Palavras-chave: avaliação baseada em objetos, previsão do tempo, precipitação, MODE, São Paulo.

scholarly journals Object-Based Verification of a Prototype Warn-on-Forecast System

2018 ◽  
Vol 33 (5) ◽  
pp. 1225-1250 ◽  
Author(s):  
Patrick S. Skinner ◽  
Dustan M. Wheatley ◽  
Kent H. Knopfmeier ◽  
Anthony E. Reinhart ◽  
Jessica J. Choate ◽  
...  
Keyword(s):  
Performance Metrics ◽  
Initial Conditions ◽  
Forecast System ◽  
Critical Success ◽  
Object Based ◽  
System Data ◽  
System Configurations ◽  
Verification Methodology ◽  
Forecast Time ◽  
Critical Success Index

Abstract An object-based verification methodology for the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e) has been developed and applied to 32 cases between December 2015 and June 2017. NEWS-e forecast objects of composite reflectivity and 30-min updraft helicity swaths are matched to corresponding reflectivity and rotation track objects in Multi-Radar Multi-Sensor system data on space and time scales typical of a National Weather Service warning. Object matching allows contingency-table-based verification statistics to be used to establish baseline performance metrics for NEWS-e thunderstorm and mesocyclone forecasts. NEWS-e critical success index (CSI) scores of reflectivity (updraft helicity) forecasts decrease from approximately 0.7 (0.4) to 0.4 (0.2) over 3 h of forecast time. CSI scores decrease through the forecast period, indicating that errors do not saturate during the 3-h forecast. Lower verification scores for rotation track forecasts are primarily a result of a high-frequency bias. Comparison of different system configurations used in 2016 and 2017 shows an increase in skill for 2017 reflectivity forecasts, attributable mainly to improvements in the forecast initial conditions. A small decrease in skill in 2017 rotation track forecasts is likely a result of sample differences between 2016 and 2017. Although large case-to-case variation is present, evidence is found that NEWS-e forecast skill improves with increasing object size and intensity.

scholarly journals Rainfall Threshold for Flash Flood Warning Based on Model Output of Soil Moisture: Case Study Wernersbach, Germany

Water ◽  
2021 ◽  
Vol 13 (8) ◽  
pp. 1061
Author(s):  
Thanh Thi Luong ◽  
Judith Pöschmann ◽  
Rico Kronenberg ◽  
Christian Bernhofer
Keyword(s):  
Soil Moisture ◽  
Flash Flood ◽  
Rainfall Threshold ◽  
Probability Of Detection ◽  
Water Levels ◽  
Critical Success ◽  
Flood Warning ◽  
Critical Discharge ◽  
Critical Success Index

Convective rainfall can cause dangerous flash floods within less than six hours. Thus, simple approaches are required for issuing quick warnings. The flash flood guidance (FFG) approach pre-calculates rainfall levels (thresholds) potentially causing critical water levels for a specific catchment. Afterwards, only rainfall and soil moisture information are required to issue warnings. This study applied the principle of FFG to the Wernersbach Catchment (Germany) with excellent data coverage using the BROOK90 water budget model. The rainfall thresholds were determined for durations of 1 to 24 h, by running BROOK90 in “inverse” mode, identifying rainfall values for each duration that led to exceedance of critical discharge (fixed value). After calibrating the model based on its runoff, we ran it in hourly mode with four precipitation types and various levels of initial soil moisture for the period 1996–2010. The rainfall threshold curves showed a very high probability of detection (POD) of 91% for the 40 extracted flash flood events in the study period, however, the false alarm rate (FAR) of 56% and the critical success index (CSI) of 42% should be improved in further studies. The proposed adjusted FFG approach has the potential to provide reliable support in flash flood forecasting.

Identification of Range Overlaid Echoes Using Polarimetric Radar Measurements Based on a Fuzzy Logic Approach

2016 ◽  
Vol 33 (1) ◽  
pp. 61-80 ◽  
Author(s):  
S.-G. Park ◽  
Ji-Hyeon Kim ◽  
Jeong-Seok Ko ◽  
Gyuwon Lee
Keyword(s):  
Fuzzy Logic ◽  
Probability Of Detection ◽  
Range Ambiguity ◽  
Critical Success ◽  
Skill Scores ◽  
Fuzzy Logic Approach ◽  
Logic Approach ◽  
Radar Measurements ◽  
Differential Reflectivity ◽  
Critical Success Index

AbstractThe Ministry of Land, Infrastructure and Transport (MOLIT) of South Korea operates two S-band dual-polarimetric radars, as of 2013, to manage water resources through quantitative rainfall estimations at the surface level. However, the radar measurements suffer from range ambiguity. In this study, an algorithm based on fuzzy logic is developed to identify range overlaid echoes using seven inputs: standard deviations of differential reflectivity SD(ZDR), differential propagation phase SD(ϕDP), correlation coefficient SD(ρHV) and spectrum width SD(συ), mean of ρHV and συ, and difference of ϕDP from the system offset ΔϕDP. An examination of the algorithm’s performance shows that these echoes can be well identified and that echoes strongly affected by second trip are highlighted by high probabilities, over 0.6; echoes weakly affected have probabilities from 0.4 to 0.6; and those with low probabilities, below 0.4, are assigned as echoes without range ambiguity. A quantitative analysis of a limited number of cases using the usual skill scores shows that when the probability of 0.4 is considered as a threshold for identifying the range overlaid echoes, they can be identified with a probability of detection of 90%, a false alarm rate of 6%, and a critical success index of 84%.

scholarly journals An Empirical Blowing Snow Forecast Technique for the Canadian Arctic and the Prairie Provinces

2005 ◽  
Vol 20 (1) ◽  
pp. 51-62 ◽  
Author(s):  
David G. Baggaley ◽  
John M. Hanesiak
Keyword(s):  
Wind Speed ◽  
Canadian Arctic ◽  
High Impact ◽  
Probability Of Detection ◽  
The Arctic ◽  
Blowing Snow ◽  
Critical Success ◽  
Critical Success Index ◽  
Success Index ◽  
Prairie Provinces

Abstract Blowing snow has a major impact on transportation and public safety. The goal of this study is to provide an operational technique for forecasting high-impact blowing snow on the Canadian arctic and the Prairie provinces using historical meteorological data. The focus is to provide some guidance as to the probability of reduced visibilities (e.g., less than 1 km) in blowing snow given a forecast wind speed and direction. The wind character associated with blowing snow was examined using a large database consisting of up to 40 yr of hourly observations at 15 locations in the Prairie provinces and at 17 locations in the arctic. Instances of blowing snow were divided into cases with and without concurrent falling snow. The latter group was subdivided by the time since the last snowfall in an attempt to account for aging processes of the snowpack. An empirical scheme was developed that could discriminate conditions that produce significantly reduced visibility in blowing snow using wind speed, air temperature, and time since last snowfall as predictors. This process was evaluated using actual hourly observations to compute the probability of detection, false alarm ratio, credibility, and critical success index. A critical success index as high as 66% was achieved. This technique can be used to give an objective first guess of the likelihood of high-impact blowing snow using common forecast parameters.

scholarly journals Evaluation of the Storm Prediction Center’s Day 1 Convective Outlooks

2012 ◽  
Vol 27 (6) ◽  
pp. 1580-1585 ◽  
Author(s):  
Nathan M. Hitchens ◽  
Harold E. Brooks
Keyword(s):  
Probability Of Detection ◽  
Grid Spacing ◽  
Forecast Performance ◽  
Critical Success ◽  
Initial Effort ◽  
Detection Frequency ◽  
Storm Prediction ◽  
Critical Success Index ◽  
Success Index

Abstract The Storm Prediction Center has issued daily convective outlooks since the mid-1950s. This paper represents an initial effort to examine the quality of these forecasts. Convective outlooks are plotted on a latitude–longitude grid with 80-km grid spacing and evaluated using storm reports to calculate verification measures including the probability of detection, frequency of hits, and critical success index. Results show distinct improvements in forecast performance over the duration of the study period, some of which can be attributed to apparent changes in forecasting philosophies.

Visualizing Multiple Measures of Forecast Quality

2009 ◽  
Vol 24 (2) ◽  
pp. 601-608 ◽  
Author(s):  
Paul J. Roebber
Keyword(s):  
Heavy Precipitation ◽  
Probability Of Detection ◽  
Ensemble Forecasts ◽  
Convective Initiation ◽  
Multiple Measures ◽  
Critical Success ◽  
Performance Statistics ◽  
Critical Success Index ◽  
Success Index ◽  
Forecast Quality

Abstract A method for visually representing multiple measures of dichotomous (yes–no) forecast quality (probability of detection, false alarm ratio, bias, and critical success index) in a single diagram is presented. Illustration of the method is provided using performance statistics from two previously published forecast verification studies (snowfall density and convective initiation) and a verification of several new forecast datasets: Storm Prediction Center forecasts of severe storms (nontornadic and tornadic), Hydrometeorological Prediction Center forecasts of heavy precipitation (greater than 12.5 mm in a 6-h period), National Weather Service Forecast Office terminal aviation forecasts (ceiling and visibility), and medium-range ensemble forecasts of 500-hPa height anomalies. The use of such verification metrics in concert with more detailed investigations to advance forecasting is briefly discussed.

scholarly journals Assessment of SM2RAIN-ASCAT and CMORPH satellite precipitation products over Maharlu Lake basin in Iran

2020 ◽  
Vol 20 (5) ◽  
pp. 1799-1806
Author(s):  
Fatemeh Moazami Goudarzi ◽  
Amirpouya Sarraf ◽  
Hassan Ahmadi
Keyword(s):  
Climate Modeling ◽  
Probability Of Detection ◽  
Coefficient Of Determination ◽  
Lake Basin ◽  
Error Statistics ◽  
Critical Success ◽  
Monthly Changes ◽  
Semi Arid Region ◽  
Critical Success Index ◽  
Total Accuracy

Abstract In recent years, the use of climatic databases and satellite products by researchers has become increasingly common in the field of climate modeling and research. These datasets play an important role in developing countries. This study evaluated two reanalyses, CMORPH and SM2RAIN-ASCAT over Maharlu Lake, a semi-arid region in Iran. The results showed that these two near-time datasets do not have accurate data over this basin. However, the probability of detection (POD), critical success index (CSI), and false alarm ratio (FAR) statistics showed acceptable accuracy in the detection of precipitation. The coefficient of determination and root mean square error statistics have unacceptable accuracy over this area. The monthly changes in each of the indices showed that the CMORPH database had more errors in the spring months, but in other months the error rate was improved. SM2RAIN-ASCAT had better accuracy over this area relative to CMORPH. The estimation of the total accuracy of the data showed that these two satellite databases were not capable of estimating precipitation in the area.

scholarly journals A comparison between high-resolution satellite precipitation estimates and gauge measured data: case study of Gorganrood basin, Iran

2018 ◽  
Vol 67 (3) ◽  
pp. 236-251 ◽  
Author(s):  
Donya Dezfooli ◽  
Banafsheh Abdollahi ◽  
Seyed-Mohammad Hosseini-Moghari ◽  
Kumars Ebrahimi
Keyword(s):  
Pearson Correlation ◽  
Absolute Error ◽  
Probability Of Detection ◽  
Critical Success ◽  
Precipitation Estimates ◽  
Seasonal Basis ◽  
Precipitation Gauge ◽  
Critical Success Index ◽  
Trmm 3B42

Abstract The aim of this paper is to evaluate the accuracy of the precipitation data gathered from satellites including PERSIANN, TRMM-3B42V7, TRMM-3B42RTV7, and CMORPH, over Gorganrood basin, Iran. The data collected from these satellites (2003–2007) were then compared with precipitation gauge observations at six stations, namely, Tamar, Ramiyan, Bahlakeh-Dashli, Sadegorgan, Fazel-Abad, and Ghaffar-Haji. To compare these two groups, mean absolute error (MAE), bias, root mean square error (RMSE), and Pearson correlation coefficient criteria were calculated on daily, monthly, and seasonal basis. Furthermore, probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI) were calculated for these datasets. Results indicate that, on a monthly scale, the highest correlation between observed and satellite-gathered data calculated is 0.404 for TRMM-3B42 at Bahlakeh-Dashli station. At a seasonal scale, the highest correlation is calculated for winter data and using PERSIANN data, while for the other seasons, TRMM-3B42 data showed the best correlation with observed data. The high values of RMSE and MAE for winter data showed that the satellites provided poor estimations at this season. The best and the worst values of RMSE for studied satellites belonged to Sadegorgan and Ramiyan stations, respectively. Furthermore, the PERSIANN gains a better CSI and POD while TRMM-3B42V7 showed a better FAR.

scholarly journals PEMODELAN ATMOSFER DENGAN WRF PADA KEJADIAN BANJIR JAKARTA 17 JANUARI 2013

2013 ◽  
Vol 14 (1) ◽  
pp. 29
Author(s):  
Ardhi Adhary Arbain ◽  
Mahally Kudsy ◽  
M. Djazim Syaifullah
Keyword(s):  
False Alarm ◽  
Extreme Precipitation ◽  
Model Performance ◽  
Probability Of Detection ◽  
Ncep Reanalysis ◽  
Cold Surge ◽  
Qualitative And Quantitative ◽  
Critical Success ◽  
Critical Success Index ◽  
Success Index

Intisari  Simulasi WRF pada tanggal 16-17 Januari 2013 dilakukan untuk menguji performa model dalam mendeteksi fenomena seruak dingin dan hujan ekstrim yang merupakan pemicu utama bencana banjir Jakarta pada periode tersebut. Metode verifikasi kualitatif dan kuantitatif pada tiap grid secara dikotomi digunakan untuk membandingkan keluaran model dengan data observasi Global Satellite Mapping of Precipitation (GSMaP) dan NCEP Reanalysis. Performa model WRF dihitung berdasarkan nilai akurasi (ACC), Critical Success Index (CSI), Probability of Detection (POD) dan False Alarm Ratio (FAR) yang diperoleh dari hasil verifikasi numerik. Hasil pengujian menunjukkan bahwa WRF mampu melakukan deteksi waktu awal kejadian hujan ekstrim dengan tepat setelah 6-7 jam sejak inisiasi model dilakukan. Performa terbaik WRF teramati pada pukul 02-09 WIB (LT) dengan nilai CSI mencapai 0,32, POD 0,82 dan FAR 0,66. Hasil verifikasi secara kualitatif dan kuantitatif juga menunjukkan bahwa WRF dapat melakukan deteksi seruak dingin dan hujan ekstrim sebelum banjir terjadi, walaupun dengan ketepatan durasi waktu dan lokasi kejadian yang masih relatif rendah bila dibandingkan dengan data observasi.  Abstract  WRF simulation on January 16-17, 2013 has been conducted to evaluate the model performance in detecting cold surge and extreme precipitation phenomena which were the triggers of Jakarta flood event during the period. Qualitative and quantitative dichotomous grid-to-grid verification methods are utilized to compare the model output with Global Satellite Mapping of Precipitation (GSMaP) observation and NCEP Reanalysis dataset. WRF model performance is calculated based on the scores of accuracy (ACC), Critical Success Index (CSI), Probability of Detection (POD) and False Alarm Ration (FAR) which are generated from numerical verification. The results show that WRF could precisely detect the onset of extreme precipitation event in 6-7 hours after the model initiation.The best performance of the model is observed at 02-09 WIB (LT) with CSI score of 0.32, POD 0.82 and FAR 0.66. Despite the model inability to accurately predict the duration and location of cold surge and extreme precipitation, the qualitative and quantitative verification results also show that WRF could detect the phenomena just before the flood event occured.

Sign in / Sign up

Export Citation Format

Share Document