Using Metocean Forecast Verification Information to Effectively Enhance Operational Decision-Making

Metocean forecast verification statistics (or ‘skill scores’), for variables such as significant wave height, are typically computed as a means of assessing the (past) weather model performance over the particular area of interest. For developers, this information is important for the measurement of model improvement, while for consumers this is commonly applied for the comparison/evaluation of potential service providers. However, an opportunity missed by many is also its considerable benefit to users in enhancing operational decision-making on a real-time (future) basis, when combined with an awareness of the context of the specific decision being made. Here, we present two categorical verification techniques and demonstrate their application in simplifying the interpretation of ensemble (probabilistic) wave forecasts out to 15 days ahead, as pioneered – in operation – in Summer 2020 to support the recent weather sensitive installation of the first phase of a 36 km subsea pipeline in the Fenja field in the North Sea. Categorical verification information (based on whether forecast and observations exceed the user-defined operational weather limits) was constructed from 1460 archive wave forecasts, issued for the two-year period 2017 to 2018, and used to characterise the past performance of the European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble prediction system (EPS) in the form of Receiver Operating Characteristic (ROC) and Relative Economic Value (REV) analysis. These data were then combined with a bespoke parameterization of the impact of adverse weather on the planned operation, allowing the relevant go/no-go ensemble probability threshold (i.e. the number of individual/constituent forecast members that must predict favourable/unfavourable conditions) for the interpretation of future forecasts to be determined. Following the computation of the probability thresholds for the Fenja location, trials on an unseen nine-month period of data from the site (Spring to Autumn 2019) confirm these approaches facilitate a simple technique for processing/interpreting the ensemble forecast, able to be readily tailored to the particular decision being made. The use of these methods achieves a considerably greater value (benefit) than equivalent deterministic (single) forecasts or traditional climate-based options at all lead times up to 15 days ahead, promising a more robust basis for effective planning than typically considered by the offshore industry. This is particularly important for tasks requiring early identification of long weather windows (e.g. for the Fenja tie-ins), but similarly relevant for maximising the exploitation of any ensemble forecast, providing a practical approach for how such data are handled and used to promote safe, efficient and successful operations.

PREDICTION OF CUMULONIMBUS (CB) CLOUD BASED ON INTEGRATED FORECAST SYSTEM (IFS) OF EUROPEAN MEDIUM-RANGE WEATHER FORECAST (ECMWF) IN THE FLIGHT INFORMATION REGION (FIR) OF JAKARTA AND UJUNG PANDANG

This study was focused on cumulonimbus (Cb) cloud prediction based on Integrated Forecast System (IFS) European Medium-Range Weather Forecast (ECMWF) model in the Flight Information Region (FIRs) Jakarta and Ujung Pandang. The Cb cloud prediction was calculated using convective cloud cover (CC) of the precipitation product. The model predictability was examined through categorical verification. The Cb cloud observation was based on brightness temperature (BT) IR1 and brightness temperature difference (BTD) IR1-IR2. The results showed that CC 50%' predictor was the best predictor to estimate the Cb cloud. The study in the period other than 2019 is suggested for the next research because Indian Ocean Dipole (IOD) is extreme that may affect the Cb cloud growth in the study area.

Comprehensive Evaluation of GPM-IMERG, CMORPH, and TMPA Precipitation Products with Gauged Rainfall over Mainland China

The comprehensive assessment of the Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (IMERG) V05B is important for benchmarking the product’s continued improvement and future development. The performance of IMERG V05B precipitation products was systematically evaluated using 542 precipitation gauges at multiple spatiotemporal scales from March 2014 to February 2017 over China. Moreover, IMERG V05B was compared with IMERG V04A, the Tropical Rainfall Measuring Mission (TRMM) 3B42, and the Climate Prediction Center Morphing technique (CMORPH)-CRT in this study. Categorical verification techniques and statistical methods are used to quantify their performance. Results illustrate the following. (1) Except for IMERG V04A’s severe underestimation over the Tibetan Plateau (TP) and Xinjiang (XJ) with high negative relative biases (RBs) and CMORPH-CRT’s overestimation over XJ with high positive RB, the four satellite-based precipitation products generally capture the same spatial patterns of precipitation over China. (2) At the annual scale over China, the IMERG products do not show an advantage over its predecessor (TRMM 3B42) in terms of RMSEs, RRMSEs, and Rs; meanwhile, the performance of IMERG products is worse than TRMM 3B42 in spring and summer according to the RMSE, RRMSE, and R metrics. Between the two IMERG products, IMERG V05B shows the anticipated improvement (over IMERG V04A) with a decrease in RMSE from 0.4496 to 0.4097 mm/day, a decrease of RRMSE from 16.95% to 15.44%, and an increase of R from 0.9689 to 0.9759 during the whole study period. Similar results are obtained at the seasonal scale. Among the four satellite products, CMORPH-CRT shows the worst seasonal performance with the highest RMSE (0.6247 mm/day), RRMSE (23.55%), and lowest R (0.9343) over China. (3) Over XJ and TP, IMERG V05B clearly improves the strong underestimation of precipitation in IMERG V04A with the RBs of 5.2% vs. −21.8% over XJ, and 2.78% vs. −46% over TP. Results at the annual scale are similar to those obtained at the seasonal scale, except for summer results over XJ. While, over the remaining subregions, the two IMERG products have a close performance; meanwhile, IMERG V04A slightly improves IMERG V05B’s overestimation according to RBs (except for HN) at the annual scale. However, all four products are unreliable over XJ at both an annual and seasonal scale. (4) Across all products, TRMM 3B42 best reproduces the probability density function (PDF) of daily precipitation intensity. (5) According to the categorical verification technique in this study, both IMERG products yield better results for the detection of precipitation events on the basis of probability of detection (POD) and critical success index (CSI) categorical evaluations compared to TRMM 3B42 and CMORPH-CRT over China and across most of the subregions. However, all four products have room for further improvement, especially in high-latitude and dry climate regions. These findings provide valuable feedback for both IMERG algorithm developers and data set users.

Evaluation of high-resolution precipitation analyses using a dense station network

The ability of radar–rain gauge merging algorithms to precisely analyse convective precipitation patterns is of high interest for many applications, e.g. hydrological modelling, thunderstorm warnings, and, as a reference, to spatially validate numerical weather prediction models. However, due to drawbacks of methods like cross-validation and due to the limited availability of reference data sets on high temporal and spatial scales, an adequate validation is usually hardly possible, especially on an operational basis. The present study evaluates the skill of very high-resolution and frequently updated precipitation analyses (rapid-INCA) by means of a very dense weather station network (WegenerNet), operated in a limited domain of the southeastern parts of Austria (Styria). Based on case studies and a longer-term validation over the convective season 2011, a general underestimation of the rapid-INCA precipitation amounts is shown by both continuous and categorical verification measures, although the temporal and spatial variability of the errors is – by convective nature – high. The contribution of the rain gauge measurements to the analysis skill is crucial. However, the capability of the analyses to precisely assess the convective precipitation distribution predominantly depends on the representativeness of the stations under the prevalent convective condition.

Impacts of Forecaster Involvement on Convective Storm Initiation and Evolution Nowcasting

A forecaster-interactive capability was added to an automated convective storm nowcasting system [Auto-Nowcaster (ANC)] to allow forecasters to enhance the performance of 1-h nowcasts of convective storm initiation and evolution produced every 6 min. This Forecaster-Over-The-Loop (FOTL-ANC) system was tested at the National Weather Service Fort Worth–Dallas, Texas, Weather Forecast Office during daily operations from 2005 to 2010. The forecaster’s role was to enter the locations of surface convergence boundaries into the ANC prior to dissemination of nowcasts to the Center Weather Service Unit. Verification of the FOTL-ANC versus ANC (no human) nowcasts was conducted on the convective scale. Categorical verification scores were computed for 30 subdomains within the forecast domain. Special focus was placed on subdomains that included convergence boundaries for evaluation of forecaster involvement and impact on the FOTL-ANC nowcasts. The probability of detection of convective storms increased by 20%–60% with little to no change observed in the false-alarm ratios. Bias values increased from 0.8–1.0 to 1.0–3.0 with human involvement. The accuracy of storm nowcasts notably improved with forecaster involvement; critical success index (CSI) values increased from 0.15–0.25 (ANC) to 0.2–0.4 (FOTL-ANC). Over short time periods, CSI values as large as 0.6 were also observed. This study demonstrated definitively that forecaster involvement led to positive improvement in the nowcasts in most cases while causing no degradation in other cases; a few exceptions are noted. Results show that forecasters can play an important role in the production of rapidly updated, convective storm nowcasts for end users.

The skill of convective parameters and indices to predict isolated and severe thunderstorms

The preconvective environment on days with ordinary, widespread, and severe thunderstorms in Southwest Germany was investigated. Various thermodynamic and kinetic parameters calculated from radiosoundings at 12:00 UTC were verified against subsequent thunderstorm observations derived from SYNOP station data, radar data, and damage reports of a building insurance company. The skill of the convective parameters and indices to predict thunderstorms was evaluated by means of probability distribution functions, probabilities of thunderstorms according to an index threshold, and skill scores like the Heidke Skill Score (HSS) that are based on categorical verification. For the ordinary decision as to whether a thunderstorm day was expected or not, the best results were obtained with the original Lifted Index (80% prediction probability for LI≤−1.73; HSS=0.57 for LI≤1.76), the Showalter Index, and the modified K-Index. Considering days with isolated compared to widespread thunderstorms, the best performance is reached by the Deep Convective Index. For days with severe thunderstorms that caused damage due to hail, local storms or floods, the best prediction skill is found again for the Lifted Index and the Deep Convective Index, but also for the Potential Instability Index, the Delta-θe Index, and a version of the CAPE, where the lifting profile is determined by averaging over the lowest 100 hPa.

Explicitly Accounting for Observation Error in Categorical Verification of Forecasts

Given an accurate representation of errors in observations it is possible to remove the effect of those errors from categorical verification scores. The errors in the observations are treated as additive white noise that is statistically independent of the true value of the quantity being observed. This method can be applied to both probabilistic and deterministic verification where the verification method uses a categorical approach. In general this improves the apparent performance of a forecasting system, indicating that forecasting systems are often performing better than they might first appear.