Seasonal Weather Forecast Verification : Final Report.

Abstract The problem of probabilistic forecast verification is approached from a theoretical point of view starting from three basic desiderata: additivity, exclusive dependence on physical observations (“locality”), and strictly proper behavior. By imposing such requirements and only using elementary mathematics, a univocal measure of forecast goodness is demonstrated to exist. This measure is the logarithmic score, based on the relative entropy between the observed occurrence frequencies and the predicted probabilities for the forecast events. Information theory is then used as a guide to choose the scoring-scale offset for obtaining meaningful and fair skill scores. Finally the Brier score is assessed and, for single-event forecasts, its equivalence to the second-order approximation of the logarithmic score is shown. The large part of the presented results are far from being new or original, nevertheless their use still meets with some resistance in the weather forecast community. This paper aims at providing a clear presentation of the main arguments for using the logarithmic score.

Download Full-text

An analysis and case study of using mobile crowdsourcing for weather forecast verification

2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS) ◽

10.1109/icsess.2015.7339048 ◽

2015 ◽

Cited By ~ 1

Author(s):

Nuwee Wiwatwattana ◽

Kritsada Sangkhatad ◽

Thitima Srivatanakul

Keyword(s):

Weather Forecast ◽

Forecast Verification ◽

Mobile Crowdsourcing

Download Full-text

An Object-Based Forecast Verification Tool for Synoptic-Scale Rossby Waveguides

Weather and Forecasting ◽

10.1175/waf-d-15-0147.1 ◽

2016 ◽

Vol 31 (3) ◽

pp. 937-946 ◽

Cited By ~ 9

Author(s):

Paraskevi Giannakaki ◽

Olivia Martius

Keyword(s):

Rossby Waves ◽

Weather Forecast ◽

Synoptic Scale ◽

Forecast Verification ◽

Complex Objects ◽

Reanalysis Dataset ◽

Verification Tool ◽

Object Based ◽

Gradient Fields ◽

Forecast Models

Abstract An accurate representation of synoptic-scale Rossby waves in numerical weather forecast models is very important as these waves are closely linked to weather formation at the surface. Enhanced potential vorticity (PV) gradients at the tropopause levels act as waveguides for synoptic-scale Rossby waves, so spatial errors in the waveguides imply errors in the amplification and propagation of Rossby waves. This paper focuses on evaluating the forecast representation of these waveguides and presents an object-based forecast verification tool. In both forecast and the verification data, Rossby waveguide objects are defined based on enhanced PV gradient fields on isentropic surfaces. The tool automatically pairs the complex objects, compares their properties, and assesses the number of objects without a matching partner in either the forecast or the reanalysis. In the last step, error measures are calculated for the area and the location of the objects. As proof-of-concept application of the method for the year 2008, five lead times of the Integrated Forecast System (IFS) from the ECMWF are compared with the ECMWF reanalysis dataset. The majority of the waveguide objects are found to be in the correct position, and there are no systematic positional errors; however, the forecast objects and hence the areas of enhanced PV gradients are smaller.

Download Full-text

Accessing seasonal weather forecasts and drought prediction information for rural households in Chirumhanzu district, Zimbabwe

Jàmbá Journal of Disaster Risk Studies ◽

10.4102/jamba.v11i1.777 ◽

2019 ◽

Vol 11 (1) ◽

Author(s):

Mashoko S. Grey

Keyword(s):

Indigenous Knowledge ◽

Early Warning ◽

Rural Households ◽

Climate Science ◽

Weather Forecast ◽

Early Warning Systems ◽

Weather Forecasts ◽

Drought Hazard ◽

Drought Prediction ◽

Seasonal Weather

Seasonal weather forecasts and drought hazard prediction through media sources and indigenous knowledge help provide an understanding of early warning systems and the preferred source information by rural households. This article focuses on the investigation of households’ access to weather forecasts and drought hazard prediction information as early warning to reduce drought risk on livelihood activities. The study was carried out in Chirumhanzu district, and the methods used for data collection included 217 household surveys, six focus group discussions, key informants’ interviews and document review. The study found that the majority of the households in the study area had access to seasonal weather forecast information (scientific), which almost half of the respondents received through radios. However, vulnerability to climate risks was exacerbated by seasonal weather forecasts, which were deemed by some households to be unreliable, inaccurate and not easily understood. In this regard, some households used indigenous knowledge to inform them on the status of the incoming rainy season and drought prediction. The use of indigenous knowledge depended on individuals’ ability to read and decode natural indicators of seasonal weather forecast and drought prediction. Indigenous knowledge is valuable for climate science as it enhances observations and interpretations on a larger spatial scale with considerable temporal depth by highlighting elements that are measured by climate science. Both scientific weather information and indigenous knowledge are important for seasonal weather forecasting and drought prediction, especially in rural settings, and complement each other if used and availed timely to households.

Download Full-text

Review article: Observations for high-impact weather and their use in verification

Natural Hazards and Earth System Science ◽

10.5194/nhess-21-1297-2021 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1297-1312

Author(s):

Chiara Marsigli ◽

Elizabeth Ebert ◽

Raghavendra Ashrit ◽

Barbara Casati ◽

Jing Chen ◽

...

Keyword(s):

Weather Forecast ◽

Review Article ◽

High Impact ◽

Forecast Evaluation ◽

Insurance Companies ◽

Forecast Verification ◽

Telecommunication Systems ◽

Open Questions ◽

High Impact Weather ◽

Definition Of

Abstract. Verification of forecasts and warnings of high-impact weather is needed by the meteorological centres, but how to perform it still presents many open questions, starting from which data are suitable as reference. This paper reviews new observations which can be considered for the verification of high-impact weather and provides advice for their usage in objective verification. Two high-impact weather phenomena are considered: thunderstorm and fog. First, a framework for the verification of high-impact weather is proposed, including the definition of forecast and observations in this context and creation of a verification set. Then, new observations showing a potential for the detection and quantification of high-impact weather are reviewed, including remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and claim/damage reports from insurance companies. The observation characteristics which are relevant for their usage in forecast verification are also discussed. Examples of forecast evaluation and verification are then presented, highlighting the methods which can be adopted to address the issues posed by the usage of these non-conventional observations and objectively quantify the skill of a high-impact weather forecast.

Download Full-text

Weather Forecast Verification

CISM International Centre for Mechanical Sciences - Atmospheric Convection: Research and Operational Forecasting Aspects ◽

10.1007/978-3-211-69291-2_17 ◽

2007 ◽

pp. 195-222

Author(s):

Fulvio Stel ◽

Dario B. Giaiotti

Keyword(s):

Weather Forecast ◽

Forecast Verification

Download Full-text

Probabilistic Contingency Tables: An Improvement to Verify Probability Forecasts

Weather and Forecasting ◽

10.1175/waf-d-19-0116.1 ◽

2019 ◽

Vol 35 (2) ◽

pp. 609-621 ◽

Cited By ~ 1

Author(s):

Sarah Gold ◽

Edward White ◽

William Roeder ◽

Mike McAleenan ◽

Christine Schubert Kabban ◽

...

Keyword(s):

Contingency Table ◽

Performance Metrics ◽

Weather Forecast ◽

Forecast Verification ◽

Forecast Performance ◽

Binary Event ◽

Probabilistic Forecasts ◽

Verification Process ◽

Statistical Metrics

Abstract The 45th Weather Squadron (45 WS) records daily rain and lightning probabilistic forecasts and the associated binary event outcomes. Subsequently, they evaluate forecast performance and determine necessary adjustments with an established verification process. For deterministic outcomes, weather forecast analysis typically utilizes a traditional contingency table (TCT) for verification; however, the 45 WS uses an alternative tool, the probabilistic contingency table (PCT). Using the TCT for verification requires a threshold, typically at 50%, to dichotomize probabilistic forecasts. The PCT maintains the valuable information in probabilities and verifies the true forecasts being reported. Simulated forecasts and outcomes as well as 2015–18 45 WS data are utilized to compare forecast performance metrics produced from the TCT and PCT to determine which verification tool better reflects the quality of forecasts. Comparisons of frequency bias and other statistical metrics computed from both dichotomized and continuous forecasts reveal misrepresentative performance metrics from the TCT as well as a loss of information necessary for verification. PCT bias better reflects forecast verification in contrast to that of TCT bias, which suggests suboptimal forecasts when in fact the forecasts are accurate.

Download Full-text

Observations for high-impact weather and their use in verification

10.5194/nhess-2020-362 ◽

2020 ◽

Author(s):

Chiara Marsigli ◽

Elizabeth Ebert ◽

Raghavendra Ashrit ◽

Barbara Casati ◽

Jing Chen ◽

...

Keyword(s):

Weather Forecast ◽

High Impact ◽

Forecast Evaluation ◽

Insurance Companies ◽

Forecast Verification ◽

Telecommunication Systems ◽

Open Questions ◽

High Impact Weather ◽

Definition Of ◽

Detection And Quantification

Abstract. Verification of high-impact weather is needed by the Meteorological Centres, but how to perform it still presents many open questions, starting from which data are suitable as reference. This paper reviews new observations which can be considered for the verification of high-impact weather, and provides advice for their usage in objective verification. Two high-impact weather phenomena are considered: Thunderstorm and fog. First, a framework for the verification of high-impact weather is proposed, including the definition of forecast and observations in this context and creation of a verification set. Then, new observations showing a potential for the detection and quantification of high-impact weather are reviewed, including remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and claim/damage reports from insurance companies. The observation characteristics which are relevant for their usage in forecast verification are also discussed. Examples of forecast evaluation and verification are then presented, highlighting the methods which can be adopted to address the issues posed by the usage of these non-conventional observations and objectively quantify the skill of a high-impact weather forecast.

Download Full-text