scholarly journals Making forecasters SAD: Verification of Scale, Anisotropy and Direction using wavelets.

2021 ◽  
Author(s):  
Sebastian Buschow ◽  
Petra Friederichs
Keyword(s):  
Wavelet Transform ◽  
Spatial Structure ◽  
Wavelet Transformation ◽  
Numerical Models ◽  
Spatial Scales ◽  
Forecast Model ◽  
Post Processing ◽  
Meteorological Field ◽  
Verification Methods ◽  
Atmospheric Phenomena

<p>Many atmospheric phenomena like fronts, convection and turbulence leave a distinct imprint on the spatial structure of meteorological fields such as precipitation, wind and temperature. Whether or not a forecast model is able to realistically simulate the resulting spatial correlation patterns is therefore a relevant question for model developers, forecasters and end users alike. Highly resolved numerical models have the potential to achieve this goal, but their realism is often difficult to assess objectively due to the sheer amount of data and wide variety of possible error contributions. </p><p>While some existing verification methods measure an overall “structure” error, most of these approaches are limited to precipitation fields and fail to produce specific, interpretable judgements. Here, we introduce a new structural verification technique based on the dual-tree complex wavelet transformation: The SAD-scores explicitly quantify how well the observed spatial Scales, degrees of Anisotropy and preferred Directions are represented by the simulation. Directional aspects in particular have previously often been neglected, but can be important in assessing the realism of predicted fronts, convergence lines and organized convection. </p><p>Unlike many established techniques, SAD is applicable not only to precipitation but to any meteorological field of interest. General verdicts like “the structure was predicted poorly” can be resolved into specific statements like “the modelled convection was too small in scale” or “the simulated front was too linear and rotated by an angle of X degrees”. The localized nature of the wavelets furthermore allows us to conveniently display the structural properties on a map. Lastly, making use of the inverse wavelet transform, we show how the detected structural errors can potentially be corrected, thereby leading the way towards future post-processing applications.</p>

scholarly journals New Developments of the Intensity-Scale Technique within the Spatial Verification Methods Intercomparison Project

2010 ◽  
Vol 25 (1) ◽  
pp. 113-143 ◽  
Author(s):  
B. Casati
Keyword(s):  
Case Studies ◽  
Single Case ◽  
Spatial Scales ◽  
Forecast Model ◽  
Skill Score ◽  
Scale Structure ◽  
Discrete Wavelet ◽  
Intensity Scale ◽  
Verification Methods ◽  
Domain Constraints

Abstract The intensity-scale verification technique introduced in 2004 by Casati, Ross, and Stephenson is revisited and improved. Recalibration is no longer performed, and the intensity-scale skill score for biased forecasts is evaluated. Energy and its percentages are introduced in order to assess the bias on different scales and to characterize the overall scale structure of the precipitation fields. Aggregation of the intensity-scale statistics for multiple cases is performed, and confidence intervals are provided by bootstrapping. Four different approaches for addressing the dyadic domain constraints are illustrated and critically compared. The intensity-scale verification is applied to the case studies of the Intercomparison of Spatial Forecast Verification Methods Project. The geometric and synthetically perturbed cases show that the intensity-scale verification statistics are sensitive to displacement and bias errors. The intensity-scale skill score assesses the skill for different precipitation intensities and on different spatial scales, separately. The spatial scales of the error are attributed to both the size of the features and their displacement. The energy percentages allow one to objectively analyze the scale structure of the fields and to understand the intensity-scale relationship. Aggregated statistics for the Storm Prediction Center/National Severe Storms Laboratory (SPC/NSSL) 2005 Spring Program case studies show no significant differences among the models’ skill; however, the 4-km simulations of the NCEP version of the Weather Research and Forecast model (WRF4 NCEP) overforecast to a greater extent than the 2- and 4-km simulations of the NCAR version of the WRF (WRF2 and WRF4 NCAR). For the aggregated multiple cases, the different approaches addressing the dyadic domain constraints lead to similar results. On the other hand, for a single case, tiling provides the most robust and reliable approach, since it smoothes the effects of the discrete wavelet support and does not alter the original precipitation fields.

scholarly journals Intercomparison of Spatial Forecast Verification Methods: Identifying Skillful Spatial Scales Using the Fractions Skill Score

2010 ◽  
Vol 25 (1) ◽  
pp. 343-354 ◽  
Author(s):  
Marion Mittermaier ◽  
Nigel Roberts
Keyword(s):  
Spatial Scales ◽  
Wrf Model ◽  
Skill Score ◽  
Area Ratio ◽  
Forecast Verification ◽  
Care Needs ◽  
Forecast Performance ◽  
Verification Methods ◽  
Upper Level ◽  
Formed Part

Abstract The fractions skill score (FSS) was one of the measures that formed part of the Intercomparison of Spatial Forecast Verification Methods project. The FSS was used to assess a common dataset that consisted of real and perturbed Weather Research and Forecasting (WRF) model precipitation forecasts, as well as geometric cases. These datasets are all based on the NCEP 240 grid, which translates to approximately 4-km resolution over the contiguous United States. The geometric cases showed that the FSS can provide a truthful assessment of displacement errors and forecast skill. In addition, the FSS can be used to determine the scale at which an acceptable level of skill is reached and this usage is perhaps more helpful than interpreting the actual FSS value. This spatial-scale approach is becoming more popular for monitoring operational forecast performance. The study also shows how the FSS responds to forecast bias. A more biased forecast always gives lower FSS values at large scales and usually at smaller scales. It is possible, however, for a more biased forecast to give a higher score at smaller scales, when additional rain overlaps the observed rain. However, given a sufficiently large sample of forecasts, a more biased forecast system will score lower. The use of percentile thresholds can remove the impacts of the bias. When the proportion of the domain that is “wet” (the wet-area ratio) is small, subtle differences introduced through near-threshold misses can lead to large changes in FSS magnitude in individual cases (primarily because the bias is changed). Reliable statistics for small wet-area ratios require a larger sample of forecasts. Care needs to be taken in the choice of verification domain. For high-resolution models, the domain should be large enough to encompass the length scale of the typical mesoscale forcing (e.g., upper-level troughs or squall lines). If the domain is too large, the wet-area ratios will always be small. If the domain is too small, fluctuations in the wet-area ratio can be large and larger spatial errors may be missed. The FSS is a good measure of the spatial accuracy of precipitation forecasts. Different methods are needed to determine other patterns of behavior.

Skill assessment of post-processing methods for ECMWF SEAS5 seasonal forecasts over Europe

2021 ◽  
Author(s):  
Alice Crespi ◽  
Marcello Petitta ◽  
Lucas Grigis ◽  
Paola Marson ◽  
Jean-Michel Soubeyroux ◽  
...  
Keyword(s):  
Large Scale ◽  
Production Management ◽  
Spatial Scales ◽  
Research Field ◽  
Skill Assessment ◽  
Post Processing ◽  
Seasonal Forecasts ◽  
Climate Services ◽  
Local Climate ◽  
Climate Conditions

<p>Seasonal forecasts provide information on climate conditions several months ahead and therefore they could represent a valuable support for decision making, warning systems as well as for the optimization of industry and energy sectors. However, forecast systems can be affected by systematic biases and have horizontal resolutions which are typically coarser than the spatial scales of the practical applications. For this reason, the reliability of forecasts needs to be carefully assessed before applying and interpreting them for specific applications. In addition, the use of post-processing approaches is recommended in order to improve the representativeness of the large-scale predictions of regional and local climate conditions. The development and evaluation downscaling and bias-correction procedures aiming at improving the skills of the forecasts and the quality of derived climate services is currently an open research field. In this context, we evaluated the skills of ECMWF SEAS5 forecasts of monthly mean temperature, total precipitation and wind speed over Europe and we assessed the skill improvements of calibrated predictions.</p><p>For the calibration, we combined a bilinear interpolation and a quantile mapping approach to obtain corrected monthly forecasts on a 0.25°x0.25° grid from the original 1°x1° values. The forecasts were corrected against the reference ERA5 reanalysis over the hindcast period 1993–2016. The processed forecasts were compared over the same domain and period with another calibrated set of ECMWF SEAS5 forecasts obtained by the ADAMONT statistical method.</p><p>The skill assessment was performed by means of both deterministic and probabilistic verification metrics evaluated over seasonal forecasted aggregations for the first lead time. Greater skills of the forecast systems in Europe were generally observed in spring and summer, especially for temperature, with a spatial distribution varying with the seasons. The calibration was proved to effectively correct the model biases for all variables, however the metrics not accounting for bias did not show significant improvements in most cases, and in some areas and seasons even small degradations in skills were observed.</p><p>The presented study supported the activities of the H2020 European project SECLI-FIRM on the improvement of the seasonal forecast applicability for energy production, management and assessment.</p>

scholarly journals Evaluation of numerical models by FerryBox and fixed platform in situ data in the southern North Sea

Ocean Science ◽  
2015 ◽  
Vol 11 (6) ◽  
pp. 879-896 ◽  
Author(s):  
M. Haller ◽  
F. Janssen ◽  
J. Siddorn ◽  
W. Petersen ◽  
S. Dick
Keyword(s):  
North Sea ◽  
High Frequency ◽  
Numerical Models ◽  
Spatial Scales ◽  
Model Performance ◽  
Eastern Coast ◽  
Southern North Sea ◽  
In Situ Data ◽  
Oyster Ground ◽  
Study Results

Abstract. For understanding and forecasting of hydrodynamics in coastal regions, numerical models have served as an important tool for many years. In order to assess the model performance, we compared simulations to observational data of water temperature and salinity. Observations were available from FerryBox transects in the southern North Sea and, additionally, from a fixed platform of the MARNET network. More detailed analyses have been made at three different stations, located off the English eastern coast, at the Oyster Ground and in the German Bight. FerryBoxes installed on ships of opportunity (SoO) provide high-frequency surface measurements along selected tracks on a regular basis. The results of two operational hydrodynamic models have been evaluated for two different time periods: BSHcmod v4 (January 2009 to April 2012) and FOAM AMM7 NEMO (April 2011 to April 2012). While they adequately simulate temperature, both models underestimate salinity, especially near the coast in the southern North Sea. Statistical errors differ between the two models and between the measured parameters. The root mean square error (RMSE) of water temperatures amounts to 0.72 °C (BSHcmod v4) and 0.44 °C (AMM7), while for salinity the performance of BSHcmod is slightly better (0.68 compared to 1.1). The study results reveal weaknesses in both models, in terms of variability, absolute levels and limited spatial resolution. Simulation of the transition zone between the coasts and the open sea is still a demanding task for operational modelling. Thus, FerryBox data, combined with other observations with differing temporal and spatial scales, can serve as an invaluable tool not only for model evaluation, but also for model optimization by assimilation of such high-frequency observations.

scholarly journals Denoising of Laser Scanning Data Using Wavelet

2014 ◽  
Vol 12 ◽  
pp. 41-47 ◽  
Author(s):  
Petr Jašek ◽  
Martin Štroner
Keyword(s):  
Image Processing ◽  
Laser Scanning ◽  
Wavelet Transformation ◽  
Terrestrial Laser Scanning ◽  
Range Image ◽  
Regular Matrix ◽  
Post Processing ◽  
Spatial Coordinates ◽  
Measured Distance ◽  
Made In

Regarding the terrestrial laser scanning accuracy, one of the main problems is the noise in measured distance which is necessary for the spatial coordinates´ determination. In this paper the technique of using the wavelet transformation for the reduction of the noise in the laser scanning data is described. This method of filtration is made in “post processing” and due to this fact any changes in the measuring procedure in the field shouldn´t be done. The creation of the regular matrix is needed to apply image processing. This matrix then makes the range image. In the paper real and simulated efficiency tests of wavelet transformation, the final summary and advantages or disadvantages of this method are introduced.

scholarly journals Wavelet-based spatial comparison technique for analysing and evaluating two-dimensional geophysical model fields

2012 ◽  
Vol 5 (1) ◽  
pp. 223-230 ◽  
Author(s):  
S. Saux Picart ◽  
M. Butenschön ◽  
J. D. Shutler
Keyword(s):  
Satellite Data ◽  
Fishery Management ◽  
Numerical Models ◽  
Spatial Scales ◽  
Marine Ecosystem ◽  
Original Method ◽  
Precipitation Forecast ◽  
Spatial And Temporal Scales ◽  
Geographical Patterns ◽  
Marine Ecosystem Model

Abstract. Complex numerical models of the Earth's environment, based around 3-D or 4-D time and space domains are routinely used for applications including climate predictions, weather forecasts, fishery management and environmental impact assessments. Quantitatively assessing the ability of these models to accurately reproduce geographical patterns at a range of spatial and temporal scales has always been a difficult problem to address. However, this is crucial if we are to rely on these models for decision making. Satellite data are potentially the only observational dataset able to cover the large spatial domains analysed by many types of geophysical models. Consequently optical wavelength satellite data is beginning to be used to evaluate model hindcast fields of terrestrial and marine environments. However, these satellite data invariably contain regions of occluded or missing data due to clouds, further complicating or impacting on any comparisons with the model. This work builds on a published methodology, that evaluates precipitation forecast using radar observations based on predefined absolute thresholds. It allows model skill to be evaluated at a range of spatial scales and rain intensities. Here we extend the original method to allow its generic application to a range of continuous and discontinuous geophysical data fields, and therefore allowing its use with optical satellite data. This is achieved through two major improvements to the original method: (i) all thresholds are determined based on the statistical distribution of the input data, so no a priori knowledge about the model fields being analysed is required and (ii) occluded data can be analysed without impacting on the metric results. The method can be used to assess a model's ability to simulate geographical patterns over a range of spatial scales. We illustrate how the method provides a compact and concise way of visualising the degree of agreement between spatial features in two datasets. The application of the new method, its handling of bias and occlusion and the advantages of the novel method are demonstrated through the analysis of model fields from a marine ecosystem model.

scholarly journals A general Cluster data and global MHD simulation comparison

2008 ◽  
Vol 26 (11) ◽  
pp. 3411-3428 ◽  
Author(s):  
P. Daum ◽  
M. H. Denton ◽  
J. A. Wild ◽  
M. G. G. T. Taylor ◽  
J. Šafránková ◽  
...  
Keyword(s):  
Large Scale ◽  
Numerical Models ◽  
Mhd Simulation ◽  
Global Context ◽  
Cluster Data ◽  
Verification Methods ◽  
Modelling Studies ◽  
Global Mhd Simulation ◽  
The Many

Abstract. Among the many challenges facing the space weather modelling community today, is the need for validation and verification methods of the numerical models available describing the complex nonlinear Sun-Earth system. Magnetohydrodynamic (MHD) models represent the latest numerical models of this environment and have the unique ability to span the enormous distances present in the magnetosphere, from several hundred kilometres to several thousand kilometres above the Earth's surface. This makes it especially difficult to develop verification and validation methods which posses the same range spans as the models. In this paper we present a first general large-scale comparison between four years (2001–2004) worth of in situ Cluster plasma observations and the corresponding simulated predictions from the coupled Block-Adaptive-Tree-Solarwind-Roe-Upwind-Scheme (BATS-R-US) MHD code. The comparison between the in situ measurements and the model predictions reveals that by systematically constraining the MHD model inflow boundary conditions a good correlation between the in situ observations and the modeled data can be found. These results have an implication for modelling studies addressing also smaller scale features of the magnetosphere. The global MHD simulation can therefore be used to place localised satellite and/or ground-based observations into a global context and fill the gaps left by measurements.

scholarly journals Spatial analysis of hourly precipitation combining observations and ensemble forecasts

2021 ◽  
Author(s):  
Cristian Lussana ◽  
Thomas N. Nipen ◽  
Ivar A. Seierstad ◽  
Christoffer A. Elo
Keyword(s):  
Spatial Analysis ◽  
Gamma Distribution ◽  
Numerical Models ◽  
Spatial Scales ◽  
Data Sources ◽  
Optimal Interpolation ◽  
Interpolation Scheme ◽  
Background Error ◽  
Hourly Precipitation ◽  
Gaussian Anamorphosis

<p>Hourly precipitation is often simultaneously simulated by numerical models and observed by multiple data sources. Accurate precipitation fields based on all available information are valuable input for numerous applications and a critical aspect of climate monitoring. </p><p>Inverse problem theory offers an ideal framework for the combination of observations with a numerical model background. In particular, we have considered a modified ensemble optimal interpolation scheme. The deviations between background and observations are used to adjust for deficiencies in the ensemble. A data transformation based on Gaussian anamorphosis has been used to optimally exploit the potential of the spatial analysis, given that precipitation is approximated with a gamma distribution and the spatial analysis requires normally distributed variables. For each point, the spatial analysis returns the shape and rate parameters of its gamma distribution. </p><p>The ensemble-based statistical interpolation scheme with Gaussian anamorphosis for precipitation (EnSI-GAP) is implemented in a way that the covariance matrices are locally stationary, and the background error covariance matrix undergoes a localization process. Concepts and methods that are usually found in data assimilation are here applied to spatial analysis, where they have been adapted in an original way to represent precipitation at finer spatial scales than those resolved by the background, at least where the observational network is dense enough.</p><p>The EnSI-GAP setup requires the specification of a restricted number of parameters, and specifically, the explicit values of the error variances are not needed, since they are inferred from the available data. </p><p>The examples of applications presented over Norway provide a better understanding of EnSI-GAP. The data sources considered are those typically used at national meteorological services, such as local area models, weather radars, and in situ observations. For this last data source, measurements from both traditional and opportunistic sensors have been considered.</p>

scholarly journals Applications of Models and Tools for Mesoscale and Microscale Thermal Analysis in Mid-Latitude Climate Regions—A Review

2021 ◽  
Vol 13 (22) ◽  
pp. 12385
Author(s):  
Gabriele Lobaccaro ◽  
Koen De Ridder ◽  
Juan Angel Acero ◽  
Hans Hooyberghs ◽  
Dirk Lauwaet ◽  
...  
Keyword(s):  
High Temperatures ◽  
Numerical Models ◽  
Spatial Scales ◽  
Built Environments ◽  
Local Climate ◽  
Climate Conditions ◽  
The Social ◽  
Climatic Environment ◽  
Intense Solar Radiation ◽  
Mobile Measurements

Urban analysis at different spatial scales (micro- and mesoscale) of local climate conditions is required to test typical artificial urban boundaries and related climate hazards such as high temperatures in built environments. The multitude of finishing materials and sheltering objects within built environments produce distinct patterns of different climate conditions, particularly during the daytime. The combination of high temperatures and intense solar radiation strongly perturb the environment by increasing the thermal heat stress at the pedestrian level. Therefore, it is becoming common practice to use numerical models and tools that enable multiple design and planning alternatives to be quantitatively and qualitatively tested to inform urban planners and decision-makers. These models and tools can be used to compare the relationships between the micro-climatic environment, the subjective thermal assessment, and the social behaviour, which can reveal the attractiveness and effectiveness of new urban spaces and lead to more sustainable and liveable public spaces. This review article presents the applications of selected environmental numerical models and tools to predict human thermal stress at the mesoscale (e.g., satellite thermal images and UrbClim) and the microscale (e.g., mobile measurements, ENVI-met, and UrbClim HR) focusing on case study cities in mid-latitude climate regions framed in two European research projects.

scholarly journals Assimilation of stratospheric and mesospheric temperatures from MLS and SABER into a global NWP model

2008 ◽  
Vol 8 (3) ◽  
pp. 8455-8490 ◽  
Author(s):  
K. W. Hoppel ◽  
N. L. Baker ◽  
L. Coy ◽  
S. D. Eckermann ◽  
J. P. McCormack ◽  
...  
Keyword(s):  
High Altitude ◽  
Spatial Scales ◽  
Three Dimensional ◽  
Forecast Model ◽  
Temperature Structure ◽  
Deep Circulation ◽  
Zonal Winds ◽  
Mesospheric Temperature ◽  
Standard Deviations ◽  
Nwp Model

Abstract. The forecast model and three-dimensional variational data assimilation components of the Navy Operational Global Atmospheric Prediction System (NOGAPS) have each been extended into the upper stratosphere and mesosphere to form an Advanced Level Physics High Altitude (ALPHA) version of NOGAPS extending to ~100 km. This NOGAPS-ALPHA NWP prototype is used to assimilate stratospheric and mesospheric temperature data from the Microwave Limb Sounder (MLS) and the Sounding of the Atmosphere using Broadband Radiometry (SABER) instruments. A 60-day analysis period in January and February, 2006, was chosen that includes a well documented stratospheric sudden warming. SABER temperatures indicate that the SSW caused the polar winter stratopause at ~40 km to disappear, then reform at ~80 km altitude and slowly descend during February. The NOGAPS-ALPHA analysis reproduces this observed stratospheric and mesospheric temperature structure, as well as realistic evolution of zonal winds, residual velocities, and Eliassen-Palm fluxes that aid interpretation of the vertically deep circulation and eddy flux anomalies that developed in response to this wave-breaking event. The observation minus forecast (O-F) standard deviations for MLS and SABER are ~2 K in the mid-stratosphere and increase monotonically to about 6 K in the upper mesosphere. Increasing O-F standard deviations in the mesosphere are expected due to increasing instrument error and increasing geophysical variance at small spatial scales in the forecast model. In the mid/high latitude winter regions, 10-day forecast skill is improved throughout the upper stratosphere and mesosphere when the model is initialized using the high-altitude analysis based on assimilation of both SABER and MLS data.

Sign in / Sign up

Export Citation Format

Share Document