A Deep Spatio-Temporal Forecasting Model for Multi-Site Weather Prediction Post-Processing

Abstract The nonlocal excess phase observation operator for assimilating the global positioning system (GPS) radio occultation (RO) sounding data has been proven by some research papers to produce significantly better analyses for numerical weather prediction (NWP) compared to the local refractivity observation operator. However, the high computational cost and the difficulties in parallelization associated with the nonlocal GPS RO operator deter its application in research and operational NWP practices. In this article, two strategies are designed and implemented in the data assimilation system for the Weather Research and Forecasting Model to demonstrate the capability of parallel assimilation of GPS RO profiles with the nonlocal excess phase observation operator. In particular, to solve the parallel load imbalance problem due to the uneven geographic distribution of the GPS RO observations, round-robin scheduling is adopted to distribute GPS RO observations among the processing cores to balance the workload. The wall clock time required to complete a five-iteration minimization on a demonstration Antarctic case with 106 GPS RO observations is reduced from more than 3.5 h with a single processing core to 2.5 min with 106 processing cores. These strategies present the possibility of application of the nonlocal GPS RO excess phase observation operator in operational data assimilation systems with a cutoff time limit.

Download Full-text

ARPEGE cloud cover forecast post-processing with convolutional neural network

10.5194/egusphere-egu2020-18325 ◽

2020 ◽

Author(s):

Florian Dupuy ◽

Olivier Mestre ◽

Léo Pfitzner

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Random Forests ◽

Cloud Cover ◽

Spatial Information ◽

Weather Prediction ◽

Excellent Result ◽

Machine Learning Algorithms ◽

Post Processing

Cloud cover is a crucial information for many applications such as planning land observation missions from space. However, cloud cover remains a challenging variable to forecast, and Numerical Weather Prediction (NWP) models suffer from significant biases, hence justifying the use of statistical post-processing techniques. In our application, the ground truth is a gridded cloud cover product derived from satellite observations over Europe, and predictors are spatial fields of various variables produced by ARPEGE (M&#233;t&#233;o-France global NWP) at the corresponding lead time.In this study, ARPEGE cloud cover is post-processed using a convolutional neural network (CNN). CNN is the most popular machine learning tool to deal with images. In our case, CNN allows to integrate spatial information contained in NWP outputs. We show that a simple U-Net architecture produces significant improvements over Europe. Compared to the raw ARPEGE forecasts, MAE drops from 25.1 % to 17.8 % and RMSE decreases from 37.0 % to 31.6 %. Considering specific needs for earth observation, special interest was put on forecasts with low cloud cover conditions (< 10 %). For this particular nebulosity class, we show that hit rate jumps from 40.6 to 70.7 (which is the order of magnitude of what can be achieved using classical machine learning algorithms such as random forests) while false alarm decreases from 38.2 to 29.9. This is an excellent result, since improving hit rates by means of random forests usually also results in a slight increase of false alarms.

Download Full-text

Unsupervised video object segmentation: An affinity and edge learning approach.

10.36227/techrxiv.16457538 ◽

2021 ◽

Author(s):

Sundaram Muthu ◽

Ruwan Tennakoon ◽

Reza Hoseinnezhad ◽

Alireza Bab-Hadiashar

Keyword(s):

Optic Flow ◽

Object Segmentation ◽

Video Object Segmentation ◽

Video Object ◽

Post Processing ◽

Multiple Objects ◽

Temporal Features ◽

Image Edge ◽

Spatio Temporal ◽

Generalization Errors

<div>This paper presents a new approach to solve unsupervised video object segmentation~(UVOS) problem (called TMNet). The UVOS is still a challenging problem as prior methods suffer from issues like generalization errors to segment multiple objects in unseen test videos (category agnostic), over reliance on inaccurate optic flow, and problem towards capturing fine details at object boundaries. These issues make the UVOS, particularly in presence of multiple objects, an ill-defined problem. Our focus is to constrain the problem and improve the segmentation results by inclusion of multiple available cues such as appearance, motion, image edge, flow edge and tracking information through neural attention. To solve the challenging category agnostic multiple object UVOS, our model is designed to predict neighbourhood affinities for being part of the same object and cluster those to obtain accurate segmentation. To achieve multi cue based neural attention, we designed a Temporal Motion Attention module, as part of our segmentation framework, to learn the spatio-temporal features. To refine and improve the accuracy of object segmentation boundaries, an edge refinement module (using image and optic flow edges) and a geometry based loss function are incorporated. The overall framework is capable of segmenting and finding accurate objects' boundaries without any heuristic post processing. This enables the method to be used for unseen videos. Experimental results on challenging DAVIS16 and multi object DAVIS17 datasets shows that our proposed TMNet performs favourably compared to the state-of-the-art methods without post processing.</div>

Download Full-text

Generating wind power scenarios for probabilistic ramp event prediction using multivariate statistical post-processing

10.5194/wes-2018-13 ◽

2018 ◽

Author(s):

Rochelle P. Worsnop ◽

Michael Scheuerer ◽

Thomas M. Hamill ◽

Julie K. Lundquist

Keyword(s):

Wind Speed ◽

Wind Power ◽

Wind Farm ◽

Weather Prediction ◽

Model Performance ◽

The United States ◽

Numerical Weather Prediction Model ◽

Post Processing ◽

Skill Scores ◽

Ramp Events

Abstract. Wind power forecasting is gaining international significance as more regions promote policies to increase the use of renewable energy. Wind ramps, large variations in wind power production during a period of minutes to hours, challenge utilities and electrical balancing authorities. A sudden decrease in wind energy production must be balanced by other power generators to meet energy demands, while a sharp increase in unexpected production results in excess power that may not be used in the power grid, leading to a loss of potential profits. In this study, we compare different methods to generate probabilistic ramp forecasts from the High Resolution Rapid Refresh (HRRR) numerical weather prediction model with up to twelve hours of lead time at two tall-tower locations in the United States. We validate model performance using 21 months of 80-m wind speed observations from towers in Boulder, Colorado and near the Columbia River Gorge in eastern Oregon. We employ four statistical post-processing methods, three of which are not currently used in the literature for wind forecasting. These procedures correct biases in the model and generate short-term wind speed scenarios which are then converted to power scenarios. This probabilistic enhancement of HRRR point forecasts provides valuable uncertainty information of ramp events and improves the skill of predicting ramp events over the raw forecasts. We compute Brier skill scores for each method at predicting up- and down-ramps to determine which method provides the best prediction. We find that the Standard Schaake Shuffle method yields the highest skill at predicting ramp events for these data sets, especially for up-ramp events at the Oregon site. Increased skill for ramp prediction is limited at the Boulder, CO site using any of the multivariate methods, because of the poor initial forecasts in this area of complex terrain. These statistical methods can be implemented by wind farm operators to generate a range of possible wind speed and power scenarios to aid and optimize decisions before ramp events occur.

Download Full-text

The impact of London on a low-level jet

10.5194/egusphere-egu21-11307 ◽

2021 ◽

Author(s):

Aristofanis Tsiringakis ◽

Natalie Theeuwes ◽

Janet Barlow ◽

Gert-Jan Steeneveld

Keyword(s):

Urban Area ◽

Prediction Models ◽

Weather Prediction ◽

Vertical Mixing ◽

Forecasting Model ◽

Low Level ◽

Low Level Jet ◽

The Difference ◽

The Impact ◽

Numerical Weather Prediction Models

The low-level jet (LLJ) is an important phenomenon that can affect (and is affected by) the turbulence in the nocturnal urban boundary layer (UBL). We investigate the interaction of a regional LLJ with the UBL during a 2-day period over London. Observations from two Doppler Lidars and two numerical weather prediction models (Weather Research & Forecasting model and UKV Met Office Unified Model) are used to compared the LLJ characteristics (height, speed and fall-off) between a urban (London) and a rural (Chilbolton) site. We find that LLJs are elevated (70m) over London, due to the deeper UBL, an effect of the increased vertical mixing over the urban area and the difference in the topography between the two sites. Wind speed and fall-off are slightly reduced with respect to the rural LLJ. The effects of the urban area and the surrounding topography on the LLJ characteristics over London are isolated through idealized sensitivity experiments. We find that topography strongly affects the LLJ characteristics (height, falloff, and speed), but there is still a substantial urban influence.

Download Full-text

A Spatio-Temporal Traffic Forecasting Model for Base Station in Cellular Network

2019 IEEE 19th International Conference on Communication Technology (ICCT) ◽

10.1109/icct46805.2019.8947294 ◽

2019 ◽

Author(s):

Hua Qu ◽

Yanpeng Zhang ◽

Jihong Zhao

Keyword(s):

Cellular Network ◽

Base Station ◽

Forecasting Model ◽

Traffic Forecasting ◽

Spatio Temporal

Download Full-text

Statistical post-processing of ensemble forecasts of the height of new snow

Nonlinear Processes in Geophysics ◽

10.5194/npg-26-339-2019 ◽

2019 ◽

Vol 26 (3) ◽

pp. 339-357 ◽

Cited By ~ 2

Author(s):

Jari-Pekka Nousu ◽

Matthieu Lafaysse ◽

Matthieu Vernay ◽

Joseph Bellier ◽

Guillaume Evin ◽

...

Keyword(s):

Real Time ◽

Spatial Scales ◽

Weather Prediction ◽

Evaluation Criteria ◽

Training Dataset ◽

Post Processing ◽

Probabilistic Forecasting ◽

Ensemble Forecasts ◽

Heterogeneous Model ◽

The Impact

Abstract. Forecasting the height of new snow (HN) is crucial for avalanche hazard forecasting, road viability, ski resort management and tourism attractiveness. Météo-France operates the PEARP-S2M probabilistic forecasting system, including 35 members of the PEARP Numerical Weather Prediction system, where the SAFRAN downscaling tool refines the elevation resolution and the Crocus snowpack model represents the main physical processes in the snowpack. It provides better HN forecasts than direct NWP diagnostics but exhibits significant biases and underdispersion. We applied a statistical post-processing to these ensemble forecasts, based on non-homogeneous regression with a censored shifted Gamma distribution. Observations come from manual measurements of 24 h HN in the French Alps and Pyrenees. The calibration is tested at the station scale and the massif scale (i.e. aggregating different stations over areas of 1000 km2). Compared to the raw forecasts, similar improvements are obtained for both spatial scales. Therefore, the post-processing can be applied at any point of the massifs. Two training datasets are tested: (1) a 22-year homogeneous reforecast for which the NWP model resolution and physical options are identical to the operational system but without the same initial perturbations; (2) 3-year real-time forecasts with a heterogeneous model configuration but the same perturbation methods. The impact of the training dataset depends on lead time and on the evaluation criteria. The long-term reforecast improves the reliability of severe snowfall but leads to overdispersion due to the discrepancy in real-time perturbations. Thus, the development of reliable automatic forecasting products of HN needs long reforecasts as homogeneous as possible with the operational systems.

Download Full-text

Microphysics of summer clouds in central West Antarctica simulated by the Polar Weather Research and Forecasting Model (WRF) and the Antarctic Mesoscale Prediction System (AMPS)

Atmospheric Chemistry and Physics ◽

10.5194/acp-19-12431-2019 ◽

2019 ◽

Vol 19 (19) ◽

pp. 12431-12454 ◽

Cited By ~ 4

Author(s):

Keith M. Hines ◽

David H. Bromwich ◽

Sheng-Hung Wang ◽

Israel Silber ◽

Johannes Verlinde ◽

...

Keyword(s):

Liquid Water ◽

Radiative Forcing ◽

Weather Prediction ◽

The Arctic ◽

Weather Research And Forecasting ◽

Forecasting Model ◽

Prediction System ◽

West Antarctic ◽

The Antarctic ◽

Weather Research

Abstract. The Atmospheric Radiation Measurement (ARM) West Antarctic Radiation Experiment (AWARE) provided a highly detailed set of remote-sensing and surface observations to study Antarctic clouds and surface energy balance, which have received much less attention than for the Arctic due to greater logistical challenges. Limited prior Antarctic cloud observations have slowed the progress of numerical weather prediction in this region. The AWARE observations from the West Antarctic Ice Sheet (WAIS) Divide during December 2015 and January 2016 are used to evaluate the operational forecasts of the Antarctic Mesoscale Prediction System (AMPS) and new simulations with the Polar Weather Research and Forecasting Model (WRF) 3.9.1. The Polar WRF 3.9.1 simulations are conducted with the WRF single-moment 5-class microphysics (WSM5C) used by the AMPS and with newer generation microphysics schemes. The AMPS simulates few liquid clouds during summer at the WAIS Divide, which is inconsistent with observations of frequent low-level liquid clouds. Polar WRF 3.9.1 simulations show that this result is a consequence of WSM5C. More advanced microphysics schemes simulate more cloud liquid water and produce stronger cloud radiative forcing, resulting in downward longwave and shortwave radiation at the surface more in agreement with observations. Similarly, increased cloud fraction is simulated with the more advanced microphysics schemes. All of the simulations, however, produce smaller net cloud fractions than observed. Ice water paths vary less between the simulations than liquid water paths. The colder and drier atmosphere driven by the Global Forecast System (GFS) initial and boundary conditions for AMPS forecasts produces lesser cloud amounts than the Polar WRF 3.9.1 simulations driven by ERA-Interim.

Download Full-text

Assessing spatio-temporal variability and biases of climate models

10.5194/dkt-12-47 ◽

2020 ◽

Author(s):

Nedjeljka Žagar

Keyword(s):

Temporal Variability ◽

Prediction Models ◽

Climate Model ◽

Weather Prediction ◽

Ocean Model ◽

Spatial And Temporal Variability ◽

Dynamical Regime ◽

Temporal Variance ◽

Slab Ocean ◽

Spatio Temporal

<div>Atmospheric spatial and temporal variability are closely related with the former being relatively well observed compared to the latter. The former is also regularly assessed in the validation of numerical weather prediction models while the latter is more difficult to estimate. Likewise, thermodynamical fields and circulation are closely coupled calling for an approach that considers them simultaneously. &#160;</div> <div>In this contribution, spatio-temporal variability spectra of the four major reanalysis datasets are discussed and applied for the validation of a climate model prototype. &#160;A relationship between deficiencies in simulated variability and model biases is derived. The underlying method includes dynamical regime decomposition thereby providing a better understanding of the role of tropical variability in global circulation.&#160;</div> <div>Results of numerical simulations are validated by a 20th century reanalysis. A climate model was forced either with the prescribed SST or with a slab ocean model that updates SST in each forecast step.&#160; Scale-dependent validation shows that missing temporal variance in the model relative to verifying reanalysis increases as the spatial scale reduces that appears associated with an increasing lack of spatial variance at smaller scales. Similar to variability, bias is strongly scale dependent; the larger the scale, the greater the bias. Biases present in the SST-forced simulation increase in the simulation using the slab ocean. The comparison of biases computed as a systematic difference between the model and reanalysis and between the SST-forced model and slab-ocean model (a perfect-model scenario) suggests that improving the atmospheric model increases the variance in the model on synoptic and subsynoptic scales but large biases associated with a poor SST remain at planetary scales.</div> &#160;

Download Full-text

A Dynamic and Flexible State Model for Rainfall Nowcasting

10.5194/egusphere-egu2020-21400 ◽

2020 ◽

Author(s):

Marc Schleiss ◽

Venkat Roy

Keyword(s):

Temporal Evolution ◽

Weather Prediction ◽

Radar Reflectivity ◽

State Model ◽

Dynamic State ◽

Model Parameters ◽

Target Area ◽

Stochastic Perturbations ◽

Spatio Temporal ◽

Linear State

We present a dynamic state model estimation method for rainfall nowcasting in which we assume that the short term spatio-temporal evolution of rainfall can be approximated by a linear state model with stochastic perturbations. &#160;We estimate the model parameters using radar reflectivity measurements for one-step as well as multiple-step ahead rainfall nowcasting. If the rainfall intensity at location x and time index t is given by ut(x), then the overall rainfall field intensity vector at any time t over N pixels (of the target area) can be represented by ut = [ut(xN),...ut(xN)]T. Following the aforementioned formalism, the spatio-temporal evolution of the rainfall field can be described by the following linear state model given by ut = Htut-1 + qt where Ht is an unknown time-varying state-transition matrix of dimensions NxN and qt is a stochastic process noise vector of length N. We present an iterative least squares based method to estimate Ht and explore simpler algebraic structures (e.g., scaled affine transformations) to reduce the numbers of unknown parameters during estimation. We evaluate the performances of the proposed model using simulations and radar reflectivity data from the Royal Netherlands Meteorological Institute (KNMI). We observe that the nowcasting performances strongly depend on the size of the target area (number of pixels N), the type of events as well as the parameterization of Ht. The key advantage of the proposed approach over classical nowcasting methods based on Lagrangian persistence is the possibility to incorporate prior information about future rainfall evolution from external sources of information such as satellites or numerical weather prediction models during the estimation of the parameters.

Download Full-text