Updating Short-Term Probabilistic Weather Forecasts of Continuous Variables Using Recent Observations

Abstract A statistical postprocessing method for improving probabilistic forecasts of continuous weather variables, given recent observations, is presented. The method updates an existing probabilistic forecast by incorporating observations reported in the intermediary time since model initialization. As such, this method provides updated short-range probabilistic forecasts at an extremely low computational cost. The method models the time sequence of cumulative distribution function (CDF) values corresponding to the observation as a first-order Markov process. Verifying CDF values are highly correlated in time, and their changes in time are modeled probabilistically by a transition function. The effect of the method is that the spread of the probabilistic forecasts for the first few hours after an observation has been made is considerably narrower than the original forecast. The updated probability distributions widen back toward the original forecast for forecast times far in the future as the effect of the recent observation diminishes. The method is tested on probabilistic forecasts produced by an operational ensemble forecasting system. The method improves the ignorance score and the continuous ranked probability score of the probabilistic forecasts significantly for the first few hours after an observation has been made. The mean absolute error of the median of the probability distribution is also shown to be improved.

Download Full-text

Comparative verification in complex topography

10.5194/ems2021-246 ◽

2021 ◽

Author(s):

Jonas Bhend ◽

Jean-Christophe Orain ◽

Vera Schönenberger ◽

Christoph Spirig ◽

Lionel Moret ◽

...

Keyword(s):

Weather Forecasting ◽

Absolute Error ◽

Time Of Day ◽

Probabilistic Forecasts ◽

Wide Range ◽

Direct Model ◽

Forecasting System ◽

The Mean ◽

Update Frequency ◽

Core Activity

Verification is a core activity in weather forecasting. Insights from verification are used for monitoring, for reporting, to support and motivate development of the forecasting system, and to allow users to maximize forecast value. Due to the broad range of applications for which verification provides valuable input, the range of questions one would like to answer can be very large. Static analyses and summary verification results are often insufficient to cover this broad range. To this end, we developed an interactive verification platform at MeteoSwiss that allows users to inspect verification results from a wide range of angles to find answers to their specific questions.We present the technical setup to achieve a flexible yet performant interactive platform and two prototype applications: monitoring of direct model output from operational NWP systems and understanding of the capabilities and limitations of our pre-operational postprocessing. We present two innovations that illustrate the user-oriented approach to comparative verification adopted as part of the platform. To facilitate the comparison of a broad range of forecasts issued with varying update frequency, we rely on the concept of time of verification to collocate the most recent available forecasts at the time of day at which the forecasts are used. In addition, we offer a matrix selection to more flexibly select forecast sources and scores for comparison. Doing so, we can for example compare the mean absolute error (MAE) for deterministic forecasts to the MAE and continuous ranked probability scores of probabilistic forecasts to illustrate the benefit of using probabilistic forecasts.

Download Full-text

Generation of Scenarios from Calibrated Ensemble Forecasts with a Dual-Ensemble Copula-Coupling Approach

Monthly Weather Review ◽

10.1175/mwr-d-15-0403.1 ◽

2016 ◽

Vol 144 (12) ◽

pp. 4737-4750 ◽

Cited By ~ 15

Author(s):

Zied Ben Bouallègue ◽

Tobias Heppelmann ◽

Susanne E. Theis ◽

Pierre Pinson

Keyword(s):

Probability Distributions ◽

Forecast Error ◽

Computational Cost ◽

Joint Probability ◽

Ensemble Prediction ◽

Small Scale ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Probabilistic Forecasts ◽

Joint Probability Distributions

Abstract Probabilistic forecasts in the form of ensembles of scenarios are required for complex decision-making processes. Ensemble forecasting systems provide such products but the spatiotemporal structures of the forecast uncertainty is lost when statistical calibration of the ensemble forecasts is applied for each lead time and location independently. Nonparametric approaches allow the reconstruction of spatiotemporal joint probability distributions at a small computational cost. For example, the ensemble copula coupling (ECC) method rebuilds the multivariate aspect of the forecast from the original ensemble forecasts. Based on the assumption of error stationarity, parametric methods aim to fully describe the forecast dependence structures. In this study, the concept of ECC is combined with past data statistics in order to account for the autocorrelation of the forecast error. The new approach, called d-ECC, is applied to wind forecasts from the high-resolution Consortium for Small-Scale Modeling (COSMO) ensemble prediction system (EPS) run operationally at the German Weather Service (COSMO-DE-EPS). Scenarios generated by ECC and d-ECC are compared and assessed in the form of time series by means of multivariate verification tools and within a product-oriented framework. Verification results over a 3-month period show that the innovative method d-ECC performs as well as or even outperforms ECC in all investigated aspects.

Download Full-text

Nanosecond Photodynamics Simulations of a Cis-Trans Isomerization Are Enabled by Machine Learning

10.26434/chemrxiv.13047863 ◽

2020 ◽

Author(s):

Jingbai Li ◽

Patrick Reiser ◽

André Eberhard ◽

Pascal Friederich ◽

Steven Lopez

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Excited State ◽

Adaptive Sampling ◽

Computational Cost ◽

Ground Truth ◽

Absolute Error ◽

Photochemical Reactions ◽

Computational Techniques ◽

Full Potential

Photochemical reactions are being increasingly used to construct complex molecular architectures with mild and straightforward reaction conditions. Computational techniques are increasingly important to understand the reactivities and chemoselectivities of photochemical isomerization reactions because they offer molecular bonding information along the excited-state(s) of photodynamics. These photodynamics simulations are resource-intensive and are typically limited to 1–10 picoseconds and 1,000 trajectories due to high computational cost. Most organic photochemical reactions have excited-state lifetimes exceeding 1 picosecond, which places them outside possible computational studies. Westermeyr et al. demonstrated that a machine learning approach could significantly lengthen photodynamics simulation times for a model system, methylenimmonium cation (CH2NH2+).We have developed a Python-based code, Python Rapid Artificial Intelligence Ab Initio Molecular Dynamics (PyRAI2MD), to accomplish the unprecedented 10 ns cis-trans photodynamics of trans-hexafluoro-2-butene (CF3–CH=CH–CF3) in 3.5 days. The same simulation would take approximately 58 years with ground-truth multiconfigurational dynamics. We proposed an innovative scheme combining Wigner sampling, geometrical interpolations, and short-time quantum chemical trajectories to effectively sample the initial data, facilitating the adaptive sampling to generate an informative and data-efficient training set with 6,232 data points. Our neural networks achieved chemical accuracy (mean absolute error of 0.032 eV). Our 4,814 trajectories reproduced the S1 half-life (60.5 fs), the photochemical product ratio (trans: cis = 2.3: 1), and autonomously discovered a pathway towards a carbene. The neural networks have also shown the capability of generalizing the full potential energy surface with chemically incomplete data (trans → cis but not cis → trans pathways) that may offer future automated photochemical reaction discoveries.

Download Full-text

DISSONANCE — A MEASURE OF VARIABILITY FOR ORDINAL RANDOM VARIABLES

International Journal of Uncertainty Fuzziness and Knowledge-Based Systems ◽

10.1142/s0218488501000594 ◽

2001 ◽

Vol 09 (01) ◽

pp. 39-53 ◽

Cited By ~ 2

Author(s):

RONALD R. YAGER

Keyword(s):

Distribution Function ◽

Cumulative Distribution Function ◽

Probability Distributions ◽

Random Variables ◽

Cumulative Distribution ◽

General Properties

We look at the issue of obtaining a variance like measure associated with probability distributions over ordinal sets. We call these dissonance measures. We specify some general properties desired in these dissonance measures. The centrality of the cumulative distribution function in formulating the concept of dissonance is pointed out. We introduce some specific examples of measures of dissonance.

Download Full-text

Scalable information-theoretic path planning for a rover-helicopter team in uncertain environments

International Journal of Advanced Robotic Systems ◽

10.1177/1729881421999587 ◽

2021 ◽

Vol 18 (2) ◽

pp. 172988142199958

Author(s):

Larkin Folsom ◽

Masahiro Ono ◽

Kyohei Otsu ◽

Hyoshin Park

Keyword(s):

Path Planning ◽

Lower Risk ◽

Information Gain ◽

Probability Distributions ◽

Single Agent ◽

Uncertain Environments ◽

Information Theoretic ◽

Highly Correlated ◽

Naive Approach

Mission-critical exploration of uncertain environments requires reliable and robust mechanisms for achieving information gain. Typical measures of information gain such as Shannon entropy and KL divergence are unable to distinguish between different bimodal probability distributions or introduce bias toward one mode of a bimodal probability distribution. The use of a standard deviation (SD) metric reduces bias while retaining the ability to distinguish between higher and lower risk distributions. Areas of high SD can be safely explored through observation with an autonomous Mars Helicopter allowing safer and faster path plans for ground-based rovers. First, this study presents a single-agent information-theoretic utility-based path planning method for a highly correlated uncertain environment. Then, an information-theoretic two-stage multiagent rapidly exploring random tree framework is presented, which guides Mars helicopter through regions of high SD to reduce uncertainty for the rover. In a Monte Carlo simulation, we compare our information-theoretic framework with a rover-only approach and a naive approach, in which the helicopter scouts ahead of the rover along its planned path. Finally, the model is demonstrated in a case study on the Jezero region of Mars. Results show that the information-theoretic helicopter improves the travel time for the rover on average when compared with the rover alone or with the helicopter scouting ahead along the rover’s initially planned route.

Download Full-text

Two-dimensional Bayesian inversion of magnetotelluric data using trans-dimensional Gaussian processes

Geophysical Journal International ◽

10.1093/gji/ggab110 ◽

2021 ◽

Author(s):

Daniel Blatter ◽

Anandaroop Ray ◽

Kerry Key

Keyword(s):

Gaussian Process ◽

Process Model ◽

Field Data ◽

Probability Distributions ◽

Bulk Composition ◽

Computational Cost ◽

Model Space ◽

Bayesian Inversion ◽

Adaptive Finite Element ◽

Magnetotelluric Data

Summary Bayesian inversion of electromagnetic data produces crucial uncertainty information on inferred subsurface resistivity. Due to their high computational cost, however, Bayesian inverse methods have largely been restricted to computationally expedient 1D resistivity models. In this study, we successfully demonstrate, for the first time, a fully 2D, trans-dimensional Bayesian inversion of magnetotelluric data. We render this problem tractable from a computational standpoint by using a stochastic interpolation algorithm known as a Gaussian process to achieve a parsimonious parametrization of the model vis-a-vis the dense parameter grids used in numerical forward modeling codes. The Gaussian process links a trans-dimensional, parallel tempered Markov chain Monte Carlo sampler, which explores the parsimonious model space, to MARE2DEM, an adaptive finite element forward solver. MARE2DEM computes the model response using a dense parameter mesh with resistivity assigned via the Gaussian process model. We demonstrate the new trans-dimensional Gaussian process sampler by inverting both synthetic and field magnetotelluric data for 2D models of electrical resistivity, with the field data example converging within 10 days on 148 cores, a non-negligible but tractable computational cost. For a field data inversion, our algorithm achieves a parameter reduction of over 32x compared to the fixed parameter grid used for the MARE2DEM regularized inversion. Resistivity probability distributions computed from the ensemble of models produced by the inversion yield credible intervals and interquartile plots that quantitatively show the non-linear 2D uncertainty in model structure. This uncertainty could then be propagated to other physical properties that impact resistivity including bulk composition, porosity and pore-fluid content.

Download Full-text

Probabilistic Precipitation Forecast Postprocessing Using Quantile Mapping and Rank-Weighted Best-Member Dressing

Monthly Weather Review ◽

10.1175/mwr-d-18-0147.1 ◽

2018 ◽

Vol 146 (12) ◽

pp. 4079-4098 ◽

Cited By ~ 13

Author(s):

Thomas M. Hamill ◽

Michael Scheuerer

Keyword(s):

Probability Distributions ◽

The United States ◽

Ensemble Forecast ◽

Distribution Functions ◽

Cumulative Distribution ◽

Precipitation Forecast ◽

Prediction System ◽

Quantile Mapping ◽

State Dependent ◽

Grid Points

Abstract Hamill et al. described a multimodel ensemble precipitation postprocessing algorithm that is used operationally by the U.S. National Weather Service (NWS). This article describes further changes that produce improved, reliable, and skillful probabilistic quantitative precipitation forecasts (PQPFs) for single or multimodel prediction systems. For multimodel systems, final probabilities are produced through the linear combination of PQPFs from the constituent models. The new methodology is applied to each prediction system. Prior to adjustment of the forecasts, parametric cumulative distribution functions (CDFs) of model and analyzed climatologies are generated using the previous 60 days’ forecasts and analyses and supplemental locations. The CDFs, which can be stored with minimal disk space, are then used for quantile mapping to correct state-dependent bias for each member. In this stage, the ensemble is also enlarged using a stencil of forecast values from the 5 × 5 surrounding grid points. Different weights and dressing distributions are assigned to the sorted, quantile-mapped members, with generally larger weights for outlying members and broader dressing distributions for members with heavier precipitation. Probability distributions are generated from the weighted sum of the dressing distributions. The NWS Global Ensemble Forecast System (GEFS), the Canadian Meteorological Centre (CMC) global ensemble, and the European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble forecast data are postprocessed for April–June 2016. Single prediction system postprocessed forecasts are generally reliable and skillful. Multimodel PQPFs are roughly as skillful as the ECMWF system alone. Postprocessed guidance was generally more skillful than guidance using the Gamma distribution approach of Scheuerer and Hamill, with coefficients generated from data pooled across the United States.

Download Full-text

A Probability Density Function Generator Based on Deep Learning

10.20944/preprints201811.0256.v1 ◽

2018 ◽

Author(s):

Chi-Hua Chen ◽

Fangying Song ◽

Feng-Jang Hwang ◽

Ling Wu

Keyword(s):

Deep Learning ◽

Probability Density Function ◽

Probability Density ◽

Density Function ◽

Probability Distributions ◽

Real Data ◽

Learning Model ◽

Cumulative Distribution ◽

Function Generator ◽

Deep Learning Model

To generate a probability density function (PDF) for fitting probability distributions of real data, this study proposes a deep learning method which consists of two stages: (1) a training stage for estimating the cumulative distribution function (CDF) and (2) a performing stage for predicting the corresponding PDF. The CDFs of common probability distributions can be adopted as activation functions in the hidden layers of the proposed deep learning model for learning actual cumulative probabilities, and the differential equation of trained deep learning model can be used to estimate the PDF. To evaluate the proposed method, numerical experiments with single and mixed distributions are performed. The experimental results show that the values of both CDF and PDF can be precisely estimated by the proposed method.

Download Full-text

Feature Extraction in Six Blocks to Detect and Recognize English Numbers

Iraqi Journal of Science ◽

10.24996/ijs.2021.62.10.37 ◽

2021 ◽

pp. 3790-3803

Author(s):

Heba Kh. Abbas ◽

Haidar J. Mohamad

Keyword(s):

Fuzzy Logic ◽

Feature Extraction ◽

Computational Cost ◽

Absolute Error ◽

Suggested Method ◽

Support Vector ◽

Horizontal Line ◽

Svm Algorithm ◽

Fuzzy Logic Method ◽

Logic Method

The Fuzzy Logic method was implemented to detect and recognize English numbers in this paper. The extracted features within this method make the detection easy and accurate. These features depend on the crossing point of two vertical lines with one horizontal line to be used from the Fuzzy logic method, as shown by the Matlab code in this study. The font types are Times New Roman, Arial, Calabria, Arabic, and Andalus with different font sizes of 10, 16, 22, 28, 36, 42, 50 and 72. These numbers are isolated automatically with the designed algorithm, for which the code is also presented. The number’s image is tested with the Fuzzy algorithm depending on six-block properties only. Groups of regions (High, Medium, and Low) for each number showed unique behavior to recognize any number. Normalized Absolute Error (NAE) equation was used to evaluate the error percentage for the suggested algorithm. The lowest error was 0.001% compared with the real number. The data were checked by the support vector machine (SVM) algorithm to confirm the quality and the efficiency of the suggested method, where the matching was found to be 100% between the data of the suggested method and SVM. The six properties offer a new method to build a rule-based feature extraction technique in different applications and detect any text recognition with a low computational cost.

Download Full-text

The Distribution of Skull Score and Skull Density Ratio in Tremor Patients for MR-Guided Focused Ultrasound Thalamotomy

Frontiers in Neuroscience ◽

10.3389/fnins.2021.612940 ◽

2021 ◽

Vol 15 ◽

Author(s):

Kevin Wen-Kai Tsai ◽

Jui-Cheng Chen ◽

Hui-Chin Lai ◽

Wei-Chieh Chang ◽

Takaomi Taira ◽

...

Keyword(s):

Trabecular Bone ◽

Cortical Bone ◽

Focused Ultrasound ◽

Density Ratio ◽

Ct Images ◽

Image Feature ◽

Cumulative Distribution ◽

Invasive Surgical Approach ◽

Highly Correlated ◽

The Brain

ObjectiveMagnetic resonance-guided focused ultrasound (MRgFUS) is a minimum-invasive surgical approach to non-incisionally cause the thermos-coagulation inside the human brain. The skull score (SS) has already been approved as one of the most dominant factors related to a successful MRgFUS treatment. In this study, we first reveal the SS distribution of the tremor patients, and correlate the SS with the image feature from customized skull density ratio (cSDR). This correlation might give a direction to future clinical studies for improving the SS.MethodsTwo hundred and forty-six patients received a computed tomography (CT) scan of the brain, and a bone-enhanced filter was applied and reconstructed to a high spatial resolution CT images. The SS of all patients would be estimated by the MRgFUS system after importing the reconstructed CT images into the MRgFUS system. The histogram and the cumulative distribution of the SS from all the patients were calculated to show the percentage of the patients whose SS lower than 0.3 and 0.4. The same CT images of all patients were utilized to calculated the cSDR by first segmented the trabecular bone and the cortical bone from the CT images and divided the average trabecular bone intensity (aTBI) by the average cortical bone intensity (aCBI). The Pearson’s correlations between the SS and the cSDR, aTBI, and the aCBI were calculated, respectively.ResultsThere were 19.19 and 50% of the patient who had the SS lower than the empirical threshold 0.3 and 0.4, respectively. The Pearson’s correlation between the SS and the cSDR, aCBI, and the aTBI were R = 0.8145, 0.5723, and 0.8842.ConclusionHalf of the patients were eligible for the MRgFUS thalamotomy based on the SS, and nearly 20% of patients were empirically difficult to achieve a therapeutic temperature during MRgFUS. The SS and our cSDR are highly correlated, and the SS had a higher correlation with aTBI than with aCBI. This is the first report to explicitly reveal the SS population and indicate a potential way to increase the chance to achieve a therapeutic temperature for those who originally have low SS.

Download Full-text