Quit Worrying About Uncertainty in Sea Level Projections

Emphasizing uncertainty in model projections of long-term sea level rise is a misguided approach. Instead, we should focus on communicating what we do know while improving model confidence.

Development and validation of the Climate Model Confidence Index (CMCI): measuring ability to reproduce historical climate conditions

This study further develops and finally validates the Climate Model Confidence Index (CMCI) as a simple and effective metric for evaluating and ranking the ability of climate models to reproduce historical climate conditions. Modelled daily climate data outputs from two different statistical downscaling techniques (PCIC: Pacific Climate Impacts Consortium; SDSM: Statistical Down-Scaling Model) are compared with observational data recorded by Environment Canada weather stations located in Kelowna, BC (Canada), for the period from 1969 to 2005. Using daily data (N > 13,000), Student’s t-tests determined if there were statistically significant differences between the modelled and observed means while ANOVA F-tests identified differences between variances. Using aggregated annual data (N = 37), CMCI values were also calculated for the individual model runs from each statistical downscaling technique. Climate model outputs were ranked according to the absolute value of the t statistics. The 20 SDSM ensembles outperformed the 27 PCIC models for both minimum and maximum temperatures, while PCIC outperformed SDSM for total precipitation. Linear regression determined the correlation between the absolute value of the t statistics and the corresponding CMCI values (R2 > 0.99, P < 0.001). Rare discrepancies (< 10% of all model rankings) between the t statistic and CMCI rankings occurred at the third decimal place and resulted in a one rank difference between models. These discrepancies are attributed to the precision of the t tests which rely on daily data and consider observed as well as modelled variance, whereas the simplicity and utility of the CMCI are demonstrated by only requiring annual data and observed variance to calculate.

TD 2611 - Previsão de Homicídios no Brasil: proposta de variável antecedente

Neste estudo, investigamos a capacidade de variáveis antecedentes, entre elas internações por agressão, na previsão do número de homicídios no Brasil. O objetivo principal desta pesquisa é suprimir a lacuna referente à defasagem de informações na divulgação sobre homicídios no país, permitindo assim análises conjunturais atualizadas. Para tanto, por intermédio do esquema rolling window e da abordagem model confidence set (MCS), investigamos se modelos de variáveis antecedentes apresentam desempenho preditivo superior ao conjunto de modelos univariados. Ao aplicar a abordagem MCS, considerando diferentes estatísticas de avaliação, funções de perda e janelas de estimação, encontramos fortes evidências da capacidade das variáveis antecedentes utilizadas fornecerem conteúdo informacional adicional na previsão da dinâmica criminal brasileira, com modelos de variáveis antecedentes sistematicamente superando modelos univariados. Na média, os melhores modelos de variáveis antecedentes apresentam melhorias relativas ao benchmark random walk, de 60% em termos de raiz do erro quadrado médio (RMSE), erro absoluto médio (MAE) e desvio absoluto médio da média (MAD).

The DURATIONS randomised trial design: Estimation targets, analysis methods and operating characteristics

Background: Designing trials to reduce treatment duration is important in several therapeutic areas, including tuberculosis and bacterial infections. We recently proposed a new randomised trial design to overcome some of the limitations of standard two-arm non-inferiority trials. This DURATIONS design involves randomising patients to a number of duration arms and modelling the so-called ‘duration-response curve’. This article investigates the operating characteristics (type-1 and type-2 errors) of different statistical methods of drawing inference from the estimated curve. Methods: Our first estimation target is the shortest duration non-inferior to the control (maximum) duration within a specific risk difference margin. We compare different methods of estimating this quantity, including using model confidence bands, the delta method and bootstrap. We then explore the generalisability of results to estimation targets which focus on absolute event rates, risk ratio and gradient of the curve. Results: We show through simulations that, in most scenarios and for most of the estimation targets, using the bootstrap to estimate variability around the target duration leads to good results for DURATIONS design-appropriate quantities analogous to power and type-1 error. Using model confidence bands is not recommended, while the delta method leads to inflated type-1 error in some scenarios, particularly when the optimal duration is very close to one of the randomised durations. Conclusions: Using the bootstrap to estimate the optimal duration in a DURATIONS design has good operating characteristics in a wide range of scenarios and can be used with confidence by researchers wishing to design a DURATIONS trial to reduce treatment duration. Uncertainty around several different targets can be estimated with this bootstrap approach.

Comparing the Performances of GARCH-type Models in Capturing Cryptocurrencies Volatility

The analysis of cryptocurrencies market behaviour is receiving significant attention from researchers and practitioners in the last decades. This paper aims at contributes to volatility estimations of the cryptocurrencies helping to highlight the main stylized facts and characteristics. The performance of different specifications of volatility modelling, within the GARCH class, have been compared through the Model Confidence Set (MCS) over four of the most capitalised cryptocurrencies, namely Bitcoin, Ethereum, Stellar and Ripple. Our empirical findings give evidence of strong asymmetric effects in cryptocurrencies volatility leading to a better performance of asymmetric GARCH specifications..

Confidence-Based Uncertainty Quantification and Model Validation for Simulations of High-Speed Impact Problems

Validation exercises for computational models of materials under impact must contend with sparse experimental data as well as with uncertainties due to microstructural stochasticity and variabilities in thermomechanical properties of the material. This paper develops statistical methods for determining confidence levels for verification and validation of computational models subject to aleatoric and epistemic uncertainties and sparse stochastic experimental datasets. To demonstrate the method, the classical problem of Taylor impact of a copper bar is simulated. Ensembles of simulations are performed to cover the range of variabilities in the material properties of copper, specifically the nominal yield strength A, the hardening constant B, and the hardening exponent n in a Johnson–Cook material model. To quantify uncertainties in the simulation models, we construct probability density functions (PDFs) of the ratios of the quantities of interest, viz., the final bar diameter Df to the original diameter D0 and the final length Lf to the original length L0. The uncertainties in the experimental data are quantified by constructing target output distributions for these QoIs (Df/D0 and Lf/L0) from the sparse experimental results reported in literature. The simulation output and the experimental output distributions are compared to compute two metrics, viz., the median of the model prediction error and the model confidence at user-specified error level. It is shown that the median is lower and the model confidence is higher for Lf/L0 compared to Df/D0, implying that the simulation models predict the final length of the bar more accurately than the diameter. The calculated confidence levels are shown to be consistent with expectations from the physics of the impact problem and the assumptions in the computational model. Thus, this paper develops and demonstrates physically meaningful metrics for validating simulation models using limited stochastic experimental datasets. The tools and techniques developed in this work can be used for validating a wide range of computational models operating under input uncertainties and sparse experimental datasets.