On Two Measure-Theoretic Aspects of the Full Bayesian Significance Test for Precise Bayesian Hypothesis Testing †

The Full Bayesian Significance Test (FBST) has been proposed as a convenient method to replace frequentist p-values for testing a precise hypothesis. Although the FBST enjoys various appealing properties, the purpose of this paper is to investigate two aspects of the FBST which are sometimes observed as measure-theoretic inconsistencies of the procedure and have not been discussed rigorously in the literature. First, the FBST uses the posterior density as a reference for judging the Bayesian statistical evidence against a precise hypothesis. However, under absolutely continuous prior distributions, the posterior density is defined only up to Lebesgue null sets which renders the reference criterion arbitrary. Second, the FBST statistical evidence seems to have no valid prior probability. It is shown that the former aspect can be circumvented by fixing a version of the posterior density before using the FBST, and the latter aspect is based on its measure-theoretic premises. An illustrative example demonstrates the two aspects and their solution. Together, the results in this paper show that both of the two aspects which are sometimes observed as measure-theoretic inconsistencies of the FBST are not tenable. The FBST thus provides a measure-theoretically coherent Bayesian alternative for testing a precise hypothesis.

The Importance of Random Slopes in Mixed Models for Bayesian Hypothesis Testing

Mixed models are gaining popularity in psychology. For frequentist mixed models, Barr, Levy, Scheepers, and Tily (2013) showed that excluding random slopes – differences between individuals in the direction and size of an effect – from a model when they are in the data can lead to a substantial increase in false-positive conclusions in null-hypothesis tests. Here I demonstrate through five simulations that the same is true for Bayesian hypothesis testing with mixed models, often yielding Bayes factors reflecting very strong evidence for a mean effect on the population level even if there was no such effect. Including random slopes in the model largely eliminates the risk of strong false positives, but reduces the chance of obtaining strong evidence for true effects. I recommend starting analysis with testing the support for random slopes in the data, and removing them from the models only if there is clear evidence against them.