scholarly journals A tutorial on testing hypotheses using the Bayes factor

2019 ◽  
Author(s):  
Herbert Hoijtink ◽  
Joris Mulder ◽  
Caspar J. Van Lissa ◽  
Xin Gu
Keyword(s):  
Statistical Models ◽  
Null Hypothesis ◽  
Multiple Testing ◽  
Bayes Factor ◽  
Additional Data ◽  
R Package ◽  
Bayes Factors ◽  
Multiple Hypotheses ◽  
Anova Model ◽  
Hypothesis Evaluation

Learning about hypothesis evaluation using the Bayes factor could enhance psychologicalresearch. In contrast to null-hypothesis significance testing: it renders the evidence in favorof each of the hypotheses under consideration (it can be used to quantify support for thenull-hypothesis) instead of a dichotomous reject/do-not-reject decision; it canstraightforwardly be used for the evaluation of multiple hypotheses without having tobother about the proper manner to account for multiple testing; and, it allows continuousre-evaluation of hypotheses after additional data have been collected (Bayesian updating).This tutorial addresses researchers considering to evaluate their hypotheses by meansof the Bayes factor. The focus is completely applied and each topic discussed is illustratedusing Bayes factors for the evaluation of hypotheses in the context of an ANOVA model,obtained using the R package bain. Readers can execute all the analyses presented whilereading this tutorial if they download bain and the R-codes used. It will be elaborated in acompletely non-technical manner: what the Bayes factor is, how it can be obtained, howBayes factors should be interpreted, and what can be done with Bayes factors. Afterreading this tutorial and executing the associated code, researchers will be able to use theirown data for the evaluation of hypotheses by means of the Bayes factor, not only in thecontext of ANOVA models, but also in the context of other statistical models.

scholarly journals Simulation Studies as a Tool to Understand Bayes Factors

2021 ◽  
Vol 4 (1) ◽  
pp. 251524592097262
Author(s):  
Don van Ravenzwaaij ◽  
Alexander Etz
Keyword(s):  
Simulation Study ◽  
Prior Distribution ◽  
Bayes Factor ◽  
R Package ◽  
Bayes Factors ◽  
Simulation Studies ◽  
Social Scientists ◽  
Simple Simulation ◽  
Set Up

When social scientists wish to learn about an empirical phenomenon, they perform an experiment. When they wish to learn about a complex numerical phenomenon, they can perform a simulation study. The goal of this Tutorial is twofold. First, it introduces how to set up a simulation study using the relatively simple example of simulating from the prior. Second, it demonstrates how simulation can be used to learn about the Jeffreys-Zellner-Siow (JZS) Bayes factor, a currently popular implementation of the Bayes factor employed in the BayesFactor R package and freeware program JASP. Many technical expositions on Bayes factors exist, but these may be somewhat inaccessible to researchers who are not specialized in statistics. In a step-by-step approach, this Tutorial shows how a simple simulation script can be used to approximate the calculation of the Bayes factor. We explain how a researcher can write such a sampler to approximate Bayes factors in a few lines of code, what the logic is behind the Savage-Dickey method used to visualize Bayes factors, and what the practical differences are for different choices of the prior distribution used to calculate Bayes factors.

scholarly journals A Markov chain representation of the multiple testing problem

2016 ◽  
Vol 27 (2) ◽  
pp. 364-383 ◽  
Author(s):  
Stefano Cabras
Keyword(s):  
Markov Chain ◽  
Multiple Testing ◽  
Bayes Factor ◽  
Alternative Hypothesis ◽  
Multiple Hypothesis Testing ◽  
Bayes Factors ◽  
Rna Seq ◽  
Markov Transition ◽  
Improper Priors ◽  
Alternative Hypotheses

The problem of multiple hypothesis testing can be represented as a Markov process where a new alternative hypothesis is accepted in accordance with its relative evidence to the currently accepted one. This virtual and not formally observed process provides the most probable set of non null hypotheses given the data; it plays the same role as Markov Chain Monte Carlo in approximating a posterior distribution. To apply this representation and obtain the posterior probabilities over all alternative hypotheses, it is enough to have, for each test, barely defined Bayes Factors, e.g. Bayes Factors obtained up to an unknown constant. Such Bayes Factors may either arise from using default and improper priors or from calibrating p-values with respect to their corresponding Bayes Factor lower bound. Both sources of evidence are used to form a Markov transition kernel on the space of hypotheses. The approach leads to easy interpretable results and involves very simple formulas suitable to analyze large datasets as those arising from gene expression data (microarray or RNA-seq experiments).

scholarly journals Simulation Studies as a Tool to Understand Bayes Factors

2020 ◽  
Author(s):  
Don van Ravenzwaaij ◽  
Alexander Etz
Keyword(s):  
Simulation Study ◽  
Prior Distribution ◽  
Bayes Factor ◽  
R Package ◽  
Bayes Factors ◽  
Simulation Studies ◽  
Social Scientists ◽  
Simple Simulation ◽  
Set Up

When social scientists wish to learn about an empirical phenomenon, they perform an experiment. When they wish to learn about a complex numerical phenomenon,they can perform a simulation study. The goal of this paper is twofold. Firstly, this paper introduces how to set up a simulation study using the relatively simple example of simulating from the prior. Secondly, this paper demonstrates how simulation can be used to learn about the Jeffreys-Zellner-Siow (JZS) Bayes factor: a currently popular implementation of the Bayes factor employed in the BayesFactor R-package and freeware program JASP. Many technical expositions exist on JZS Bayes factors, but these may be somewhat inaccessible to researchers that are not specialized in statistics. This paper aims to show in a step-by-step approach how a simple simulation script can be used to approximate the calculation of the JZS Bayes factor. We explain how a researcher can write such a sampler to approximate JZS Bayes factors in a few lines of code, what the logic is behind the Savage Dickey method used to visualize JZS Bayes factors, and what the practical differences are for different choices of the prior distribution for calculating Bayes factors.

scholarly journals Significance Tests for Binomial Experiments: Ordering the Sample Space by Bayes Factors and Using Adaptive Significance Levels for Decisions

2017 ◽  
Author(s):  
Carlos A. de B. Pereira ◽  
Adriano Polpo ◽  
Eduardo Y. Nakano
Keyword(s):  
Null Hypothesis ◽  
Bayes Factor ◽  
Adaptive Significance ◽  
Bayes Factors ◽  
P Value ◽  
Significance Tests ◽  
Close Link ◽  
Significance Levels ◽  
Level Of Significance ◽  
Lindley’S Paradox

The main objective of this paper is to find a close link between the adaptive level of significance, presented here, and the sample size. We, statisticians, know of the inconsistency, or paradox, in the current classical tests of significance that are based on p-value statistics that is compared to the canonical significance levels (10%, 5% and 1%): "Raise the sample to reject the null hypothesis" is the recommendation of some ill-advised scientists! This paper will show that it is possible to eliminate this problem of significance tests. The Bayesian Lindley's paradox – "increase the sample to accept the hypothesis" – also disappears. Obviously, we present here only the beginning of a possible prominent research. The intention is to extend its use to more complex applications such as survival analysis, reliability tests and other areas. The main tools used here are the Bayes Factor and the extended Neyman-Pearson Lemma.

scholarly journals A Bayesian bird's eye view of ‘Replications of important results in social psychology’

2017 ◽  
Vol 4 (1) ◽  
pp. 160426 ◽  
Author(s):  
Maarten Marsman ◽  
Felix D. Schönbrodt ◽  
Richard D. Morey ◽  
Yuling Yao ◽  
Andrew Gelman ◽  
...  
Keyword(s):  
Social Psychology ◽  
Effect Size ◽  
Null Hypothesis ◽  
Bayes Factor ◽  
Alternative Hypothesis ◽  
Hypothesis Test ◽  
Meta Analysis ◽  
Bayes Factors ◽  
Credible Intervals ◽  
Complementary Techniques

We applied three Bayesian methods to reanalyse the preregistered contributions to the Social Psychology special issue ‘Replications of Important Results in Social Psychology’ (Nosek & Lakens. 2014 Registered reports: a method to increase the credibility of published results. Soc. Psychol. 45 , 137–141. ( doi:10.1027/1864-9335/a000192 )). First, individual-experiment Bayesian parameter estimation revealed that for directed effect size measures, only three out of 44 central 95% credible intervals did not overlap with zero and fell in the expected direction. For undirected effect size measures, only four out of 59 credible intervals contained values greater than 0.10 (10% of variance explained) and only 19 intervals contained values larger than 0.05 . Second, a Bayesian random-effects meta-analysis for all 38 t -tests showed that only one out of the 38 hierarchically estimated credible intervals did not overlap with zero and fell in the expected direction. Third, a Bayes factor hypothesis test was used to quantify the evidence for the null hypothesis against a default one-sided alternative. Only seven out of 60 Bayes factors indicated non-anecdotal support in favour of the alternative hypothesis ( BF 10 > 3 ), whereas 51 Bayes factors indicated at least some support for the null hypothesis. We hope that future analyses of replication success will embrace a more inclusive statistical approach by adopting a wider range of complementary techniques.

scholarly journals Testing theories with Bayes factors

2021 ◽  
Author(s):  
Zoltan Dienes
Keyword(s):  
Social Sciences ◽  
First Principles ◽  
Multiple Testing ◽  
Bayes Factor ◽  
Bayes Factors ◽  
Optional Stopping ◽  
Link Theory ◽  
Cherry Picking

Bayes factors are a useful tool for researchers in the behavioural and social sciences, partly because they can provide evidence for no effect relative to the sort of effect expected. By contrast, a non-significant result does not provide evidence for the H0 tested. So, if non-significance does not in itself count against any theory predicting an effect, how could a theory fail a test? Bayes factors provide a measure of evidence from first principles. A severe test is one that is likely to obtain evidence against a theory if it were false; that is, to obtain an extreme Bayes factor against the theory. Bayes factors show why hacking and cherry picking degrade evidence; how to deal with multiple testing situations; and how optional stopping is consistent with severe testing. Further, informed Bayes factors can be used to link theory tightly to how that theory is tested, so that the measured evidence does relate to the theory.

scholarly journals The Bayes factor: A bridge to Bayesian inference

2017 ◽  
Author(s):  
Matt Williams ◽  
Rasmus A. Bååth ◽  
Michael Carl Philipp
Keyword(s):  
Bayesian Inference ◽  
Bayesian Estimation ◽  
Null Hypothesis ◽  
General Framework ◽  
Bayes Factor ◽  
A Priori ◽  
Bayes Factors ◽  
Point Null Hypothesis

This paper will discuss the concept of Bayes factors as inferential tools that can directly replace NHST in the day-to-day work of developmental researchers. A Bayes factor indicates the degree which data observed should increase (or decrease) our support for one hypothesis in comparison to another. This framework allows researchers to not just reject but also produce evidence in favor of null hypotheses. Bayes factor alternatives to common tests used by developmental psychologists are available in easy-to-use software. However, we note that Bayesian estimation (rather than Bayes factors) may be a more appealing and general framework when a point null hypothesis is a priori implausible.

scholarly journals Network Autocorrelation Modeling: A Bayes Factor Approach for Testing (Multiple) Precise and Interval Hypotheses

2017 ◽  
Vol 48 (3) ◽  
pp. 642-676 ◽  
Author(s):  
Dino Dittrich ◽  
Roger Th. A. J. Leenders ◽  
Joris Mulder
Keyword(s):  
Bayes Factor ◽  
Bayes Factors ◽  
Network Effect ◽  
Superior Performance ◽  
Data Sets ◽  
Efficient Computation ◽  
P Values ◽  
Testing Procedures ◽  
Network Autocorrelation ◽  
Multiple Hypotheses

Currently available (classical) testing procedures for the network autocorrelation can only be used for falsifying a precise null hypothesis of no network effect. Classical methods can be neither used for quantifying evidence for the null nor for testing multiple hypotheses simultaneously. This article presents flexible Bayes factor testing procedures that do not have these limitations. We propose Bayes factors based on an empirical and a uniform prior for the network effect, respectively, first. Next, we develop a fractional Bayes factor where a default prior is automatically constructed. Simulation results suggest that the first two Bayes factors show superior performance and are the Bayes factors we recommend. We apply the recommended Bayes factors to three data sets from the literature and compare the results to those coming from classical analyses using p values. R code for efficient computation of the Bayes factors is provided.

scholarly journals A Bayesian solution to the Behrens–Fisher problem

2021 ◽  
Vol 115 (4) ◽  
Author(s):  
Fco. Javier Girón ◽  
Carmen del Castillo
Keyword(s):  
Closed Form ◽  
Hierarchical Model ◽  
Bayes Factor ◽  
Bayes Factors ◽  
Simple Solution ◽  
Asymptotic Approximations ◽  
Normal Distributions ◽  
Fisher Distribution ◽  
Leibler Divergence

AbstractA simple solution to the Behrens–Fisher problem based on Bayes factors is presented, and its relation with the Behrens–Fisher distribution is explored. The construction of the Bayes factor is based on a simple hierarchical model, and has a closed form based on the densities of general Behrens–Fisher distributions. Simple asymptotic approximations of the Bayes factor, which are functions of the Kullback–Leibler divergence between normal distributions, are given, and it is also proved to be consistent. Some examples and comparisons are also presented.

scholarly journals A region-based multiple testing method for hypotheses ordered in space or time

2015 ◽  
Vol 14 (1) ◽  
pp. 1-19 ◽  
Author(s):  
Rosa J. Meijer ◽  
Thijmen J.P. Krebs ◽  
Jelle J. Goeman
Keyword(s):  
Error Rate ◽  
Null Hypothesis ◽  
Multiple Testing ◽  
Testing Procedure ◽  
Familywise Error Rate ◽  
Expected Number ◽  
Multiple Testing Procedure ◽  
Exact Location ◽  
Testing Method

AbstractWe present a multiple testing method for hypotheses that are ordered in space or time. Given such hypotheses, the elementary hypotheses as well as regions of consecutive hypotheses are of interest. These region hypotheses not only have intrinsic meaning but testing them also has the advantage that (potentially small) signals across a region are combined in one test. Because the expected number and length of potentially interesting regions are usually not available beforehand, we propose a method that tests all possible region hypotheses as well as all individual hypotheses in a single multiple testing procedure that controls the familywise error rate. We start at testing the global null-hypothesis and when this hypothesis can be rejected we continue with further specifying the exact location/locations of the effect present. The method is implemented in the

Sign in / Sign up

Export Citation Format

Share Document