scholarly journals A Critical Evaluation of the FBST ev for Bayesian Hypothesis Testing

2021 ◽  
Author(s):  
Alexander Ly ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Critical Evaluation ◽  
Significance Test ◽  
Bayesian Hypothesis Testing ◽  
Absence Of Evidence ◽  
Evidence Of Absence ◽  
Foregone Conclusion

AbstractThe “Full Bayesian Significance Test e-value”, henceforth FBST ev, has received increasing attention across a range of disciplines including psychology. We show that the FBST ev leads to four problems: (1) the FBST ev cannot quantify evidence in favor of a null hypothesis and therefore also cannot discriminate “evidence of absence” from “absence of evidence”; (2) the FBST ev is susceptible to sampling to a foregone conclusion; (3) the FBST ev violates the principle of predictive irrelevance, such that it is affected by data that are equally likely to occur under the null hypothesis and the alternative hypothesis; (4) the FBST ev suffers from the Jeffreys-Lindley paradox in that it does not include a correction for selection. These problems also plague the frequentist p-value. We conclude that although the FBST ev may be an improvement over the p-value, it does not provide a reasonable measure of evidence against the null hypothesis.

scholarly journals A Critical Evaluation of the FBST ev for Bayesian Hypothesis Testing

2021 ◽  
Author(s):  
Alexander Ly ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Critical Evaluation ◽  
Significance Test ◽  
P Value ◽  
Bayesian Hypothesis Testing ◽  
Absence Of Evidence ◽  
Evidence Of Absence ◽  
Foregone Conclusion

he “Full Bayesian Significance Test e-value”, henceforth FBST ev, has received increasing attention across a range of disciplines including psychology. We show that the FBST ev leads to four problems: (1) the FBST ev cannot quantify evidence in favor of a null hypothesis and therefore also cannot discriminate “evidence of absence” from “absence of evidence”; (2) the FBST ev is susceptible to sampling to a foregone conclusion; (3) the FBST ev violates the principle of predictive irrelevance, such that it is affected by data that are equally likely to occur under the null hypothesis and the alternative hypothesis; (4) the FBST ev suffers from the Jeffreys-Lindley paradox in that it does not include a correction for selection. These problems also plague the frequentist p-value. We conclude that although the FBST ev may be an improvement over the p-value, it does not provide a reasonable measure of evidence against the null hypothesis.

Hypothesis testing with error correction models

2021 ◽  
pp. 1-9
Author(s):  
Patrick W. Kraft ◽  
Ellen M. Key ◽  
Matthew J. Lebo
Keyword(s):  
Hypothesis Testing ◽  
Error Correction ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Correction Coefficient ◽  
Correction Model ◽  
Long Run ◽  
Independent Variable ◽  
The Right ◽  
General Error

Abstract Grant and Lebo (2016) and Keele et al. (2016) clarify the conditions under which the popular general error correction model (GECM) can be used and interpreted easily: In a bivariate GECM the data must be integrated in order to rely on the error correction coefficient, $\alpha _1^\ast$ , to test cointegration and measure the rate of error correction between a single exogenous x and a dependent variable, y. Here we demonstrate that even if the data are all integrated, the test on $\alpha _1^\ast$ is misunderstood when there is more than a single independent variable. The null hypothesis is that there is no cointegration between y and any x but the correct alternative hypothesis is that y is cointegrated with at least one—but not necessarily more than one—of the x's. A significant $\alpha _1^\ast$ can occur when some I(1) regressors are not cointegrated and the equation is not balanced. Thus, the correct limiting distributions of the right-hand-side long-run coefficients may be unknown. We use simulations to demonstrate the problem and then discuss implications for applied examples.

scholarly journals Asymptotic relative submajorization of multiple-state boxes

2021 ◽  
Vol 111 (4) ◽  
Author(s):  
Gergely Bunth ◽  
Péter Vrana
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Standard Form ◽  
Point Of View ◽  
State Discrimination ◽  
Simple Alternative ◽  
Multiple State ◽  
Error Probabilities

AbstractPairs of states, or “boxes” are the basic objects in the resource theory of asymmetric distinguishability (Wang and Wilde in Phys Rev Res 1(3):033170, 2019. 10.1103/PhysRevResearch.1.033170), where free operations are arbitrary quantum channels that are applied to both states. From this point of view, hypothesis testing is seen as a process by which a standard form of distinguishability is distilled. Motivated by the more general problem of quantum state discrimination, we consider boxes of a fixed finite number of states and study an extension of the relative submajorization preorder to such objects. In this relation, a tuple of positive operators is greater than another if there is a completely positive trace nonincreasing map under which the image of the first tuple satisfies certain semidefinite constraints relative to the other one. This preorder characterizes error probabilities in the case of testing a composite null hypothesis against a simple alternative hypothesis, as well as certain error probabilities in state discrimination. We present a sufficient condition for the existence of catalytic transformations between boxes, and a characterization of an associated asymptotic preorder, both expressed in terms of sandwiched Rényi divergences. This characterization of the asymptotic preorder directly shows that the strong converse exponent for a composite null hypothesis is equal to the maximum of the corresponding exponents for the pairwise simple hypothesis testing tasks.

Interval estimation and inference

2019 ◽  
pp. 111-138
Author(s):  
M. D. Edge
Keyword(s):  
Standard Deviation ◽  
Hypothesis Testing ◽  
Confidence Intervals ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Interval Estimation ◽  
Research Literature ◽  
Significance Level ◽  
Specific Hypothesis ◽  
True Value

Interval estimation is the attempt to define intervals that quantify the degree of uncertainty in an estimate. The standard deviation of an estimate is called a standard error. Confidence intervals are designed to cover the true value of an estimand with a specified probability. Hypothesis testing is the attempt to assess the degree of evidence for or against a specific hypothesis. One tool for frequentist hypothesis testing is the p value, or the probability that if the null hypothesis is in fact true, the data would depart as extremely or more extremely from expectations under the null hypothesis than they were observed to do. In Neyman–Pearson hypothesis testing, the null hypothesis is rejected if p is less than a pre-specified value, often chosen to be 0.05. A test’s power function gives the probability that the null hypothesis is rejected given the significance level γ‎, a sample size n, and a specified alternative hypothesis. This chapter discusses some limitations of hypothesis testing as commonly practiced in the research literature.

scholarly journals Utility Theory as a Method to Minimise the Risk in Deformation Analysis Decisions

2014 ◽  
Vol 8 (4) ◽  
Author(s):  
Yin Zhang ◽  
Ingo Neumann
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Utility Theory ◽  
Alternative Hypothesis ◽  
Main Idea ◽  
Deformation Monitoring ◽  
Deformation Analysis ◽  
Utility Values ◽  
Alternative Hypotheses ◽  
The Given

AbstractDeformation monitoring usually focuses on the detection of whether the monitored objects satisfy the given properties (e.g. being stable or not), and makes further decisions to minimise the risks, for example, the consequences and costs in case of collapse of artificial objects and/or natural hazards. With this intention, a methodology relying on hypothesis testing and utility theory is reviewed in this paper. The main idea of utility theory is to judge each possible outcome with a utility value. The presented methodology makes it possible to minimise the risk of an individual monitoring project by considering the costs and consequences of overall possible situations within the decision process. It is not the danger that the monitored object may collapse that can be reduced. The risk (based on the utility values multiplied by the danger) can be described more appropriately and therefore more valuable decisions can be made. Especially, the opportunity for the measurement process to minimise the risk is an important key issue. In this paper, application of the methodology to two of the classical cases in hypothesis testing will be discussed in detail: 1) both probability density functions (pdfs) of tested objects under null and alternative hypotheses are known; 2) only the pdf under the null hypothesis is known and the alternative hypothesis is treated as the pure negation of the null hypothesis. Afterwards, a practical example in deformation monitoring is introduced and analysed. Additionally, the way in which the magnitudes of utility values (consequences of a decision) influence the decision will be considered and discussed at the end.

Hypothesis Testing

2005 ◽  
Author(s):  
D. Brynn Hibbert ◽  
J. Justin Gooding
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Hypothesis Test ◽  
Significance Test ◽  
Type I ◽  
Test Statistic ◽  
The Difference ◽  
Set Up ◽  
Type Ii Errors ◽  
Normally Distributed

• To understand the concept of the null hypothesis and the role of Type I and Type II errors. • To test that data are normally distributed and whether a datum is an outlier. • To determine whether there is systematic error in the mean of measurement results. • To perform tests to compare the means of two sets of data.… One of the uses to which data analysis is put is to answer questions about the data, or about the system that the data describes. In the former category are ‘‘is the data normally distributed?’’ and ‘‘are there any outliers in the data?’’ (see the discussions in chapter 1). Questions about the system might be ‘‘is the level of alcohol in the suspect’s blood greater than 0.05 g/100 mL?’’ or ‘‘does the new sensor give the same results as the traditional method?’’ In answering these questions we determine the probability of finding the data given the truth of a stated hypothesis—hence ‘‘hypothesis testing.’’ A hypothesis is a statement that might, or might not, be true. Usually the hypothesis is set up in such a way that it is possible to calculate the probability (P) of the data (or the test statistic calculated from the data) given the hypothesis, and then to make a decision about whether the hypothesis is to be accepted (high P) or rejected (low P). A particular case of a hypothesis test is one that determines whether or not the difference between two values is significant—a significance test. For this case we actually put forward the hypothesis that there is no real difference and the observed difference arises from random effects: it is called the null hypothesis (H<sub>0</sub>). If the probability that the data are consistent with the null hypothesis falls below a predetermined low value (say 0.05 or 0.01), then the hypothesis is rejected at that probability. Therefore, p<0.05 means that if the null hypothesis were true we would find the observed data (or more accurately the value of the statistic, or greater, calculated from the data) in less than 5% of repeated experiments.

scholarly journals Advantages Masquerading as ‘Issues’ in Bayesian Hypothesis Testing: A Commentary on Tendeiro and Kiers (2019)

2019 ◽  
Author(s):  
Don van Ravenzwaaij ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Bayes Factor ◽  
Statistical Evidence ◽  
Central Component ◽  
P Values ◽  
Bayesian Hypothesis Testing ◽  
Bayesian Testing

Tendeiro and Kiers (2019) provide a detailed and scholarly critique of Null Hypothesis Bayesian Testing (NHBT) and its central component –the Bayes factor– that allows researchers to update knowledge and quantify statistical evidence. Tendeiro and Kiers conclude that NHBT constitutes an improvement over frequentist p-values, but primarily elaborate on a list of eleven ‘issues’ of NHBT. In this commentary, we provide context to each issue and conclude that many issues may in fact be conceived as pronounced advantages of NHBT.

scholarly journals Possibility Measure of Accepting Statistical Hypothesis

Mathematics ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 551
Author(s):  
Jung-Lin Hung ◽  
Cheng-Che Chen ◽  
Chun-Mei Lai
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Statistical Testing ◽  
Statistical Hypothesis ◽  
Statistical Hypothesis Testing ◽  
Test Statistic ◽  
Possibility Measure ◽  
Testing Approach ◽  
Hypothesis Testing Approach

Taking advantage of the possibility of fuzzy test statistic falling in the rejection region, a statistical hypothesis testing approach for fuzzy data is proposed in this study. In contrast to classical statistical testing, which yields a binary decision to reject or to accept a null hypothesis, the proposed approach is to determine the possibility of accepting a null hypothesis (or alternative hypothesis). When data are crisp, the proposed approach reduces to the classical hypothesis testing approach.

scholarly journals The Importance of Random Slopes in Mixed Models for Bayesian Hypothesis Testing

2021 ◽  
Author(s):  
Klaus Oberauer
Keyword(s):  
Hypothesis Testing ◽  
Strong Evidence ◽  
Mixed Models ◽  
False Positive ◽  
Null Hypothesis ◽  
Population Level ◽  
False Positives ◽  
Bayes Factors ◽  
Bayesian Hypothesis Testing ◽  
Random Slopes

Mixed models are gaining popularity in psychology. For frequentist mixed models, Barr, Levy, Scheepers, and Tily (2013) showed that excluding random slopes – differences between individuals in the direction and size of an effect – from a model when they are in the data can lead to a substantial increase in false-positive conclusions in null-hypothesis tests. Here I demonstrate through five simulations that the same is true for Bayesian hypothesis testing with mixed models, often yielding Bayes factors reflecting very strong evidence for a mean effect on the population level even if there was no such effect. Including random slopes in the model largely eliminates the risk of strong false positives, but reduces the chance of obtaining strong evidence for true effects. I recommend starting analysis with testing the support for random slopes in the data, and removing them from the models only if there is clear evidence against them.

Sign in / Sign up

Export Citation Format

Share Document