Nonsignificant P values cannot prove null hypothesis: Absence of evidence is not evidence of absence

AbstractThe “Full Bayesian Significance Test e-value”, henceforth FBST ev, has received increasing attention across a range of disciplines including psychology. We show that the FBST ev leads to four problems: (1) the FBST ev cannot quantify evidence in favor of a null hypothesis and therefore also cannot discriminate “evidence of absence” from “absence of evidence”; (2) the FBST ev is susceptible to sampling to a foregone conclusion; (3) the FBST ev violates the principle of predictive irrelevance, such that it is affected by data that are equally likely to occur under the null hypothesis and the alternative hypothesis; (4) the FBST ev suffers from the Jeffreys-Lindley paradox in that it does not include a correction for selection. These problems also plague the frequentist p-value. We conclude that although the FBST ev may be an improvement over the p-value, it does not provide a reasonable measure of evidence against the null hypothesis.

Download Full-text

A Critical Evaluation of the FBST ev for Bayesian Hypothesis Testing

10.31234/osf.io/x9t6n ◽

2021 ◽

Author(s):

Alexander Ly ◽

Eric-Jan Wagenmakers

Keyword(s):

Hypothesis Testing ◽

Null Hypothesis ◽

Alternative Hypothesis ◽

Critical Evaluation ◽

Significance Test ◽

P Value ◽

Bayesian Hypothesis Testing ◽

Absence Of Evidence ◽

Evidence Of Absence ◽

Foregone Conclusion

he “Full Bayesian Significance Test e-value”, henceforth FBST ev, has received increasing attention across a range of disciplines including psychology. We show that the FBST ev leads to four problems: (1) the FBST ev cannot quantify evidence in favor of a null hypothesis and therefore also cannot discriminate “evidence of absence” from “absence of evidence”; (2) the FBST ev is susceptible to sampling to a foregone conclusion; (3) the FBST ev violates the principle of predictive irrelevance, such that it is affected by data that are equally likely to occur under the null hypothesis and the alternative hypothesis; (4) the FBST ev suffers from the Jeffreys-Lindley paradox in that it does not include a correction for selection. These problems also plague the frequentist p-value. We conclude that although the FBST ev may be an improvement over the p-value, it does not provide a reasonable measure of evidence against the null hypothesis.

Download Full-text

Genotype Pattern Mining for Pairs of Interacting Variants Underlying Digenic Traits

Genes ◽

10.3390/genes12081160 ◽

2021 ◽

Vol 12 (8) ◽

pp. 1160

Author(s):

Atsuko Okazaki ◽

Sukanya Horpaopan ◽

Qingrun Zhang ◽

Matthew Randesi ◽

Jurg Ott

Keyword(s):

Null Hypothesis ◽

Pattern Mining ◽

Genetic Diseases ◽

Frequent Pattern Mining ◽

Case Control ◽

Frequent Pattern ◽

Permutation Testing ◽

Case Control Studies ◽

P Values ◽

Dna Variants

Some genetic diseases (“digenic traits”) are due to the interaction between two DNA variants, which presumably reflects biochemical interactions. For example, certain forms of Retinitis Pigmentosa, a type of blindness, occur in the presence of two mutant variants, one each in the ROM1 and RDS genes, while the occurrence of only one such variant results in a normal phenotype. Detecting variant pairs underlying digenic traits by standard genetic methods is difficult and is downright impossible when individual variants alone have minimal effects. Frequent pattern mining (FPM) methods are known to detect patterns of items. We make use of FPM approaches to find pairs of genotypes (from different variants) that can discriminate between cases and controls. Our method is based on genotype patterns of length two, and permutation testing allows assigning p-values to genotype patterns, where the null hypothesis refers to equal pattern frequencies in cases and controls. We compare different interaction search approaches and their properties on the basis of published datasets. Our implementation of FPM to case-control studies is freely available.

Download Full-text

Comparison of photobiomodulation and photodynamic therapy as adjuncts to mechanical debridement for the treatment of peri-implantitis

Technology and Health Care ◽

10.3233/thc-213062 ◽

2021 ◽

pp. 1-10

Author(s):

Mansour H. Al-Askar ◽

Fahad A. Abdullatif ◽

Abdulmonem A. Alshihri ◽

Asma Ahmed ◽

Darshan Devang Divakar ◽

...

Keyword(s):

Photodynamic Therapy ◽

Null Hypothesis ◽

Gingival Index ◽

P Values ◽

Inflammatory Parameters ◽

Probing Depth ◽

Crestal Bone Loss ◽

Group 3 ◽

Soft Tissue Inflammation

BACKGROUND AND OBJECTIVE: The aim of this study was to compare the efficacy of photobiomodulation therapy (PBMT) and photodynamic therapy (PDT) as adjuncts to mechanical debridement (MD) for the treatment of peri-implantitis. The present study is based on the null hypothesis that there is no difference in the peri-implant inflammatory parameters (modified plaque index [mPI], modified gingival index [mGI], probing depth [PD]) and crestal bone loss (CBL) following MD either with PBMT or PDT in patients with peri-implantitis. METHODS: Forty-nine patients with peri-implantitis were randomly categorized into three groups. In Groups 1 and 2, patients underwent MD with adjunct PBMT and PDT, respectively. In Group 3, patients underwent MD alone (controls). Peri-implant inflammatory parameters were measured at baseline and 3-months follow-up. P-values < 0.01 were considered statistically significant. RESULTS: At baseline, peri-implant clinicoradiographic parameters were comparable in all groups. Compared with baseline, there was a significant reduction in mPI (P< 0.001), mGI (P< 0.001) and PD (P< 0.001) in Groups 1 and 2 at 3-months follow-up. In Group 3, there was no difference in the scores of mPI, mGI and PD at follow-up. At 3-months follow-up, there was no difference in mPI, mGI and PD among patients in Groups 1 and 2. The mPI (P< 0.001), mGI (P< 0.001) and PD (P< 0.001) were significantly higher in Group 3 than Groups 1 and 2. The CBL was comparable in all groups at follow-up. CONCLUSION: PBMT and PDT seem to be useful adjuncts to MD for the treatment of peri-implant soft-tissue inflammation among patients with peri-implantitis.

Download Full-text

A Frequentist Alternative to Significance Testing, p-Values, and Confidence Intervals

Econometrics ◽

10.3390/econometrics7020026 ◽

2019 ◽

Vol 7 (2) ◽

pp. 26 ◽

Cited By ~ 7

Author(s):

David Trafimow

Keyword(s):

Present Article ◽

Confidence Intervals ◽

Null Hypothesis ◽

A Priori ◽

Significance Testing ◽

Population Parameters ◽

Null Hypothesis Significance Testing ◽

P Values ◽

Statistical Procedures ◽

Major Section

There has been much debate about null hypothesis significance testing, p-values without null hypothesis significance testing, and confidence intervals. The first major section of the present article addresses some of the main reasons these procedures are problematic. The conclusion is that none of them are satisfactory. However, there is a new procedure, termed the a priori procedure (APP), that validly aids researchers in obtaining sample statistics that have acceptable probabilities of being close to their corresponding population parameters. The second major section provides a description and review of APP advances. Not only does the APP avoid the problems that plague other inferential statistical procedures, but it is easy to perform too. Although the APP can be performed in conjunction with other procedures, the present recommendation is that it be used alone.

Download Full-text

CONCEPTUAL KNOWLEDGE OF CONFIDENCE INTERVALS IN PSYCHOLOGY UNDERGRADUATE AND GRADUATE STUDENTS

STATISTICS EDUCATION RESEARCH JOURNAL ◽

10.52041/serj.v18i1.149 ◽

2019 ◽

Vol 18 (1) ◽

pp. 46-62

Author(s):

NOELLE M. CROOKS ◽

ANNA N. BARTEL ◽

MARTHA W. ALIBALI

Keyword(s):

Graduate Students ◽

Confidence Intervals ◽

Null Hypothesis ◽

Conceptual Knowledge ◽

Statistics Education ◽

Psychology Students ◽

Sample Mean ◽

P Values ◽

Research Journal ◽

Graduate Psychology Students

In recent years, there have been calls for researchers to report and interpret confidence intervals (CIs) rather than relying solely on p-values. Such reforms, however, may be hindered by a general lack of understanding of CIs and how to interpret them. In this study, we assessed conceptual knowledge of CIs in undergraduate and graduate psychology students. CIs were difficult and prone to misconceptions for both groups. Connecting CIs to estimation and sample mean concepts was associated with greater conceptual knowledge of CIs. Connecting CIs to null hypothesis significance testing, however, was not associated with conceptual knowledge of CIs. It may therefore be beneficial to focus on estimation and sample mean concepts in instruction about CIs. First published May 2019 at Statistics Education Research Journal Archives

Download Full-text

NO EVIDENCE, OR NOT ENOUGH EVIDENCE?

PEDIATRICS ◽

10.1542/peds.98.6.a22a ◽

1996 ◽

Vol 98 (6) ◽

pp. A22-A22

Author(s):

Student

Keyword(s):

Confidence Intervals ◽

P Value ◽

P Values ◽

Considerable Uncertainty ◽

Absence Of Evidence

When we are told that "there's no evidence that A causes B," we should first ask whether absence of evidence means simply that there is no information at all. If there are data, we should look for quantification of the association rather than just a P value. Where risks are small, P values may well mislead: confidence intervals are likely to be wide, indicating considerable uncertainty.

Download Full-text

INTRODUCTORY STATISTICS COURSE TERTIARY STUDENTS’ UNDERSTANDING OF P-VALUES

STATISTICS EDUCATION RESEARCH JOURNAL ◽

10.52041/serj.v13i1.298 ◽

2014 ◽

Vol 13 (1) ◽

pp. 53-65 ◽

Cited By ~ 1

Author(s):

ROBYN REABURN

Keyword(s):

Null Hypothesis ◽

Probabilistic Reasoning ◽

Statistics Education ◽

Introductory Statistics ◽

P Values ◽

Testing Procedures ◽

Research Journal ◽

Null Hypothesis Testing ◽

Statistics Course ◽

Teaching Programs

This study aimed to gain knowledge of students’ beliefs and difficulties in understanding p-values, and to use this knowledge to develop improved teaching programs. This study took place over four consecutive teaching semesters of a one-semester tertiary statistics unit. The study was cyclical, in that the results of each semester were used to inform the instructional design for the following semester. Over the semesters, the following instructional techniques were introduced: computer simulation, the introduction of hypothetical probabilistic reasoning using a familiar context, and the use of alternative representations. The students were also encouraged to write about their work. As the interventions progressed, a higher proportion of students successfully defined and used p-values in Null Hypothesis Testing procedures. First published May 2014 at Statistics Education Research Journal Archives

Download Full-text