scholarly journals Are You Satisfied with Statistical Significance? Issues in and Resolutions for Null Hypothesis Significance Testing

2019 ◽  
Vol 31 (3) ◽  
pp. 82-90
Author(s):  
Hideaki SHIMADA ◽  
Ryuta ISEKI
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing

The Numbers Will Love You Back in Return—I Promise

2016 ◽  
Vol 11 (4) ◽  
pp. 551-554 ◽  
Author(s):  
Martin Buchheit
Keyword(s):  
Sample Size ◽  
Null Hypothesis ◽  
Clinical Medicine ◽  
Statistical Significance ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Sport Science ◽  
Size Dependent ◽  
Research Questions ◽  
Per Se

The first sport-science-oriented and comprehensive paper on magnitude-based inferences (MBI) was published 10 y ago in the first issue of this journal. While debate continues, MBI is today well established in sport science and in other fields, particularly clinical medicine, where practical/clinical significance often takes priority over statistical significance. In this commentary, some reasons why both academics and sport scientists should abandon null-hypothesis significance testing and embrace MBI are reviewed. Apparent limitations and future areas of research are also discussed. The following arguments are presented: P values and, in turn, study conclusions are sample-size dependent, irrespective of the size of the effect; significance does not inform on magnitude of effects, yet magnitude is what matters the most; MBI allows authors to be honest with their sample size and better acknowledge trivial effects; the examination of magnitudes per se helps provide better research questions; MBI can be applied to assess changes in individuals; MBI improves data visualization; and MBI is supported by spreadsheets freely available on the Internet. Finally, recommendations to define the smallest important effect and improve the presentation of standardized effects are presented.

scholarly journals Failing Grade: 89% of Introduction-to-Psychology Textbooks That Define or Explain Statistical Significance Do So Incorrectly

2019 ◽  
Vol 2 (3) ◽  
pp. 233-239 ◽  
Author(s):  
Scott A. Cassidy ◽  
Ralitza Dimova ◽  
Benjamin Giguère ◽  
Jeffrey R. Spence ◽  
David J. Stanley
Keyword(s):  
United States ◽  
Null Hypothesis ◽  
Classroom Instruction ◽  
Statistical Significance ◽  
Psychology Students ◽  
The United States ◽  
Introductory Psychology ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Do So

Null-hypothesis significance testing (NHST) is commonly used in psychology; however, it is widely acknowledged that NHST is not well understood by either psychology professors or psychology students. In the current study, we investigated whether introduction-to-psychology textbooks accurately define and explain statistical significance. We examined 30 introductory-psychology textbooks, including the best-selling books from the United States and Canada, and found that 89% incorrectly defined or explained statistical significance. Incorrect definitions and explanations were most often consistent with the odds-against-chance fallacy. These results suggest that it is common for introduction-to-psychology students to be taught incorrect interpretations of statistical significance. We hope that our results will create awareness among authors of introductory-psychology books and provide the impetus for corrective action. To help with classroom instruction, we provide slides that correctly describe NHST and may be useful for introductory-psychology instructors.

Tests of Statistical Significance

2018 ◽  
Author(s):  
Brian D. Haig
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Lessons Learned ◽  
Significance Testing ◽  
Behavioral Sciences ◽  
Null Hypothesis Significance Testing ◽  
Tests Of Significance ◽  
Philosophy Of Statistics ◽  
Critical Literature

Chapter 3 provides a brief overview of null hypothesis significance testing and points out its primary defects. It then outlines the neo-Fisherian account of tests of statistical significance, along with a second option contained in the philosophy of statistics known as the error-statistical philosophy, both of which are defensible. Tests of statistical significance are the most widely used means for evaluating hypotheses and theories in psychology. A massive critical literature has developed in psychology, and the behavioral sciences more generally, regarding the worth of these tests. The chapter provides a list of important lessons learned from the ongoing debates about tests of significance.

Statistical significance testing, hypothetico-deductive method, and theory evaluation

2000 ◽  
Vol 23 (2) ◽  
pp. 292-293 ◽  
Author(s):  
Brian D. Haig
Keyword(s):  
Null Hypothesis ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Statistical Significance Testing ◽  
Theory Evaluation ◽  
Limited Role ◽  
Deductive Method

Chow's endorsement of a limited role for null hypothesis significance testing is a needed corrective of research malpractice, but his decision to place this procedure in a hypothetico-deductive framework of Popperian cast is unwise. Various failures of this version of the hypothetico-deductive method have negative implications for Chow's treatment of significance testing, meta-analysis, and theory evaluation.

Costs and benefits of statistical significance tests

1998 ◽  
Vol 21 (2) ◽  
pp. 218-219
Author(s):  
Michael G. Shafto
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Testing Procedure ◽  
Significance Testing ◽  
Costs And Benefits ◽  
Significance Tests ◽  
Null Hypothesis Significance Testing ◽  
Social Scientists ◽  
Conventional Tests

Chow's book provides a thorough analysis of the confusing array of issues surrounding conventional tests of statistical significance. This book should be required reading for behavioral and social scientists. Chow concludes that the null-hypothesis significance-testing procedure (NHSTP) plays a limited, but necessary, role in the experimental sciences. Another possibility is that – owing in part to its metaphorical underpinnings and convoluted logic – the NHSTP is declining in importance in those few sciences in which it ever played a role.

scholarly journals Do psychology students interpret null hypothesis significance testing critically?

2017 ◽  
Author(s):  
Ivan Flis
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Statistical Significance ◽  
Psychology Students ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Graduate Studies ◽  
Null Hypothesis Testing ◽  
Level Of Understanding ◽  
Grade Average

The goal of the study was to descriptively analyze the understanding of null hypothesis significance testing among Croatian psychology students considering how it is usually understood in textbooks, which is subject to Bayesian and interpretative criticism. Also, the thesis represents a short overview of the discussions on the meaning of significance testing and how it is taught to students. There were 350 participants from undergraduate and graduate programs at five faculties in Croatia (Zagreb – Centre for Croatian Studies and Faculty of Humanities and Social Sciences, Rijeka, Zadar, Osijek). Another goal was to ascertain if the understanding of null hypothesis testing among psychology students can be predicted by their grades, attitudes and interests. The level of understanding of null hypothesis testing was measured by the Test of statistical significance misinterpretations (NHST test) (Oakes, 1986; Haller and Krauss, 2002). The attitudes toward null hypothesis significance testing were measured by a questionnaire that was constructed for this study. The grades were operationalized as the grade average of courses taken during undergraduate studies, and as a separate grade average of methodological courses taken during undergraduate and graduate studies. The students have shown limited understanding of null hypothesis testing – the percentage of correct answers in the NHST test was not higher than 56% for any of the six items. Croatian students have also shown less understanding on each item when compared to the German students in Haller and Krauss’s (2002) study. None of the variables – general grade average, average in the methodological courses, two variables measuring the attitude toward null hypothesis significance testing, failing at least one methodological course, and the variable of main interest in psychology – were predictive for the odds of answering the items in the NHST test correctly. The conclusion of the study is that education practices in teaching students the meaning and interpretation of null hypothesis significance testing have to be taken under consideration at Croatian psychology departments.

scholarly journals The null hypothesis is always rejected with statistical tricks: Why do you need it?

2019 ◽  
Vol 53 (1) ◽  
pp. 17-27
Author(s):  
Freddy A. Paniagua
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Effect Sizes ◽  
Practical Significance ◽  
Significance Testing ◽  
Behavioral Sciences ◽  
Null Hypothesis Significance Testing ◽  
Very High

Ferguson (2015) observed that the proportion of studies supporting the experimental hypothesis and rejecting the null hypothesis is very high. This paper argues that the reason for this scenario is that researchers in the behavioral sciences have learned that the null hypothesis can always be rejected if one knows the statistical tricks to reject it (e.g., the probability of rejecting the null hypothesis increases with p = 0.05 compare to p = 0.01). Examples of the advancement of science without the need to formulate the null hypothesis are also discussed, as well as alternatives to null hypothesis significance testing-NHST (e.g., effect sizes), and the importance to distinguish the statistical significance from the practical significance of results.  

scholarly journals Against statistical significance testing in corpus linguistics

2019 ◽  
Vol 15 (2) ◽  
pp. 321-346 ◽  
Author(s):  
Alexander Koplenig
Keyword(s):  
Corpus Linguistics ◽  
Null Hypothesis ◽  
Statistical Significance ◽  
Linguistic Theory ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Statistical Significance Testing ◽  
Naturally Occurring ◽  
Language Data

Abstract In the first volume of Corpus Linguistics and Linguistic Theory, Gries (2005. Null-hypothesis significance testing of word frequencies: A follow-up on Kilgarriff. Corpus Linguistics and Linguistic Theory 1(2). doi:10.1515/cllt.2005.1.2.277. http://www.degruyter.com/view/j/cllt.2005.1.issue-2/cllt.2005.1.2.277/cllt.2005.1.2.277.xml: 285) asked whether corpus linguists should abandon null-hypothesis significance testing. In this paper, I want to revive this discussion by defending the argument that the assumptions that allow inferences about a given population – in this case about the studied languages – based on results observed in a sample – in this case a collection of naturally occurring language data – are not fulfilled. As a consequence, corpus linguists should indeed abandon null-hypothesis significance testing.

Significance testing – does it need this defence?

1998 ◽  
Vol 21 (2) ◽  
pp. 214-215
Author(s):  
Günther Palm
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing

Chow's (1996) Statistical significance is a defence of null-hypothesis significance testing (NHSTP). The most common and straightforward use of significance testing is for the statistical corroboration of general hypotheses. In this case, criticisms of NHSTP, at least those mentioned in the book, are unfounded or misdirected. This point is driven home by the author a bit too forcefully and meticulously. The awkward and cumbersome organisation and argumentation of the book makes it even harder to read.

scholarly journals Tests of Statistical Significance Made Sound

2016 ◽  
Vol 77 (3) ◽  
pp. 489-506 ◽  
Author(s):  
Brian D. Haig
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Psychological Research ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Alternative Account ◽  
Definite Improvement ◽  
Enormous Amount ◽  
Uncritical Acceptance ◽  
The Subject

This article considers the nature and place of tests of statistical significance (ToSS) in science, with particular reference to psychology. Despite the enormous amount of attention given to this topic, psychology’s understanding of ToSS remains deficient. The major problem stems from a widespread and uncritical acceptance of null hypothesis significance testing (NHST), which is an indefensible amalgam of ideas adapted from Fisher’s thinking on the subject and from Neyman and Pearson’s alternative account. To correct for the deficiencies of the hybrid, it is suggested that psychology avail itself of two important and more recent viewpoints on ToSS, namely the neo-Fisherian and the error-statistical perspectives. The neo-Fisherian perspective endeavors to improve on Fisher’s original account and rejects key elements of Neyman and Pearson’s alternative. In contrast, the error-statistical perspective builds on the strengths of both statistical traditions. It is suggested that these more recent outlooks on ToSS are a definite improvement on NHST, especially the error-statistical position. It is suggested that ToSS can play a useful, if limited, role in psychological research. At the end, some lessons learnt from the extensive debates about ToSS are presented.

Sign in / Sign up

Export Citation Format

Share Document