scholarly journals New-day statistical thinking: A bold proposal for a radical change in practices

2019 ◽  
Vol 51 (2) ◽  
pp. 274-278 ◽  
Author(s):  
Arjen van Witteloostuijn
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Radical Change ◽  
Significance Testing ◽  
Statistical Thinking ◽  
Special Issue ◽  
Statistical Significance Testing ◽  
Questionable Research Practices ◽  
Know How ◽  
Null Hypothesis Testing

AbstractIn this commentary, I argue why we should stop engaging in null hypothesis statistical significance testing altogether. Artificial and misleading it may be, but we know how to play the p value threshold and null hypothesis-testing game. We feel secure; we love the certainty. The fly in the ointment is that the conventions have led to questionable research practices. Wasserstein, Schirm, & Lazar (Am Stat 73(sup1):1–19, 2019. 10.1080/00031305.2019.1583913) explain why, in their thought-provoking editorial introducing a special issue of The American Statistician: “As ‘statistical significance’ is used less, statistical thinking will be used more.” Perhaps we empirical researchers can together find a way to work ourselves out of the straitjacket that binds us.

Statistical significance testing, hypothetico-deductive method, and theory evaluation

2000 ◽  
Vol 23 (2) ◽  
pp. 292-293 ◽  
Author(s):  
Brian D. Haig
Keyword(s):  
Null Hypothesis ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Statistical Significance Testing ◽  
Theory Evaluation ◽  
Limited Role ◽  
Deductive Method

Chow's endorsement of a limited role for null hypothesis significance testing is a needed corrective of research malpractice, but his decision to place this procedure in a hypothetico-deductive framework of Popperian cast is unwise. Various failures of this version of the hypothetico-deductive method have negative implications for Chow's treatment of significance testing, meta-analysis, and theory evaluation.

Lady Justice Versus Cult of Statistical Significance

2015 ◽  
pp. 351-364
Author(s):  
Stephen T. Ziliak ◽  
Deirdre McCloskey
Keyword(s):  
Loss Function ◽  
Null Hypothesis ◽  
Statistical Significance ◽  
Ethical Problem ◽  
Supreme Court Decision ◽  
Court Decision ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
Economic Significance ◽  
Random Variance

Economics and other sciences use null hypothesis statistical significance testing without a loss function and avoid asking “how big is a big loss or gain?.” Statistical significance is not equivalent to economic significance; the mistake is evident when one reflects that the estimated payoff from a lottery is not the same as the odds of winning that lottery. Yet a widespread failure to make the distinction between an estimate of human consequence and an estimate of its probability—between the meaning of an estimated average and the random variance around it—is killing people in medicine and impoverishing people in economics. The ethical problem created by a test of statistical significance is made worse by the method’s blatant illogic at the foundational level, a fact unacknowledged by most of those depending on it. Several changes to the literature and a recent Supreme Court decision could help.

scholarly journals Do psychology students interpret null hypothesis significance testing critically?

2017 ◽  
Author(s):  
Ivan Flis
Keyword(s):  
Hypothesis Testing ◽  
Null Hypothesis ◽  
Statistical Significance ◽  
Psychology Students ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Graduate Studies ◽  
Null Hypothesis Testing ◽  
Level Of Understanding ◽  
Grade Average

The goal of the study was to descriptively analyze the understanding of null hypothesis significance testing among Croatian psychology students considering how it is usually understood in textbooks, which is subject to Bayesian and interpretative criticism. Also, the thesis represents a short overview of the discussions on the meaning of significance testing and how it is taught to students. There were 350 participants from undergraduate and graduate programs at five faculties in Croatia (Zagreb – Centre for Croatian Studies and Faculty of Humanities and Social Sciences, Rijeka, Zadar, Osijek). Another goal was to ascertain if the understanding of null hypothesis testing among psychology students can be predicted by their grades, attitudes and interests. The level of understanding of null hypothesis testing was measured by the Test of statistical significance misinterpretations (NHST test) (Oakes, 1986; Haller and Krauss, 2002). The attitudes toward null hypothesis significance testing were measured by a questionnaire that was constructed for this study. The grades were operationalized as the grade average of courses taken during undergraduate studies, and as a separate grade average of methodological courses taken during undergraduate and graduate studies. The students have shown limited understanding of null hypothesis testing – the percentage of correct answers in the NHST test was not higher than 56% for any of the six items. Croatian students have also shown less understanding on each item when compared to the German students in Haller and Krauss’s (2002) study. None of the variables – general grade average, average in the methodological courses, two variables measuring the attitude toward null hypothesis significance testing, failing at least one methodological course, and the variable of main interest in psychology – were predictive for the odds of answering the items in the NHST test correctly. The conclusion of the study is that education practices in teaching students the meaning and interpretation of null hypothesis significance testing have to be taken under consideration at Croatian psychology departments.

scholarly journals Statistical Significance Testing at CHI PLAY: Challenges and Opportunities for More Transparency

2020 ◽  
Author(s):  
Jan Benjamin Vornhagen ◽  
April Tyack ◽  
Elisa D Mekler
Keyword(s):  
Statistical Tests ◽  
Statistical Significance ◽  
Reporting Quality ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
Questionable Research Practices ◽  
Reporting Practices ◽  
Challenges And Opportunities ◽  
Statistical Hypotheses

Statistical Significance Testing -- or Null Hypothesis Significance Testing (NHST) -- is common to quantitative CHI PLAY research. Drawing from recent work in HCI and psychology promoting transparent statistics and the reduction of questionable research practices, we systematically review the reporting quality of 119 CHI PLAY papers using NHST (data and analysis plan at https://osf.io/4mcbn/. We find that over half of these papers employ NHST without specific statistical hypotheses or research questions, which may risk the proliferation of false positive findings. Moreover, we observe inconsistencies in the reporting of sample sizes and statistical tests. These issues reflect fundamental incompatibilities between NHST and the frequently exploratory work common to CHI PLAY. We discuss the complementary roles of exploratory and confirmatory research, and provide a template for more transparent research and reporting practices.

scholarly journals Against statistical significance testing in corpus linguistics

2019 ◽  
Vol 15 (2) ◽  
pp. 321-346 ◽  
Author(s):  
Alexander Koplenig
Keyword(s):  
Corpus Linguistics ◽  
Null Hypothesis ◽  
Statistical Significance ◽  
Linguistic Theory ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Statistical Significance Testing ◽  
Naturally Occurring ◽  
Language Data

Abstract In the first volume of Corpus Linguistics and Linguistic Theory, Gries (2005. Null-hypothesis significance testing of word frequencies: A follow-up on Kilgarriff. Corpus Linguistics and Linguistic Theory 1(2). doi:10.1515/cllt.2005.1.2.277. http://www.degruyter.com/view/j/cllt.2005.1.issue-2/cllt.2005.1.2.277/cllt.2005.1.2.277.xml: 285) asked whether corpus linguists should abandon null-hypothesis significance testing. In this paper, I want to revive this discussion by defending the argument that the assumptions that allow inferences about a given population – in this case about the studied languages – based on results observed in a sample – in this case a collection of naturally occurring language data – are not fulfilled. As a consequence, corpus linguists should indeed abandon null-hypothesis significance testing.

scholarly journals From Statistical Significance to Effect Estimation: Statistical Reform in Psychology, Medicine and Ecology

2018 ◽  
Author(s):  
Fiona Fidler
Keyword(s):  
Null Hypothesis ◽  
Statistical Significance ◽  
Life Sciences ◽  
Biology Education ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
Scientific Disciplines ◽  
The Social ◽  
Effect Estimation ◽  
Alternative Practices

Compelling criticisms of statistical significance testing (or Null Hypothesis Significance Testing, NHST) can be found in virtually all areas of the social and life sciences—including economics, sociology, ecology, biology, education and psychology. Because it is the overwhelmingly dominant statistical method in these sciences, criticisms need to be taken seriously. Yet, after half a century of cogent arguments against NHST and calls to adopt alternative practices some disciplines show little sign of change. One obvious question is ‘why?’ Why are researchers so unwilling to abandon this flawed practice? In this thesis I attempt to answer this question, and compare practice across scientific disciplines.

Sign in / Sign up

Export Citation Format

Share Document