scholarly journals Quantifying Support for the Null Hypothesis in Psychology: An Empirical Investigation

2017 ◽  
Author(s):  
Balazs Aczel ◽  
Bence Palfi ◽  
Aba Szollosi ◽  
Marton Kovacs ◽  
Szaszi Barnabas ◽  
...  
Keyword(s):  
Strong Evidence ◽  
Experimental Psychology ◽  
Null Hypothesis ◽  
Bayes Factor ◽  
Empirical Investigation ◽  
Alternative Hypothesis ◽  
Review Journal ◽  
Psychonomic Bulletin ◽  
Psychological Science ◽  
Statistical Framework

In the traditional statistical framework, nonsignificant results leave researchers in a state of suspended disbelief. This study examines, empirically, the treatment and evidential impact of nonsignificant results. Our specific goals were twofold: to explore how psychologists interpret and communicate nonsignificant results, and to assess how much these results constitute evidence in favor of the null hypothesis. Firstly, we examined all nonsignificant findings mentioned in the abstracts of the 2015 volume of Psychonomic Bulletin & Review, Journal of Experimental Psychology: General, and Psychological Science (N = 137). In 72% of cases, nonsignificant results were misinterpreted, in the sense that authors inferred that the effect was absent. Secondly, a Bayes factor reanalysis revealed that fewer than 5% of the nonsignificant findings provided strong evidence (i.e., BF01 > 10) in favor of the null hypothesis compared to the alternative hypothesis. We recommend that researchers expand their statistical toolkit in order to correctly interpret nonsignificant results and to be able to evaluate the evidence for and against the null hypothesis.

scholarly journals Quantifying Support for the Null Hypothesis in Psychology: An Empirical Investigation

2018 ◽  
Vol 1 (3) ◽  
pp. 357-366 ◽  
Author(s):  
Balazs Aczel ◽  
Bence Palfi ◽  
Aba Szollosi ◽  
Marton Kovacs ◽  
Barnabas Szaszi ◽  
...  
Keyword(s):  
Strong Evidence ◽  
Null Hypothesis ◽  
Bayes Factor ◽  
Empirical Investigation ◽  
Alternative Hypothesis ◽  
Review Journal ◽  
Psychonomic Bulletin ◽  
Statistical Tool ◽  
Psychological Science ◽  
Statistical Framework

In the traditional statistical framework, nonsignificant results leave researchers in a state of suspended disbelief. In this study, we examined, empirically, the treatment and evidential impact of nonsignificant results. Our specific goals were twofold: to explore how psychologists interpret and communicate nonsignificant results and to assess how much these results constitute evidence in favor of the null hypothesis. First, we examined all nonsignificant findings mentioned in the abstracts of the 2015 volumes of Psychonomic Bulletin & Review, Journal of Experimental Psychology: General, and Psychological Science ( N = 137). In 72% of these cases, nonsignificant results were misinterpreted, in that the authors inferred that the effect was absent. Second, a Bayes factor reanalysis revealed that fewer than 5% of the nonsignificant findings provided strong evidence (i.e., BF01 > 10) in favor of the null hypothesis over the alternative hypothesis. We recommend that researchers expand their statistical tool kit in order to correctly interpret nonsignificant results and to be able to evaluate the evidence for and against the null hypothesis.

scholarly journals A Primer on Bayesian Model-Averaged Meta-Analysis

2020 ◽  
Author(s):  
Quentin Frederik Gronau ◽  
Daniel W. Heck ◽  
Sophie Wilhelmina Berkhout ◽  
Julia M. Haaf ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Random Effects ◽  
Null Hypothesis ◽  
Bayesian Model ◽  
Alternative Hypothesis ◽  
Meta Analysis ◽  
Fixed Effect ◽  
Random Effects Model ◽  
Psychological Science ◽  
Analysis Models ◽  
Key Questions

Meta-analysis is the predominant approach for quantitatively synthesizing a set of studies. If the studies themselves are of high quality, meta-analysis can provide valuable insights into the current scientific state of knowledge about a particular phenomenon. In psychological science, the most common approach is to conduct frequentist meta-analysis. In this primer, we discuss an alternative method, Bayesian model-averaged meta-analysis. This procedure combines the results of four Bayesian meta-analysis models: (1) fixed-effect null hypothesis, (2) fixed-effect alternative hypothesis, (3) random-effects null hypothesis, and (4) random-effects alternative hypothesis. These models are combined according to their plausibilities in light of the observed data to address the two key questions "Is the overall effect non-zero?" and "Is there between-study variability in effect size?". Bayesian model-averaged meta-analysis therefore avoids the need to select either a fixed-effect or random-effects model and instead takes into account model uncertainty in a principled manner.

scholarly journals A Bayesian bird's eye view of ‘Replications of important results in social psychology’

2017 ◽  
Vol 4 (1) ◽  
pp. 160426 ◽  
Author(s):  
Maarten Marsman ◽  
Felix D. Schönbrodt ◽  
Richard D. Morey ◽  
Yuling Yao ◽  
Andrew Gelman ◽  
...  
Keyword(s):  
Social Psychology ◽  
Effect Size ◽  
Null Hypothesis ◽  
Bayes Factor ◽  
Alternative Hypothesis ◽  
Hypothesis Test ◽  
Meta Analysis ◽  
Bayes Factors ◽  
Credible Intervals ◽  
Complementary Techniques

We applied three Bayesian methods to reanalyse the preregistered contributions to the Social Psychology special issue ‘Replications of Important Results in Social Psychology’ (Nosek & Lakens. 2014 Registered reports: a method to increase the credibility of published results. Soc. Psychol. 45 , 137–141. ( doi:10.1027/1864-9335/a000192 )). First, individual-experiment Bayesian parameter estimation revealed that for directed effect size measures, only three out of 44 central 95% credible intervals did not overlap with zero and fell in the expected direction. For undirected effect size measures, only four out of 59 credible intervals contained values greater than 0.10 (10% of variance explained) and only 19 intervals contained values larger than 0.05 . Second, a Bayesian random-effects meta-analysis for all 38 t -tests showed that only one out of the 38 hierarchically estimated credible intervals did not overlap with zero and fell in the expected direction. Third, a Bayes factor hypothesis test was used to quantify the evidence for the null hypothesis against a default one-sided alternative. Only seven out of 60 Bayes factors indicated non-anecdotal support in favour of the alternative hypothesis ( BF 10 > 3 ), whereas 51 Bayes factors indicated at least some support for the null hypothesis. We hope that future analyses of replication success will embrace a more inclusive statistical approach by adopting a wider range of complementary techniques.

scholarly journals Does mental fatigue impair physical performance? A replication study

2019 ◽  
Author(s):  
Darias Holgado ◽  
Esther Troya ◽  
José C. Perales ◽  
Miguel A. Vadillo ◽  
Daniel Sanabria
Keyword(s):  
Heart Rate ◽  
Physical Performance ◽  
Null Hypothesis ◽  
Bayes Factor ◽  
Alternative Hypothesis ◽  
Negative Influence ◽  
Sport Performance ◽  
Mental Fatigue ◽  
Time To Exhaustion ◽  
Subject Experiment

Objective: to replicate the hypothesis that mental fatigue impairs physical performance. Design: a pre-registered (https://osf.io/wqkap/), randomized, within-subject experiment. Methods: 30 recreationally active adults completed a time-to-exhaustion test (TTE) at 80% VO2max in two separate sessions, after completing a mental fatigue task or watching a documen-tary for 90 min. We measured power output, heart rate, RPE and subjective mental fatigue state. Results: Bayes factor analyses revealed extreme evidence supporting the alternative hypothesis that the mental fatigue task was more mentally fatiguing than the control task, BF01 = 0.009. However, we found moderate-to-strong evidence for the null hypothesis (i.e., no evidence of reduced performance) for average time in TTE (BF01 = 9.762) and anecdotal evidence for the null hypothesis in RPE (BF01 = 2.902) and heart rate (BF01 = 2.587). Conclusions: our data seem to challenge the idea that mental fatigue has a negative influence on exercise performance. Although we did succeed at manipulating subjective mental fatigue, this did not impair physical performance. However, we cannot discard the possibility that mental fatigue may have a negative influence under conditions not explored here, e.g., individualizing mentally fatiguing tasks. In sum, further research is warranted to determine the role of mental fatigue on exercise and sport performance.

scholarly journals Why optional stopping can be a problem for Bayesians

2020 ◽  
Author(s):  
Rianne de Heide ◽  
Peter D. Grünwald
Keyword(s):  
Bayesian Methods ◽  
Bayes Factor ◽  
Type Ii ◽  
Psychonomic Bulletin ◽  
Practical Applications ◽  
Psychological Science ◽  
Optional Stopping ◽  
Psychology Community ◽  
Severe Type ◽  
Parameters Of Interest

AbstractRecently, optional stopping has been a subject of debate in the Bayesian psychology community. Rouder (Psychonomic Bulletin & Review21(2), 301–308, 2014) argues that optional stopping is no problem for Bayesians, and even recommends the use of optional stopping in practice, as do (Wagenmakers, Wetzels, Borsboom, van der Maas & Kievit, Perspectives on Psychological Science7, 627–633, 2012). This article addresses the question of whether optional stopping is problematic for Bayesian methods, and specifies under which circumstances and in which sense it is and is not. By slightly varying and extending Rouder’s (Psychonomic Bulletin & Review21(2), 301–308, 2014) experiments, we illustrate that, as soon as the parameters of interest are equipped with default or pragmatic priors—which means, in most practical applications of Bayes factor hypothesis testing—resilience to optional stopping can break down. We distinguish between three types of default priors, each having their own specific issues with optional stopping, ranging from no-problem-at-all (type 0 priors) to quite severe (type II priors).

scholarly journals A Primer on Bayesian Model-Averaged Meta-Analysis

2021 ◽  
Vol 4 (3) ◽  
pp. 251524592110312
Author(s):  
Quentin F. Gronau ◽  
Daniel W. Heck ◽  
Sophie W. Berkhout ◽  
Julia M. Haaf ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Random Effects ◽  
Null Hypothesis ◽  
Bayesian Model ◽  
Alternative Hypothesis ◽  
Meta Analysis ◽  
Fixed Effect ◽  
Random Effects Model ◽  
Psychological Science ◽  
Analysis Models ◽  
Key Questions

Meta-analysis is the predominant approach for quantitatively synthesizing a set of studies. If the studies themselves are of high quality, meta-analysis can provide valuable insights into the current scientific state of knowledge about a particular phenomenon. In psychological science, the most common approach is to conduct frequentist meta-analysis. In this primer, we discuss an alternative method, Bayesian model-averaged meta-analysis. This procedure combines the results of four Bayesian meta-analysis models: (a) fixed-effect null hypothesis, (b) fixed-effect alternative hypothesis, (c) random-effects null hypothesis, and (d) random-effects alternative hypothesis. These models are combined according to their plausibilities given the observed data to address the two key questions “Is the overall effect nonzero?” and “Is there between-study variability in effect size?” Bayesian model-averaged meta-analysis therefore avoids the need to select either a fixed-effect or random-effects model and instead takes into account model uncertainty in a principled manner.

The Effect of Applying POSSE (Predict- Organize-Search-Summarize-Evaluate) on the Students’ Reading Comprehension

2013 ◽  
Vol 2 (1) ◽  
Author(s):  
Eka Rejeki Maha ◽  
Berlin Sibarani
Keyword(s):  
Reading Comprehension ◽  
Experimental Design ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Choice Test ◽  
Control Group ◽  
Multiple Choice Test ◽  
Level Of Significance ◽  
And Control ◽  
Experimental Group

This study was aimed at finding out The Effect of Applying POSSE (Predict-Organize-Search-Summarize-Evaluate) Strategy on the Students’ Reading Comprehension. This study used the experimental design. The population of this study was the students of SMA Negeri 2 Medan. There were sixty students taken as the sample of the research. This study was conducted with two randomized groups namely experimental and control group. The experimental group was taught by applying POSSE strategy while control group was taught without applying POSSE strategy. The instrument of collecting the data was multiple choice test which consists of 40 items. To obtain the reliability of the test, the researcher used Kuder-Richardson (KR-21). The calculation shows the reliability was 0.81(high). The data were analyzed by using t-test. The calculation shows that t-observed (4.76) is higher than t-table (2.00) at the level of significance (α) 0.05 with the degree of freedom (df) 58. Therefore, the null hypothesis (Ho) is rejected and alternative hypothesis (Ha) is accepted. It means that there was a significant effect of applying POSSE strategy on the students’ reading comprehension. Keywords: POSSE Strategy, Reading Comprehension.

THE EFFECT OF USING NOTING INTERACTING SUMMARIZING AND PRIORITIZING STRATEGY ON STUDENTS’ ACHIEVEMENT IN READING COMPREHENSION

2013 ◽  
Vol 2 (4) ◽  
Author(s):  
Maruh Sianturi And Berlin Sibarani
Keyword(s):  
Reading Comprehension ◽  
Experimental Design ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Control Group ◽  
First Year ◽  
Teaching Reading ◽  
First Year Students ◽  
Level Of Significance ◽  
Experimental Group

This study was aimed at finding out the effect of using Noting, Interacting, Summarizing, and Prioritizing Strategy on Students’ Achievement in Reading Comprehension. This study was designed with the experimental design. The population of this study was the first year students at academic 2013/2012 of SMA swasta YP St. Paulus Martubung, Medan. There were fourty students taken as the sample of the research. The sample was divided into two groups: the first group (20 students) as the experimental group and the second group (20 students) as the control group. The experimental group was taught by Using Noting, Interacting, Summarizing, and Prioritizing Strategy, while the control group was taught by using conventional method. The instrument for collecting the data was multiple choices which consisted of 40 items. To obtain the reliability of the test, the researcher used Kuder -Richardson (KR-21) formula. The calculation showed that the reliability of the test was 0.75. The data were calculated by using t-test formula. The result of the analysis shows that t-observed (4.98) was higher than t-table (2.025) at the level of significance (α) 0.05 and the degree of freedom (df) 38. Therefore, the null hypothesis (H0) was rejected and alternative hypothesis (Ha) was accepted. It meant that teaching reading comprehension by using Noting, Interacting, Summarizing, and Prioritizing Strategy significantly affects reading comprehension.

Suggestopedia method as an effective method in teaching reading comprehension for senior high school

2020 ◽  
Vol 3 (1) ◽  
pp. 98-101
Author(s):  
Deavy Safitri Murfita ◽  
Dian Novita
Keyword(s):  
Reading Comprehension ◽  
Data Analysis ◽  
Null Hypothesis ◽  
Quantitative Research ◽  
Alternative Hypothesis ◽  
Teaching Reading ◽  
Program Calculation ◽  
Post Test ◽  
Main Activity ◽  
Test Treatment

This research is to know is there is significant effect in reading comprehension achievement in X grade at SMK Walisongo 1 Gempol, being taught by suggestopedia method. This research uses quantitative research. The research’s object is X AK-2 of SMK Walisongo 1 Gempol has 28 students. The researcher divides the main activity into pre-test, treatment and post-test. Based on the SPSS 23 Program calculation from mean result of pre-test 70.1 and the result mean of post-test is 83.5. The students’ score is increasing after being given by the treatment. The result of the calculation of this research showed that (sig 2-tailed = 0.002) which is less from (<) 0.05 and significance 5%. It mean that the alternative hypothesis is accepted and the null hypothesis is rejected. From the result of the data analysis above, the use suggestopedia method gives significant effect in students’ reading comprehension achievement This article investigates major points of the speech verbs. On this case, different meaning of verbs was analyzed from Russian into English.  Therefore, analyses of the theory with methodology were described to make better diffusion. To conclude with both outcomes and shortcomings were outlined to get further analyses as the whole.

Hypothesis testing with error correction models

2021 ◽  
pp. 1-9
Author(s):  
Patrick W. Kraft ◽  
Ellen M. Key ◽  
Matthew J. Lebo
Keyword(s):  
Hypothesis Testing ◽  
Error Correction ◽  
Null Hypothesis ◽  
Alternative Hypothesis ◽  
Correction Coefficient ◽  
Correction Model ◽  
Long Run ◽  
Independent Variable ◽  
The Right ◽  
General Error

Abstract Grant and Lebo (2016) and Keele et al. (2016) clarify the conditions under which the popular general error correction model (GECM) can be used and interpreted easily: In a bivariate GECM the data must be integrated in order to rely on the error correction coefficient, $\alpha _1^\ast$ , to test cointegration and measure the rate of error correction between a single exogenous x and a dependent variable, y. Here we demonstrate that even if the data are all integrated, the test on $\alpha _1^\ast$ is misunderstood when there is more than a single independent variable. The null hypothesis is that there is no cointegration between y and any x but the correct alternative hypothesis is that y is cointegrated with at least one—but not necessarily more than one—of the x's. A significant $\alpha _1^\ast$ can occur when some I(1) regressors are not cointegrated and the equation is not balanced. Thus, the correct limiting distributions of the right-hand-side long-run coefficients may be unknown. We use simulations to demonstrate the problem and then discuss implications for applied examples.

Sign in / Sign up

Export Citation Format

Share Document