Proper experimental design and implementation are necessary conditions for a balanced social psychology

2004 ◽  
Vol 27 (3) ◽  
pp. 352-353
Author(s):  
Andreas Ortmann ◽  
Michal Ostatnicky
Keyword(s):  
Social Psychology ◽  
Decision Making ◽  
Experimental Design ◽  
Bayesian Analysis ◽  
Null Hypothesis ◽  
Necessary Conditions ◽  
Judgment And Decision Making ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Design And Implementation

We applaud the authors' basic message. We note that the negative research emphasis is not special solely to social psychology and judgment and decision-making. We argue that the proposed integration of null hypothesis significance testing (NHST) and Bayesian analysis is promising but will ultimately succeed only if more attention is paid to proper experimental design and implementation.

scholarly journals A Bayesian Analysis of Evidence in Support of the Null Hypothesis in Gerontological Psychology (or Lack Thereof)

2019 ◽  
Author(s):  
Christopher Brydges
Keyword(s):  
Bayesian Analysis ◽  
Strong Evidence ◽  
Null Hypothesis ◽  
Statistical Technique ◽  
Bayes Factors ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Bayesian Analyses ◽  
P Values ◽  
Gerontological Research

Objective: Non-significant p values derived from null hypothesis significance testing do not distinguish between true null effects or cases where the data are insensitive in distinguishing the hypotheses. This study aimed to investigate the prevalence of Bayesian analyses in gerontological psychology, a statistical technique that can distinguish between conclusive and inconclusive non-significant results, by using Bayes factors (BFs) to reanalyze non-significant results from published gerontological research.Method: Non-significant results mentioned in abstracts of articles published in 2017 volumes of ten top gerontological psychology journals were extracted (N = 409) and categorized based on whether Bayesian analyses were conducted. BFs were calculated from non-significant t-tests within this sample to determine how frequently the null hypothesis was strongly supported.Results: Non-significant results were directly tested with Bayes factors in 1.22% of studies. Bayesian reanalyses of 195 non-significant ¬t-tests found that only 7.69% of the findings provided strong evidence in support of the null hypothesis.Conclusions: Bayesian analyses are rarely used in gerontological research, and a large proportion of null findings were deemed inconclusive when reanalyzed with BFs. Researchers are encouraged to use BFs to test the validity of non-significant results, and ensure that sufficient sample sizes are used so that the meaningfulness of null findings can be evaluated.

Is social psychological research really so negatively biased?

2004 ◽  
Vol 27 (3) ◽  
pp. 340-341 ◽  
Author(s):  
Aiden P. Gregg ◽  
Constantine Sedikides
Keyword(s):  
Social Psychology ◽  
Null Hypothesis ◽  
Psychological Research ◽  
Negativity Bias ◽  
Significance Testing ◽  
Social Psychological ◽  
Null Hypothesis Significance Testing ◽  
Mental Abilities ◽  
Social Psychological Research

Krueger & Funder (K&F) overstate the defects of Null Hypothesis Significance Testing (NHST), and with it the magnitude of negativity bias within social psychology. We argue that replication matters more than NHST, that the pitfalls of NHST are not always or necessarily realized, and that not all biases are harmless offshoots of adaptive mental abilities.

scholarly journals Null Hypothesis Significance Testing: A Brief Review

2021 ◽  
Author(s):  
Xuejun Ryan Ji
Keyword(s):  
Social Psychology ◽  
Null Hypothesis ◽  
Quantitative Data ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Interval Estimates ◽  
Applied Social Psychology ◽  
Quantitative Data Analysis ◽  
Definition Of ◽  
Education Psychology

Null hypothesis significance testing (NHST) dominates the interpretation of quantitative data analysis in education, psychology, and other social science fields (Shaver, 1993). Meanwhile, the use of NHST has been under enduring and intense criticisms (Carver, 1978; Cohen, 1997; Cumming, 2013; Thompson, 1993, 1996, 1999). In 2015, the journal, Basic and Applied Social Psychology (BASP; Trafimow & Marks, 2015) banned the use of NHST, reigniting another round of intense discussions about whether continue using the NHST technique. In the present paper, I have elaborated the definition of NHST and six most commonmisinterpretations/false beliefs, and suggested reporting strategies, including reporting effect size along with its interval estimates. Finally, I briefly commented on the causes of misconceptions

scholarly journals A Bayesian Analysis of Evidence in Support of the Null Hypothesis in Gerontological Psychology (or Lack Thereof)

2019 ◽  
Vol 75 (1) ◽  
pp. 58-66 ◽  
Author(s):  
Christopher R Brydges ◽  
Allison A M Bielak
Keyword(s):  
Bayesian Analysis ◽  
Strong Evidence ◽  
Null Hypothesis ◽  
Statistical Technique ◽  
Bayes Factors ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Bayesian Analyses ◽  
P Values ◽  
Gerontological Research

Abstract Objectives Nonsignificant p values derived from null hypothesis significance testing do not distinguish between true null effects or cases where the data are insensitive in distinguishing the hypotheses. This study aimed to investigate the prevalence of Bayesian analyses in gerontological psychology, a statistical technique that can distinguish between conclusive and inconclusive nonsignificant results, by using Bayes factors (BFs) to reanalyze nonsignificant results from published gerontological research. Methods Nonsignificant results mentioned in abstracts of articles published in 2017 volumes of 10 top gerontological psychology journals were extracted (N = 409) and categorized based on whether Bayesian analyses were conducted. BFs were calculated from nonsignificant t-tests within this sample to determine how frequently the null hypothesis was strongly supported. Results Nonsignificant results were directly tested with BFs in 1.22% of studies. Bayesian reanalyses of 195 nonsignificant t-tests found that only 7.69% of the findings provided strong evidence in support of the null hypothesis. Conclusions Bayesian analyses are rarely used in gerontological research, and a large proportion of null findings were deemed inconclusive when reanalyzed with BFs. Researchers are encouraged to use BFs to test the validity of nonsignificant results and ensure that sufficient sample sizes are used so that the meaningfulness of null findings can be evaluated.

The Effects of Company Image and Communication Platform Alignment on Investor Information Processing

2021 ◽  
Author(s):  
Ryan D. Guggenmos ◽  
G. Bradley Bennett
Keyword(s):  
Decision Making ◽  
New Media ◽  
Bayesian Methods ◽  
Null Hypothesis ◽  
Judgment And Decision Making ◽  
Media Technology ◽  
Null Hypothesis Significance Testing ◽  
Media Communication ◽  
Communication Platform ◽  
Expérience Subjective

Motivated by firms' increasing use of new media technology for investor communications, we investigate how alignment between company image and communication platform affects investor judgment and decision making. In our first experiment, we demonstrate that investors expect alignment between firm image and the perception of the new media communication platform managers choose for investor relations. In a second experiment, we examine how this alignment affects investor judgment and decision-making. We predict and find that greater platform-image alignment leads investors to experience subjective ease of processing, but does not change investment amounts. Additionally, we demonstrate an approach to conducting an explicit test of a null hypothesis by evaluating the convergence of null hypothesis significance testing (NHST) and Bayesian methods. Our findings have implications for researchers, firms, and investors and add to a growing literature on new media disclosure.

scholarly journals A Primer on Bayesian Analysis for Experimental Psychopathologists

2017 ◽  
Vol 8 (2) ◽  
pp. 140-157 ◽  
Author(s):  
Angelos-Miltiadis Krypotos ◽  
Tessa F. Blanken ◽  
Inna Arnaudova ◽  
Dora Matzke ◽  
Tom Beckers
Keyword(s):  
Bayesian Analysis ◽  
Bayesian Statistics ◽  
Present Article ◽  
Null Hypothesis ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Data Set ◽  
Experimental Psychopathology ◽  
Continued Use ◽  
Main Message

The principal goals of experimental psychopathology (EPP) research are to offer insights into the pathogenic mechanisms of mental disorders and to provide a stable ground for the development of clinical interventions. The main message of the present article is that those goals are better served by the adoption of Bayesian statistics than by the continued use of null-hypothesis significance testing (NHST). In the first part of the article we list the main disadvantages of NHST and explain why those disadvantages limit the conclusions that can be drawn from EPP research. Next, we highlight the advantages of Bayesian statistics. To illustrate, we then pit NHST and Bayesian analysis against each other using an experimental data set from our lab. Finally, we discuss some challenges when adopting Bayesian statistics. We hope that the present article will encourage experimental psychopathologists to embrace Bayesian statistics, which could strengthen the conclusions drawn from EPP research.

scholarly journals A Frequentist Alternative to Significance Testing, p-Values, and Confidence Intervals

Econometrics ◽  
2019 ◽  
Vol 7 (2) ◽  
pp. 26 ◽  
Author(s):  
David Trafimow
Keyword(s):  
Present Article ◽  
Confidence Intervals ◽  
Null Hypothesis ◽  
A Priori ◽  
Significance Testing ◽  
Population Parameters ◽  
Null Hypothesis Significance Testing ◽  
P Values ◽  
Statistical Procedures ◽  
Major Section

There has been much debate about null hypothesis significance testing, p-values without null hypothesis significance testing, and confidence intervals. The first major section of the present article addresses some of the main reasons these procedures are problematic. The conclusion is that none of them are satisfactory. However, there is a new procedure, termed the a priori procedure (APP), that validly aids researchers in obtaining sample statistics that have acceptable probabilities of being close to their corresponding population parameters. The second major section provides a description and review of APP advances. Not only does the APP avoid the problems that plague other inferential statistical procedures, but it is easy to perform too. Although the APP can be performed in conjunction with other procedures, the present recommendation is that it be used alone.

The Numbers Will Love You Back in Return—I Promise

2016 ◽  
Vol 11 (4) ◽  
pp. 551-554 ◽  
Author(s):  
Martin Buchheit
Keyword(s):  
Sample Size ◽  
Null Hypothesis ◽  
Clinical Medicine ◽  
Statistical Significance ◽  
Significance Testing ◽  
Null Hypothesis Significance Testing ◽  
Sport Science ◽  
Size Dependent ◽  
Research Questions ◽  
Per Se

The first sport-science-oriented and comprehensive paper on magnitude-based inferences (MBI) was published 10 y ago in the first issue of this journal. While debate continues, MBI is today well established in sport science and in other fields, particularly clinical medicine, where practical/clinical significance often takes priority over statistical significance. In this commentary, some reasons why both academics and sport scientists should abandon null-hypothesis significance testing and embrace MBI are reviewed. Apparent limitations and future areas of research are also discussed. The following arguments are presented: P values and, in turn, study conclusions are sample-size dependent, irrespective of the size of the effect; significance does not inform on magnitude of effects, yet magnitude is what matters the most; MBI allows authors to be honest with their sample size and better acknowledge trivial effects; the examination of magnitudes per se helps provide better research questions; MBI can be applied to assess changes in individuals; MBI improves data visualization; and MBI is supported by spreadsheets freely available on the Internet. Finally, recommendations to define the smallest important effect and improve the presentation of standardized effects are presented.

Роль показника розміру ефекту в сучасних психологічних дослідженнях

2021 ◽  
Author(s):  
Валерій Боснюк
Keyword(s):  
Null Hypothesis ◽  
Significance Testing ◽  
P Value ◽  
Null Hypothesis Significance Testing

Для підтвердження результатів дослідження в психологічних наукових роботах протягом багатьох років використовується процедура перевірки значущості нульової гіпотези (загальноприйнята абревіатура NHST – Null Hypothesis Significance Testing) із застосуванням спеціальних статистичних критеріїв. При цьому здебільшого значення статистики «p» (p-value) розглядається як еквівалент важливості отриманих результатів і сили наукових доказів на користь практичного й теоретичного ефекту дослідження. Таке некоректне використання та інтерпретації p-value ставить під сумнів застосування статистики взагалі та загрожує розвитку психології як науки. Ототожнення статистичного висновку з науковим висновком, орієнтація виключно на новизну в наукових дослідженнях, ритуальна прихильність дослідників до рівня значущості 0,05, опора на статистичну категоричність «так/ні» під час прийняття рішення призводить до того, що психологія примножує тільки результати про наявність ефекту без врахування його величини, практичної цінності. Дана робота призначена для аналізу обмеженості p-value при інтерпретації результатів психологічних досліджень та переваг представлення інформації про розмір ефекту. Застосування розмірів ефекту дозволить здійснити перехід від дихотомічного мислення до оціночного, визначати цінність результатів незалежно від рівня статистичної значущості, приймати рішення більш раціонально та обґрунтовано. Обґрунтовується позиція, що автор наукової роботи при формулюванні висновків дослідження не повинен обмежуватися одним єдиним показником рівня статистичної значущості. Осмислені висновки повинні базуватися на розумному балансуванні p-value та інших не менш важливих параметрів, одним з яких виступає розмір ефекту. Ефект (відмінність, зв’язок, асоціація) може бути статистично значущим, а його практична (клінічна) цінність – незначною, тривіальною. «Статистично значущий» не означає «корисний», «важливий», «цінний», «значний». Тому звернення уваги психологів до питання аналізу виявленого розміру ефекту має стати обов’язковим при інтерпретації результатів дослідження.

Sign in / Sign up

Export Citation Format

Share Document