scholarly journals Why most psychological research findings are not even wrong

2021 ◽  
Author(s):  
Anne M. Scheel

Psychology’s replication crisis is typically conceptualised as the insight that the published literature contains a worrying amount of unreplicable, false-positive findings. At the same time, meta-scientific attempts to assess the crisis in more detail have reported substantial difficulties to identify unambiguous definitions of the scientific claims in published articles and to determine how they are connected to the presented evidence. I argue that most claims in the literature are so critically underspecified that attempts to empirically evaluate them are doomed to failure — they are not even wrong. Meta-scientists should beware of the flawed assumption that the psychological literature is a collection of well-defined claims. To move beyond the crisis, psychologists must reconsider and rebuild the conceptual basis of their hypotheses before trying to test them.

2021 ◽  
Author(s):  
Michele B. Nuijten

Increasing evidence indicates that many published findings in psychology may be overestimated or even false. An often-heard response to this “replication crisis” is to replicate more: replication studies should weed out false positives over time and increase robustness of psychological science. However, replications take time and money – resources that are often scarce. In this chapter, I propose an efficient alternative strategy: a four-step robustness check that first focuses on verifying reported numbers through reanalysis before replicating studies in a new sample.


2016 ◽  
Vol 18 (2) ◽  
pp. 155-182 ◽  
Author(s):  
Hannah Proctor

Alexander Luria played a prominent role in the psychoanalytic community that flourished briefly in Soviet Russia in the decade following the 1917 October Revolution. In 1925 he co-wrote an introduction to Sigmund Freud's Beyond the Pleasure Principle with Lev Vygotsky, which argued that the conservatism of the instincts that Freud described might be overcome through the kind of radical social transformation then taking place in Russia. In attempting to bypass the backward looking aspects of Freud's theory, however, Luria and Vygotsky also did away with the tension between Eros and the death drive; precisely the element of Freud's essay they praised for being ‘dialectical’. This article theoretically unpicks Luria and Vygotsky's critique of psychoanalysis. It concludes by considering their optimistic ideological argument against the death drive with Luria's contemporaneous psychological research findings, proposing that Freud's ostensibly conservative theory may not have been as antithetical to revolutionary goals as Luria and Vygotsky assumed.


2020 ◽  
pp. 1-37
Author(s):  
Tal Yarkoni

Abstract Most theories and hypotheses in psychology are verbal in nature, yet their evaluation overwhelmingly relies on inferential statistical procedures. The validity of the move from qualitative to quantitative analysis depends on the verbal and statistical expressions of a hypothesis being closely aligned—that is, that the two must refer to roughly the same set of hypothetical observations. Here I argue that many applications of statistical inference in psychology fail to meet this basic condition. Focusing on the most widely used class of model in psychology—the linear mixed model—I explore the consequences of failing to statistically operationalize verbal hypotheses in a way that respects researchers' actual generalization intentions. I demonstrate that whereas the "random effect" formalism is used pervasively in psychology to model inter-subject variability, few researchers accord the same treatment to other variables they clearly intend to generalize over (e.g., stimuli, tasks, or research sites). The under-specification of random effects imposes far stronger constraints on the generalizability of results than most researchers appreciate. Ignoring these constraints can dramatically inflate false positive rates, and often leads researchers to draw sweeping verbal generalizations that lack a meaningful connection to the statistical quantities they are putatively based on. I argue that failure to take the alignment between verbal and statistical expressions seriously lies at the heart of many of psychology's ongoing problems (e.g., the replication crisis), and conclude with a discussion of several potential avenues for improvement.


2012 ◽  
Vol 7 (6) ◽  
pp. 562-571 ◽  
Author(s):  
Roger Giner-Sorolla

The current crisis in psychological research involves issues of fraud, replication, publication bias, and false positive results. I argue that this crisis follows the failure of widely adopted solutions to psychology’s similar crisis of the 1970s. The untouched root cause is an information-economic one: Too many studies divided by too few publication outlets equals a bottleneck. Articles cannot pass through just by showing theoretical meaning and methodological rigor; their results must appear to support the hypothesis perfectly. Consequently, psychologists must master the art of presenting perfect-looking results just to survive in the profession. This favors aesthetic criteria of presentation in a way that harms science’s search for truth. Shallow standards of statistical perfection distort analyses and undermine the accuracy of cumulative data; narrative expectations encourage dishonesty about the relationship between results and hypotheses; criteria of novelty suppress replication attempts. Concerns about truth in research are emerging in other sciences and may eventually descend on our heads in the form of difficult and insensitive regulations. I suggest a more palatable solution: to open the bottleneck, putting structures in place to reward broader forms of information sharing beyond the exquisite art of present-day journal publication.


2003 ◽  
Vol 8 (3) ◽  
pp. 375-383 ◽  
Author(s):  
Rodolfo de Castro Ribas Jr. ◽  
Maria Lucia Seidl de Moura ◽  
Isabela Dias Soares ◽  
Alessandra Aparecida do Nascimento Gomes ◽  
Marc H. Bornstein

This review has several objectives: To describe and discuss theoretical conceptions of the construct of socioeconomic status (SES) and to argue for its vital role in psychological research; to present and analyze procedures employed to measure SES and trends in their utilization; and to review and discuss the use of SES measures in Brazilian psychological literature. The relative position of individuals, families, and groups in a given hierarchy (frequently converted into a score produced by a scale) is what has usually been called SES. The main indicators and procedures used to measure SES are discussed in regard to its advantages and disadvantages. A review of the literature offers evidence of the importance of the SES in different psychological processes. A systematic evaluation of articles from the PsycARTICLES database was conducted and revealed that the percentage of articles published annually that employed socioeconomic status increased steadily and substantially from 1988 through 2000 and that SES has been consistently applied more in some research areas (e.g., developmental, clinical, social psychology). A content analysis of the use of SES in articles published from 1981 through 2001 in three prominent Brazilian psychology journals was conducted showing that reliable SES measures are not commonly used in the Brazilian psychological literature. The results of these reviews and analyses are discussed in terms of their implications for further progress of psychological literature, especially in Brazil, with regard SES.


Psychology ◽  
2020 ◽  
Author(s):  
David C. Devonis

The Stanford Prison Experiment (SPE) took place at a time when the sources of authoritarianism and evil were a focal concern in psychology. It emerged from a tradition of activist social psychological research beginning with Solomon Asch in the 1940s and extending through Stanley Milgram’s obedience experiments in the early 1960s. The SPE was a product of the research program of social psychologist Philip Zimbardo, a member of the Stanford psychology faculty since 1968. Discussions among Zimbardo’s students in spring 1971 led to a plan to simulate a prison environment. They converted portions of the basement of a University building into a combination booking room and jail. Zimbardo and a number of his graduate and undergraduate students took on supervisory roles. Before the Experiment began, paid participants recruited through newspaper advertisements were screened to eliminate obvious psychopathology, then randomly assigned to either the role of ‘guard’ or ‘prisoner.’ On the first experimental morning August 14, 1971, actual local police simulated an arrest of each of the prisoner participants. After they arrived, blindfolded, a simulated booking took place. Guards escorted them to the prison hallway where prisoners were required to strip and exchange their clothing for simple shifts and slippers. After a simulated spray delousing, they entered makeshift cells. After this, the Experiment evolved as an extended improvisation, by both the guards and prisoners, on prison-related themes. Episodes of deprivation, bullying, and humiliation emerged unplanned. Originally planned to run for two weeks, the Experiment lasted only six days, prematurely terminated when its supervising personnel judged that the simulation had gotten out of their control. The coincidence of its termination with the Attica prison uprising in New York led to its immediate dissemination in the news. Since then the SPE has become one of the most iconic psychological studies of psychology’s modern era. Although intended to expose and ameliorate bad prison conditions, its effectiveness in this regard diminished during a rapid shift in US prison policy, in the mid-1970’s, from reform to repression. Over succeeding decades, the Experiment continued to stimulate the popular imagination, leading to an extensive replication on British television and its portrayal in two feature films. Soon after its original publication, the SPE attracted criticisms of its methodology. After 2010, critical scrutiny of the SPE as well as similar iconic studies from the 1960s and 1970s increased, fueled by the growing ‘replication crisis’ in psychology. This most recent phase of criticism reflects not just a turn toward reflexive disciplinary self-criticism but also the increased availability of archival sources for examination. The SPE continues to excite both passionate support and equally passionate obloquy, much as have other comparable simulations of human social behavior.


2019 ◽  
Vol 3 (Supplement_1) ◽  
pp. S400-S400
Author(s):  
Walter R Boot

Abstract Psychology and other sciences have been in the midst of a replication crisis. One proposal to address this crisis is the preregistration of studies, including study hypotheses, methods, measures, and analysis approaches to reduce false positive findings resulting from “experimenter degrees of freedom.” This talk will explore the benefits, and also the challenges, of preregistration, along with common misconceptions about preregistration. A preregistration case study will be presented involving a series of experiments exploring different hypotheses regarding the mechanism behind changes in attentional processing associated with aging (http://doi.org/10.1525/collabra.26). This talk will present a brief tutorial of how to preregister studies and where to preregister them. The importance of preregistration for intervention studies will be emphasized.


ruffin_darden ◽  
1998 ◽  
Vol 1 ◽  
pp. 149-172 ◽  
Author(s):  
David M. Messick ◽  

In this article, I want to draw attention to one strand ofthe complex web of processes that are involved when people group others, including themselves, into social categories. I will focus on the tendency to treat members of one's own group more favorably than nonmembers, a tendency that has been called ingroup favoritism. The structure of the article has three parts. First I will offer anevolutionary argument as to why ingroup favoritism, or something very much like it, is required by theories of the evolution of altruism. I will then review some of the basic social psychological research findings dealing with social categorization generally, and ingroup favoritism specifically. Finally, I will examine two problems in business ethics from the point of view of ingroup favoritism to suggest ways in which social psychological principles and findings may be mobilized to help solve problems of racial or gender discrimination in business contexts.


Sign in / Sign up

Export Citation Format

Share Document