The Replication Crisis and Open Science in Psychology – Progress and Yet Unsolved Problems

2018 ◽  
Vol 226 (3) ◽  
pp. 204-205 ◽  
Keyword(s):  
Open Science ◽  
Replication Crisis

scholarly journals Researcher Degrees of Freedom in the Psychology of Religion

2018 ◽  
Author(s):  
Sarah Jane Charles ◽  
James Edward Bartlett ◽  
Kyle J. Messick ◽  
Thomas Joseph Coleman ◽  
Alex Uzdavines
Keyword(s):  
Systematic Review ◽  
Degrees Of Freedom ◽  
Psychology Of Religion ◽  
Open Science ◽  
Science Practices ◽  
Replication Crisis ◽  
The Way

There is a push in psychology toward more transparent practices, stemming partially as a response to the replication crisis. We argue that the psychology of religion should help lead the way toward these new, more transparent practices to ensure a robust and dynamic subfield. One of the major issues that proponents of Open Science practices hope to address is researcher degrees of freedom (RDF). We pre-registered and conducted a systematic review of the 2017 issues from three psychology of religion journals. We aimed to identify the extent to which the psychology of religion has embraced Open Science practices and the role of RDF within the subfield. We found that many of the methodologies that help to increase transparency, such as pre-registration, have yet to be adopted by those in the subfield. In light of these findings, we present recommendations for addressing the issue of transparency in the psychology of religion and outline how to move toward these new Open Science practices.

scholarly journals The past, present, and future of ego depletion

2019 ◽  
Author(s):  
Michael Inzlicht ◽  
Malte Friese
Keyword(s):  
Social Psychology ◽  
Ego Depletion ◽  
Open Science ◽  
Science Practices ◽  
Current State ◽  
Scientific Status ◽  
Replication Crisis ◽  
History Of ◽  
Controversial Topic ◽  
Mere Existence

At the center of social psychology just a few years ago, ego depletion is now widely seen as a controversial topic, one of the chief victims of the replication crisis. Despite over 600 studies of apparent support, many are now asking if ego depletion is even real. Here, we comment on the articles included in this Special Issue: Ego Depletion. Specifically, we delineate the contributions and limitations of these articles by embedding them in a brief history of ego depletion, describing the current state of uncertainty about ego depletion’s scientific status, and outlining necessary steps for the study of ego depletion to have a healthy future. To us, the most troubling aspect of this controversy is not what is suggests about ego depletion; but what it suggests about social psychology more broadly. If the mere existence of ego depletion is seriously doubted by many, what can be confidently regarded as real in social psychology? By increasing the precision of our theories, continuously validating our manipulations and measures, and practicing the full suite of open science practices we have the potential to identify legitimate and robust effects and build a cumulative and trustworthy psychological science.

scholarly journals ‘Trust Us’: Open Data and Preregistration in Political Science and International Relations

2022 ◽  
Author(s):  
Bermond Scoggins ◽  
Matthew Peter Robertson
Keyword(s):  
International Relations ◽  
Political Science ◽  
Statistical Inference ◽  
Large Scale ◽  
Open Data ◽  
Population Level ◽  
Open Science ◽  
Science Practices ◽  
Level Data ◽  
Replication Crisis

The scientific method is predicated on transparency -- yet the pace at which transparent research practices are being adopted by the scientific community is slow. The replication crisis in psychology showed that published findings employing statistical inference are threatened by undetected errors, data manipulation, and data falsification. To mitigate these problems and bolster research credibility, open data and preregistration have increasingly been adopted in the natural and social sciences. While many political science and international relations journals have committed to implementing these reforms, the extent of open science practices is unknown. We bring large-scale text analysis and machine learning classifiers to bear on the question. Using population-level data -- 93,931 articles across the top 160 political science and IR journals between 2010 and 2021 -- we find that approximately 21% of all statistical inference papers have open data, and 5% of all experiments are preregistered. Despite this shortfall, the example of leading journals in the field shows that change is feasible and can be effected quickly.

scholarly journals Open Science and Educational Technology Research

2021 ◽  
Vol 37 (4) ◽  
pp. 1-6
Author(s):  
Jason M. Lodge ◽  
Linda Corrin ◽  
Gwo-Jen Hwang ◽  
Kate Thompson
Keyword(s):  
Educational Technology ◽  
Empirical Research ◽  
Quantitative Research ◽  
Open Science ◽  
Educational Technology Research ◽  
Technology Research ◽  
Psychological Science ◽  
Replication Crisis

Over the last decade a spate of issues has been emerging in empirical research spanning diverse fields such as biology, medicine, economics, and psychological science. The crisis has already led to fundamental shifts in how research is being conducted in several fields, particularly psychological science. Broadly labelled the ‘replication crisis’, these issues place substantial doubt on the robustness of peer-reviewed quantitative research across many disciplines. In this editorial, we will delve into the replication crisis and what it means for educational technology research. We will address two key areas, describing the extent to which the replication crisis applies to educational technology research and suggestions for responses by our community.

scholarly journals Advancing Transparency and Impact of Research: Initiating Crosstalk between Indigenous Research and Mainstream “Open Science”

2021 ◽  
Author(s):  
P. Priscilla Lui ◽  
Monica C. Skewes ◽  
Sarah Gobrial ◽  
David Rollock
Keyword(s):  
Negative Impact ◽  
Community Based Participatory Research ◽  
Open Science ◽  
Research Approach ◽  
Indigenous Research ◽  
Native Communities ◽  
Psychological Science ◽  
Human Functioning ◽  
Replication Crisis ◽  
Science Movement

To answer questions about human psychology, psychological science needs to yield credible findings. Because of their goals of understanding people’s lived experiences and advocating for the needs of the Native communities, Indigenous scholars tend to use community-based participatory research (CBPR) or approach science from a constructivist framework. The primary goal of mainstream psychological science is to uncover generalizable facts about human functioning. Approached from a postpositivist framework, mainstream psychological scholars tend to assume the possibility of identifying researcher biases and achieving objective science. Recently, many psychological findings fail to replicate in new samples. The replication crisis raised concerns about the validity of psychological science. The mainstream open science has been promoted as a solution to this replication crisis; the open science movement encourages researchers to emphasize transparency and accountability to the broad research community. The notion of transparency aligns with the principles of CBPR—research approach common in Indigenous research. Yet, open science practices are not widely adopted in Indigenous research, and mainstream open science does not emphasize researchers’ accountability to the communities that their science is intended to serve. We examined Indigenous researchers’ awareness and concerns about mainstream open science. Participants endorsed the value of transparency with the participants and their communities. They also were concerned about being disadvantaged and the possible negative impact of data sharing on the Native communities. We suggest that there is value in connecting mainstream open science and Indigenous research to advance science that empowers people and makes positive community impact.

What Psychology Teachers Should Know About Open Science and the New Statistics

2020 ◽  
Vol 47 (2) ◽  
pp. 169-179
Author(s):  
Beth Morling ◽  
Robert J. Calin-Jageman
Keyword(s):  
Open Science ◽  
Research Practice ◽  
Replication Crisis ◽  
Psychology Courses ◽  
Psychology Teachers ◽  
New Statistics ◽  
Science Movement

Psychology teachers have likely heard about the “replication crisis” and the “open science movement” in psychology, and they are probably aware that psychologists have proposed new standards for research practice. How should our psychology courses reflect these new standards? We describe several modern practices that have transformed our field and that seem likely to endure: preregistration of studies, transparency of reporting, norms for replication, and the new statistical focus on estimation and precision. We offer suggestions for how to integrate these new practices into psychology courses.

scholarly journals Emerging Privacy Issues in Times of Open Science

2021 ◽  
Author(s):  
Benjamin Erb ◽  
Christoph Bösch ◽  
Cornelia Herbert ◽  
Frank Kargl ◽  
Christian Montag
Keyword(s):  
Research Process ◽  
Open Science ◽  
Primary Data ◽  
Future Research ◽  
Data Sets ◽  
Replication Studies ◽  
Replication Crisis ◽  
Important Challenge ◽  
Meta Analyses ◽  
Privacy Issues

The open science movement has taken up the important challenge to increase transparency of statistical analyses, to facilitate reproducibility of studies, and to enhance reusability of data sets. To counter the replication crisis in the psychological and related sciences, the movement also urges researchers to publish their primary data sets alongside their articles. While such data publications represent a desirable improvement in terms of transparency and are also helpful for future research (e.g., subsequent meta-analyses or replication studies), we argue that such a procedure can worsen existing privacy issues that are insufficiently considered so far in this context. Recent advances in de-anonymization and re-identification techniques render privacy protection increasingly difficult, as prevalent anonymization mechanisms for handling participants' data might no longer be adequate. When exploiting publicly shared primary data sets, data from multiple studies can be linked with contextual data and eventually, participants can be de-anonymized. Such attacks can either re-identify specific individuals of interest, or they can be used to de-anonymize entire participant cohorts. The threat of de-anonymization attacks can endanger the perceived confidentiality of responses by participants, and ultimately, lower the overall trust of potential participants into the research process due to privacy concerns.

scholarly journals Privacy versus Open Science

2019 ◽  
Author(s):  
Simon Dennis ◽  
Paul Michael Garrett ◽  
Hyungwook Yim ◽  
Jihun Hamm ◽  
Adam F Osth ◽  
...  
Keyword(s):  
Open Science ◽  
Privacy Preserving ◽  
Research Practices ◽  
Psychological Science ◽  
Replication Crisis ◽  
Good Research

Pervasive internet and sensor technologies promise to revolutionize psychological science. However, the data collected using these technologies is often very personal - indeed the value of the data is often directly related to how personal it is. At the same time, driven by the replication crisis, there is a sustained push to publish data to open repositories. These movements are in fundamental conflict. In this paper, we propose a way to navigate this issue. We argue that there are significant advantages to be gained by ceding the ownership of data to the participants who generate it. Then we provide desiderata for a privacy-preserving platform. In particular, we suggest that researchers should use an interface to perform experiments and run analyses rather than observing the stimuli themselves. We argue that this method not only improves privacy but will also encourage greater compliance with good research practices than is possible with open repositories.

scholarly journals What Psychology Teachers Should Know about Open Science and the New Statistics (Morling & Calin-Jageman, 2020)

2019 ◽  
Author(s):  
Beth Morling ◽  
Robert Calin-Jageman
Keyword(s):  
Open Science ◽  
Research Practice ◽  
Teaching Of Psychology ◽  
Replication Crisis ◽  
Psychology Courses ◽  
Psychology Teachers ◽  
New Statistics ◽  
Science Movement

This article is in press at the journal Teaching of Psychology (Generalists' Corner). ABSTRACT: Psychology teachers have likely heard about the “replication crisis” and the “open science movement” in psychology, and they are probably aware that psychologists have proposed newstandards for research practice. How should our psychology courses reflect these new practices?We describe several modern practices that have transformed our field and that seem likely toendure: preregistration of studies, transparency of reporting, norms for replication, and the newstatistical focus on estimation and precision. We offer suggestions for how to integrate these newpractices into psychology courses.

Sign in / Sign up

Export Citation Format

Share Document