Improving norms in research culture to incentivize transparency and rigor

Mapping Intimacies ◽

10.35542/osf.io/thny5 ◽

2021 ◽

Author(s):

David Thomas Mellor

Keyword(s):

Educational Psychology ◽

Next Generation ◽

Research Culture ◽

Grassroots Groups

Improving research culture to value transparency and rigor is necessary to engage in a productive “Credibility Revolution.” The field of educational psychology is well positioned to act toward this goal. It will take specific actions by both grassroots groups plus leadership to set standards that will ensure that getting published, funded, or hired is determined by universally supported ideals. These improved standards must ensure that transparency, rigor, and credibility are valued above novelty, impact, and incredibility. Grassroots groups advocate for change and share experience so that the next generation of researchers have the experience needed to sustain these early moves. Each community can take inspiration from others that have made shifts toward better practices. These instances provide opportunities for emulating trail-blazers, training for new practices such as preregistration, and constructively evaluating or criticizing practice in ways that advances the reputation of all involved.

Download Full-text

Improving a research culture in educational psychology to value transparency over novelty takes establishing new norms, grassroots advocacy, and top-down leadership

10.35542/osf.io/8tdgu ◽

2020 ◽

Author(s):

David Thomas Mellor

Keyword(s):

Communities Of Practice ◽

Educational Psychology ◽

Scientific Community ◽

New World ◽

Statistical Significance ◽

Research Culture ◽

Policy Makers ◽

Grassroots Groups ◽

Scientific Disciplines ◽

Grassroots Advocacy

Many scientific disciplines are in the midst of reproducibility crisis. The cumulative harm of many seemingly modest sins are adding up to a diminished confidence in disturbingly large areas of research. These practices, which are enabled by a default towards opacity and by valuing statistical significance, impact, or novelty, undermine our credibility as a community that is expected to act in accordance with scientific ideals. Changing this reality can no longer be in the hands of the idealistic or privileged few, but must be the focus of every stakeholder in the scientific community. The field of educational psychology is well positioned to act toward this goal. This change is possible, and will transform the reproducibility crisis into a credibility revolution. It will take specific actions by both grassroots groups to build communities of practice plus the leadership of policy makers who can set standards for evaluating articles, grants, and hiring packets. These improved standards must ensure that transparency, rigor, and credibility are valued above novelty, impact, and incredibility. Grassroots groups advocate for change, share experience, and become an opportunity to create trainings so that the next generation of educational psychology researchers have the experience needed to sustain these early moves. We can take inspiration from other communities that have made shifts toward better practices, and can specifically prepare for this new world by modeling constructive evaluations of such open practices as those that take place through badging initiatives. These instances provide opportunities for emulating trail-blazers, training for new practices such as preregistration, and constructively evaluating or criticizing practice in ways that advances the reputation of all involved. This article is available as a preprint on EdArXiv:

Download Full-text

Faculty Opinions recommendation of Next generation diabetes scientists shape global research culture: A reflective proposal from postdoctoral researchers in diabetes research.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.737399302.793571401 ◽

2020 ◽

Author(s):

Michael Symonds

Keyword(s):

Diabetes Research ◽

Next Generation ◽

Research Culture ◽

Postdoctoral Researchers ◽

Global Research

Download Full-text

Strengthening the foundation of educational psychology by integrating construct validation into open science reform

10.31234/osf.io/vqjb8 ◽

2021 ◽

Author(s):

Jessica Kay Flake

Keyword(s):

Educational Psychology ◽

Large Scale ◽

Construct Validation ◽

Open Science ◽

Measurement Development ◽

Reform Movement ◽

Research Culture ◽

Replication Studies ◽

Replication Research ◽

Access To Data

An increased focus on transparency and replication in science has stimulated reform in research practices and dissemination. As a result, the research culture is changing: the use of preregistration is on the rise, access to data and materials is increasing, and large-scale replication studies are more common. In this paper, I discuss two problems the methodological reform movement is now ready to tackle given the progress thus far and how educational psychology is particularly well suited to contribute. The first problem is that there is a lack of transparency and rigor in measurement development and use. The second problem is caused by the first; replication research is difficult and potentially futile as long as the first problem persists. I describe how to expand transparent practices into measure use and how construct validation can be implemented to bolster the validity of replication studies.

Download Full-text