Comparing dream to reality: an assessment of adherence of the first generation of preregistered studies

Preregistration is a method to increase research transparency by documenting research decisions on a public, third-party repository prior to any influence by data. It is becoming increasingly popular in all subfields of psychology and beyond. Adherence to the preregistration plan may not always be feasible and even is not necessarily desirable, but without disclosure of deviations, readers who do not carefully consult the preregistration plan might get the incorrect impression that the study was exactly conducted and reported as planned. In this paper, we have investigated adherence and disclosure of deviations for all articles published with the Preregistered badge in Psychological Science between February 2015 and November 2017 and shared our findings with the corresponding authors for feedback. Two out of 27 preregistered studies contained no deviations from the preregistration plan. In one study, all deviations were disclosed. Nine studies disclosed none of the deviations. We mainly observed (un)disclosed deviations from the plan regarding the reported sample size, exclusion criteria and statistical analysis. This closer look at preregistrations of the first generation reveals possible hurdles for reporting preregistered studies and provides input for future reporting guidelines. We discuss the results and possible explanations, and provide recommendations for preregistered research.

How (Not) to Reproduce: Practical Considerations to Improve Research Transparency in Political Science

ABSTRACT In recent years, scholars, journals, and professional organizations in political science have been working to improve research transparency. Although better transparency is a laudable goal, the implementation of standards for reproducibility still leaves much to be desired. This article identifies two practices that political science should adopt to improve research transparency: (1) journals must provide detailed replication guidance and run provided material; and (2) authors must begin their work with replication in mind. We focus on problems that occur when scholars provide research materials to journals for replication, and we outline best practices regarding documentation and code structure for researchers to use.

Evidence on Research Transparency in Economics

A decade ago, the term “research transparency” was not on economists' radar screen, but in a few short years a scholarly movement has emerged to bring new open science practices, tools and norms into the mainstream of our discipline. The goal of this article is to lay out the evidence on the adoption of these approaches – in three specific areas: open data, pre-registration and pre-analysis plans, and journal policies – and, more tentatively, begin to assess their impacts on the quality and credibility of economics research. The evidence to date indicates that economics (and related quantitative social science fields) are in a period of rapid transition toward new transparency-enhancing norms. While solid data on the benefits of these practices in economics is still limited, in part due to their relatively recent adoption, there is growing reason to believe that critics' worst fears regarding onerous adoption costs have not been realized. Finally, the article presents a set of frontier questions and potential innovations.

UKRN Position on Supporting Research Transparency among Research Students

UKRN position statement on supporting research transparency among research students. Written by the UKRN Steering Group. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA).

Public access to protocols of contemporary cancer randomized clinical trials

Access to randomized clinical trial (RCT) protocols is necessary for the interpretation and reproducibility of the study results, but protocol availability has been lacking. We determined the prevalence of protocol availability for all published cancer RCTs in January 2020. We found that only 36.1% (48/133) of RCTs had an accessible protocol and only 11.3% of RCTs (15/133) had a publicly accessible protocol that was not behind a paywall. Only 18.0% (24/133) of RCTs were published in conjunction with the protocol on the journal website. In conclusion, few cancer RCTs have an accessible research protocol. Journals should require publication of RCT protocols along with manuscripts to improve research transparency.