scholarly journals Open peer review: promoting transparency in open science

2020 ◽  
Vol 125 (2) ◽  
pp. 1033-1051
Author(s):  
Dietmar Wolfram ◽  
Peiling Wang ◽  
Adam Hembree ◽  
Hyoungjoo Park
Keyword(s):  
Peer Review ◽  
Open Science ◽  
Turn Of The Century ◽  
Bibliographic Databases ◽  
Early Adopters ◽  
Steady Growth ◽  
Scientific Disciplines ◽  
Review Reports ◽  
Science Movement ◽  
Different Levels

AbstractOpen peer review (OPR), where review reports and reviewers’ identities are published alongside the articles, represents one of the last aspects of the open science movement to be widely embraced, although its adoption has been growing since the turn of the century. This study provides the first comprehensive investigation of OPR adoption, its early adopters and the implementation approaches used. Current bibliographic databases do not systematically index OPR journals, nor do the OPR journals clearly state their policies on open identities and open reports. Using various methods, we identified 617 OPR journals that published at least one article with open identities or open reports as of 2019 and analyzed their wide-ranging implementations to derive emerging OPR practices. The findings suggest that: (1) there has been a steady growth in OPR adoption since 2001, when 38 journals initially adopted OPR, with more rapid growth since 2017; (2) OPR adoption is most prevalent in medical and scientific disciplines (79.9%); (3) five publishers are responsible for 81% of the identified OPR journals; (4) early adopter publishers have implemented OPR in different ways, resulting in different levels of transparency. Across the variations in OPR implementations, two important factors define the degree of transparency: open identities and open reports. Open identities may include reviewer names and affiliation as well as credentials; open reports may include timestamped review histories consisting of referee reports and author rebuttals or a letter from the editor integrating reviewers’ comments. When and where open reports can be accessed are also important factors indicating the OPR transparency level. Publishers of optional OPR journals should add metric data in their annual status reports.

scholarly journals Open Science in the Humanities, or: Open Humanities?

Publications ◽  
2019 ◽  
Vol 7 (4) ◽  
pp. 65 ◽  
Author(s):  
Marcel Knöchelmann
Keyword(s):  
Open Access ◽  
Peer Review ◽  
Scholarly Communication ◽  
Open Science ◽  
Essential Elements ◽  
Research Practices ◽  
Open Research ◽  
Scientific Disciplines ◽  
Communication Practices ◽  
Do So

Open science refers to both the practices and norms of more open and transparent communication and research in scientific disciplines and the discourse on these practices and norms. There is no such discourse dedicated to the humanities. Though the humanities appear to be less coherent as a cluster of scholarship than the sciences are, they do share unique characteristics which lead to distinct scholarly communication and research practices. A discourse on making these practices more open and transparent needs to take account of these characteristics. The prevalent scientific perspective in the discourse on more open practices does not do so, which confirms that the discourse’s name, open science, indeed excludes the humanities so that talking about open science in the humanities is incoherent. In this paper, I argue that there needs to be a dedicated discourse for more open research and communication practices in the humanities, one that integrates several elements currently fragmented into smaller, unconnected discourses (such as on open access, preprints, or peer review). I discuss three essential elements of open science—preprints, open peer review practices, and liberal open licences—in the realm of the humanities to demonstrate why a dedicated open humanities discourse is required.

scholarly journals The Cultivation of Social Work Knowledge: Towards a More Robust System of Peer Review

2020 ◽  
Author(s):  
Daniel J. Dunleavy
Keyword(s):  
Social Work ◽  
Peer Review ◽  
Social Workers ◽  
Open Science ◽  
Recent Issue ◽  
Double Blind ◽  
Recent Trends ◽  
Robust System ◽  
Work Knowledge ◽  
Science Movement

In a recent issue of Families in Society, Caputo (2019) argues for the "centrality" of peer review in the cultivation of social work knowledge. Specifically, he favors the double-blind model of peer review. In this paper, I argue that social work should adopt a more dynamic set of reviewing practices. First, I define some terminology and discuss the limitations of the current double-blind model. Then, I describe recent trends in peer review, which I argue foster a more robust and open system. I frame this discussion within the context of the wider open science movement and urge social workers to engage with these scholarly practices. In line with these practices and values, a freely accessible preprint is available at: xxxxxxxx.

scholarly journals Open Peer Review in Scientific Publishing: A Web Mining Study of PeerJ Authors and Reviewers

2017 ◽  
Vol 1 (4) ◽  
pp. 60-80 ◽  
Author(s):  
Peiling Wang ◽  
Sukjin You ◽  
Rath Manasa ◽  
Dietmar Wolfram
Keyword(s):  
Peer Review ◽  
Web Mining ◽  
Review Process ◽  
Peer Review Process ◽  
Open Science ◽  
Open Access Journal ◽  
Peer Reviews ◽  
Current Characteristics ◽  
Review Reports ◽  
Analytical Tools

AbstractPurposeTo understand how authors and reviewers are accepting and embracing Open Peer Review (OPR), one of the newest innovations in the Open Science movement.Design/methodology/approachThis research collected and analyzed data from the Open Access journal PeerJ over its first three years (2013–2016). Web data were scraped, cleaned, and structured using several Web tools and programs. The structured data were imported into a relational database. Data analyses were conducted using analytical tools as well as programs developed by the researchers.FindingsPeerJ, which supports optional OPR, has a broad international representation of authors and referees. Approximately 73.89% of articles provide full review histories. Of the articles with published review histories, 17.61% had identities of all reviewers and 52.57% had at least one signed reviewer. In total, 43.23% of all reviews were signed. The observed proportions of signed reviews have been relatively stable over the period since the Journal’s inception.Research limitationsThis research is constrained by the availability of the peer review history data. Some peer reviews were not available when the authors opted out of publishing their review histories. The anonymity of reviewers made it impossible to give an accurate count of reviewers who contributed to the review process.Practical implicationsThese findings shed light on the current characteristics of OPR. Given the policy that authors are encouraged to make their articles’ review history public and referees are encouraged to sign their review reports, the three years of PeerJ review data demonstrate that there is still some reluctance by authors to make their reviews public and by reviewers to identify themselves.Originality/valueThis is the first study to closely examine PeerJ as an example of an OPR model journal. As Open Science moves further towards open research, OPR is a final and critical component. Research in this area must identify the best policies and paths towards a transparent and open peer review process for scientific communication.

scholarly journals Reviewers' Decision to Sign Reviews is Related to Their Recommendation

2021 ◽  
Vol 5 ◽  
Author(s):  
Nino Van Sambeek ◽  
Daniel Lakens
Keyword(s):  
Peer Review ◽  
Royal Society ◽  
Positive Attitude ◽  
Open Science ◽  
Behavioral Data ◽  
Raw Data ◽  
Peer Reviews ◽  
Scientific Disciplines ◽  
Wide Range

Surveys indicate that researchers generally have a positive attitude towards open peer review when this consists of making reviews available alongside published articles. Researchers are more negative about revealing the identity of reviewers. They worry reviewers will be less likely to express criticism if their identity is known to authors. Experiments suggest that reviewers are somewhat less likely to recommend rejection when they are told their identity will be communicated to authors, than when they will remain anonymous. One recent study revealed reviewers in five journals who voluntarily signed their reviews gave more positive recommendations than those who did not sign their reviews. We replicate and extend this finding by analyzing 12010 open reviews in PeerJ and 4188 reviews in the Royal Society Open Science where authors can voluntarily sign their reviews. These results based on behavioral data from real peer reviews across a wide range of scientific disciplines demonstrate convincingly that reviewers’ decision to sign is related to their recommendation. The proportion of signed reviews was higher for more positive recommendations, than for more negative recommendations. We also share all 23649 text-mined reviews as raw data underlying our results that can be re-used by researchers interested in peer review.

scholarly journals What is open peer review? A systematic review

F1000Research ◽  
2017 ◽  
Vol 6 ◽  
pp. 588 ◽  
Author(s):  
Tony Ross-Hellauer
Keyword(s):  
Systematic Review ◽  
Peer Review ◽  
Peer Review Process ◽  
Open Science ◽  
Subject Area ◽  
Open Review ◽  
Broad Subject ◽  
Review Reports ◽  
The Many ◽  
Definition Of

Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions. While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are then systematically analysed to build a coherent typology of the many different innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how  ambiguously the phrase “open peer review”  has been used thus far, for the literature offers a total of 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature. Conclusions: Based on this work, I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the ethos of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

scholarly journals The Promise of the Open Science Movement for Research on Identity

2019 ◽  
Author(s):  
Moin Syed
Keyword(s):  
Cancer Biology ◽  
Empirical Work ◽  
Open Science ◽  
Scientific Disciplines ◽  
Core Issues ◽  
Science Movement

The open science movement has been gaining steam in numerous scientific disciplines (e.g., ecology, cancer biology, economics) as well as sub-disciplines of psychology (e.g., social, personality). These issues, however, have been scantly discussed in the context of identity research. This presentation will include an overview of core issues in the open science movement and how they apply to research on identity. Emphasis will be placed on how incorporating open science principles can improve both theoretical and empirical work on identity.

scholarly journals What is open peer review? A systematic review

F1000Research ◽  
2017 ◽  
Vol 6 ◽  
pp. 588 ◽  
Author(s):  
Tony Ross-Hellauer
Keyword(s):  
Systematic Review ◽  
Peer Review ◽  
Peer Review Process ◽  
Open Science ◽  
Subject Area ◽  
Open Review ◽  
Rich Information ◽  
Broad Subject ◽  
Review Reports ◽  
Definition Of

Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions: I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

scholarly journals The Open Science Movement is For All of Us

2019 ◽  
Author(s):  
Moin Syed
Keyword(s):  
Cancer Biology ◽  
Empirical Work ◽  
Psychological Research ◽  
Open Science ◽  
Scientific Disciplines ◽  
Core Issues ◽  
Science Movement

The open science movement has been gaining steam in numerous scientific disciplines (e.g., ecology, cancer biology, economics) as well as sub-disciplines of psychology (e.g., social, personality). These issues, however, are still not widely understood nor seen as applicable to all types of research. This presentation will include an overview of core issues in the open science movement and how they apply to all types of psychological research (any sub-discipline, any method). Emphasis will be placed on how incorporating open science principles can improve both theoretical and empirical work in psychology.

scholarly journals The State of The Art in Peer Review

2018 ◽  
Author(s):  
Jonathan Tennant
Keyword(s):  
Peer Review ◽  
State Of The Art ◽  
Parallel Evolution ◽  
Scholarly Communication ◽  
Open Science ◽  
Changing Environments ◽  
Open Research ◽  
Explosive Wave ◽  
Science Movement ◽  
Insight Into

Scholarly communication is in a perpetual state of disruption. Within this, peer review of research articles remains an essential part of the formal publication process, distinguishing it from virtually all other modes of communication. In the last several years, there has been an explosive wave of innovation in peer review research, platforms, discussions, tools, and services. This is largely coupled with the ongoing and parallel evolution of scholarly communication as it adapts to rapidly changing environments, within what is widely considered as the ‘open research’ or ‘open science’ movement. Here, we summarise the current ebb and flow around changes to peer review and consider its role in a modern digital research and communications infrastructure and discuss why uptake of new models of peer review appears to have been so low compared to what is often viewed as the ‘traditional’ method of peer review. Finally, we offer some insight into the potential futures of scholarly peer review and consider what impacts this might have on the broader scholarly research ecosystem.

The Cultivation of Social Work Knowledge: Toward a More Robust System of Peer Review

2021 ◽  
pp. 104438942110122
Author(s):  
Daniel J. Dunleavy
Keyword(s):  
Social Work ◽  
Peer Review ◽  
Social Workers ◽  
Open Science ◽  
Recent Issue ◽  
Double Blind ◽  
Recent Trends ◽  
Robust System ◽  
Work Knowledge ◽  
Science Movement

In a recent issue of Families in Society, Caputo argues for the “centrality” of peer review in the cultivation of social work knowledge. Specifically, he favors the double-blind model of peer review. In this paper, I argue that social work should adopt a more dynamic set of reviewing practices. First, I define some terminology and discuss the limitations of the current double-blind model. Then, I describe recent trends in peer review, which I argue foster a more robust and open system. I frame this discussion within the context of the wider open science movement and urge social workers to engage with these scholarly practices. In line with these practices and values, a freely accessible preprint is available at: https://osf.io/preprints/socarxiv/rvqze/

Sign in / Sign up

Export Citation Format

Share Document