Skin in the Game: Personal Accountability and Journal Peer Review

Two preregistered studies explored the likelihood paper reviewers would request clarification from authors regarding potential questionable research practices (QRPs). Study 1 participants were instructed to imagine reviewing a journal manuscript as either a coauthor or peer reviewer and rate the extent to which they would request clarification from the author when encountering potential QRPs. Participants reported greater likelihood of requesting clarification when assigned to the coauthor relative to the peer reviewer role. Study 2 participants were assigned to either an anonymous or open-review condition and rated the extent to which they would seek clarification from an author regarding potential QRPs. Men (but not women) in the open review condition reported greater likelihood of seeking clarification about potential QRPs than men in the blind review condition. Results provide tentative evidence that motivational factors influence the peer review process, and suggestions are made for improving peer review practices.

Download Full-text

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

PLoS ONE ◽

10.1371/journal.pone.0260558 ◽

2021 ◽

Vol 16 (11) ◽

pp. e0260558

Author(s):

Bridget C. O’Brien ◽

Anthony R. Artino ◽

Joseph A. Costello ◽

Erik Driessen ◽

Lauren A. Maggio

Keyword(s):

Peer Review ◽

Review Process ◽

Qualitative Content Analysis ◽

Peer Review Process ◽

Argument Quality ◽

Blind Review ◽

Coding Scheme ◽

Explicit Reference ◽

Education Journal ◽

Single Blind

Purpose Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. Results 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%). Conclusion Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors.

Download Full-text

Peer review blinding practices of highly ranked dental journals: analysis and discussion

BDJ ◽

10.1038/s41415-021-3319-y ◽

2021 ◽

Author(s):

Clovis Mariano Faggion Jr

Keyword(s):

Logistic Regression ◽

Peer Review ◽

Odds Ratio ◽

Review Process ◽

Peer Review Process ◽

Linear Regression Analysis ◽

Double Blind ◽

Lower Odds ◽

Blind Review ◽

Single Blind

AbstractObjectives To evaluate the type of peer review blinding used in highly ranked dental journals and to discuss the influence of the blinding approaches on the peer review process.Methods All 91 dental journals classified by impact factor (IF) had their websites scrutinised for the type of peer review blinding used for submissions. If the information was not reported, the journals were contacted to obtain the information. Linear and logistic regression were applied to evaluate the association between type of peer review blinding and IF.Results The selected journals reported the following peer review blinding approaches: single-blind (N = 36, 39.6%), double-blind (N = 46, 50.5%), transparent (N = 2, 2.2%) and open (N = 1, 1.1%). Information from six (6.6%) journals was not available. A linear regression analysis demonstrated that journals with lower IFs were associated with double-blind review (p = 0.001). A logistic regression suggested lower odds of association between single-blind peer review and journals with IFs below a threshold of 2 (odds ratio 0.157, confidence interval 0.059 to 0.417, p <0.001).Conclusions The majority of highly ranked dental journals had single- and double-blind peer review; journals with higher IFs presented single-blind peer review and those with lower IFs reported double-blind peer review.

Download Full-text

Peer review declaration

Journal of Physics Conference Series ◽

10.1088/1742-6596/2161/1/011002 ◽

2022 ◽

Vol 2161 (1) ◽

pp. 011002

Keyword(s):

Peer Review ◽

Review Process ◽

Peer Review Process ◽

Assistant Professor ◽

Committee Chair ◽

Programme Committee ◽

Blind Review ◽

Institute Of Technology ◽

Scientific Standards ◽

Single Blind

All papers published in this volume of Journal of Physics: Conference Series have been peer reviewed through processes administered by the Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a proceedings journal published by IOP Publishing. • Type of peer review: Single-blind All the articles of AICECS 2021 followed the “Single-blind” peer review process, where the reviewers were aware of the authors’ identity but not vice-versa • Conference submission management system: EasyChair All the submission and communication to all the AICECS 2021 authors were through EasyChair (https://easychair.org/conferences/?conf=aicecs2021) • Number of submissions received: 149 • Number of submissions sent for review: 136 • Number of submissions accepted: 78 • Acceptance Rate: 52.3% (Number of Submissions Accepted/Number of Submissions Received X 100): (78/149) x 100 = 52.3%) • Average number of reviews per paper: 2 • Total number of reviewers involved: 106 Any additional info on review process: All the initial submissions went through a single-blind review, based on the subject experts (reviewers) and Technical Programme Committee Chair (TPC) and General Chair decision (accept or major revision or minor revision or reject) communicated to the authors through EasyChair. Based on the recommendation, the authors revised the articles and submitted their revised papers. The revised submission was verified by the TPC and General Chair for their final recommendation for the submission. Online similarity check has been carried out using Turnitin software at all the stages from submission to acceptance. Contact person for queries: Name : Dr. Tanweer Assistant Professor-Senior Scale, Department of E&C, General Chair, AICECS 2021, Manipal Institute of Technology, Manipal, Karnataka, India Email : [email protected]; [email protected]

Download Full-text

Announcing a New Registered Report Article Type at the Journal of Speech, Language, and Hearing Research

Journal of Speech Language and Hearing Research ◽

10.1044/2021_jslhr-21-00513 ◽

2022 ◽

pp. 1-4

Author(s):

Holly L. Storkel ◽

Frederick J. Gallun

Keyword(s):

Peer Review ◽

Publication Bias ◽

Peer Review Process ◽

Research Integrity ◽

Research Practices ◽

Scientific Publications ◽

Hearing Research ◽

Questionable Research Practices ◽

Structural Problems ◽

Article Type

Purpose: This editorial introduces the new registered reports article type for the Journal of Speech, Language, and Hearing Research . The goal of registered reports is to create a structural solution to address issues of publication bias toward results that are unexpected and sensational, questionable research practices that are used to produce novel results, and a peer-review process that occurs at the end of the research process when changes in fundamental design are difficult or impossible to implement. Conclusion: Registered reports can be a positive addition to scientific publications by addressing issues of publication bias, questionable research practices, and the late influence of peer review. This article type does so by requiring reviewers and authors to agree in advance that the experimental design is solid, the questions are interesting, and the results will be publishable regardless of the outcome. This procedure ensures that replication studies and null results make it into the published literature and that authors are not incentivized to alter their analyses based on the results that they obtain. Registered reports represent an ongoing commitment to research integrity and finding structural solutions to structural problems inherent in a research and publishing landscape in which publications are such a high-stakes aspect of individual and institutional success.

Download Full-text

Peer review analysis in the field of radiation oncology: results from a web-based survey of the Young DEGRO working group

Strahlentherapie und Onkologie ◽

10.1007/s00066-020-01729-2 ◽

2020 ◽

Author(s):

Lukas Käsmann ◽

◽

Annemarie Schröder ◽

Benjamin Frey ◽

Daniel F. Fleischmann ◽

...

Keyword(s):

Peer Review ◽

Radiation Oncology ◽

Working Group ◽

Review Process ◽

Peer Review Process ◽

Free Access ◽

Scientific Content ◽

Double Blind ◽

Blind Review ◽

Scientific Background

Abstract Purpose To evaluate the reviewing behaviour in the German-speaking countries in order to provide recommendations to increase the attractiveness of reviewing activity in the field of radiation oncology. Methods In November 2019, a survey was conducted by the Young DEGRO working group (jDEGRO) using the online platform “eSurveyCreator”. The questionnaire consisted of 29 items examining a broad range of factors that influence reviewing motivation and performance. Results A total of 281 responses were received. Of these, 154 (55%) were completed and included in the evaluation. The most important factors for journal selection criteria and peer review performance in the field of radiation oncology are the scientific background of the manuscript (85%), reputation of the journal (59%) and a high impact factor (IF; 40%). Reasons for declining an invitation to review include the scientific background of the article (60%), assumed effort (55%) and a low IF (27%). A double-blind review process is preferred by 70% of respondents to a single-blind (16%) or an open review process (14%). If compensation was offered, 59% of participants would review articles more often. Only 12% of the participants have received compensation for their reviewing activities so far. As compensation for the effort of reviewing, 55% of the respondents would prefer free access to the journal’s articles, 45% a discount for their own manuscripts, 40% reduced congress fees and 39% compensation for expenses. Conclusion The scientific content of the manuscript, reputation of the journal and a high IF determine the attractiveness for peer reviewing in the field of radiation oncology. The majority of participants prefer a double-blind peer review process and would conduct more reviews if compensation was available. Free access to journal articles, discounts for publication costs or congress fees, or an expense allowance were identified to increase attractiveness of the review process.

Download Full-text

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

10.20944/preprints201905.0098.v1 ◽

2019 ◽

Author(s):

Lonni Besançon ◽

Niklas Rönnberg ◽

Jonas Löwgren ◽

Jonathan P. Tennant ◽

Matthew Cooper

Keyword(s):

Human Computer Interaction ◽

Peer Review ◽

Review Process ◽

Peer Review Process ◽

Open Science ◽

Practical Implementation ◽

Science Practices ◽

Peer Reviewing ◽

Open Review

We present a discussion and analysis regarding the benefits and limitations of open and non-anonymized peer review based on literature results and responses to a survey on the reviewing process of alt.chi, a more or less open-review track within the CHI conference, the predominant conference in the field of human-computer interaction (HCI). This track currently is the only implementation of an open-peer-review process in the field of HCI while, with the recent increase in interest in open science practices, open review is now being considered and used in other fields. We collected 30 responses from alt.chi authors and reviewers and found that, while the benefits are quite clear and the system is generally well liked by alt.chi participants, they are reluctant to see it used in other venues. This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

Download Full-text

Reviewers’ Perceptions of the Peer Review Process in Journalism and Mass Communication

Journalism & Mass Communication Quarterly ◽

10.1177/1077699017736031 ◽

2017 ◽

Vol 95 (1) ◽

pp. 278-299 ◽

Cited By ~ 6

Author(s):

Patricia A. Curtin ◽

John Russial ◽

Alec Tefertiller

Keyword(s):

Peer Review ◽

Mass Communication ◽

Career Advancement ◽

Review Process ◽

Response Rate ◽

Peer Review Process ◽

Double Blind ◽

Blind Review ◽

Helping Others ◽

Intrinsic Motivations

This survey of the Association for Education in Journalism and Mass Communication (AEJMC) scholars ( N = 547; response rate = 39.3%) examines perceptions of peer review, a study last undertaken in 1990. Respondents rated intrinsic motivations, such as helping others, more highly than extrinsic motivations, such as reviewing as a consideration for career advancement. Respondents believed they did a significantly better job as reviewers than did scholars who reviewed their work. More experienced reviewers saw their role as balanced between critic and coach, whereas less experienced reviewers saw their role as significantly more critical. Overall, respondents were ambivalent about the state of peer review, yet they resisted adopting approaches other than double-blind review.

Download Full-text

Referees’ Attitudes toward Open Peer Review and Electronic Transmission of Papers

Food Science and Technology International ◽

10.1106/0mxd-ym6f-3lm6-g9eb ◽

2001 ◽

Vol 7 (6) ◽

pp. 521-527 ◽

Cited By ~ 11

Author(s):

R. Melero ◽

F. López-Santoveña

Keyword(s):

Peer Review ◽

Review Process ◽

Response Rate ◽

Peer Review Process ◽

Personal Characteristics ◽

Point Of View ◽

Food Science ◽

Open Review ◽

And Gender ◽

The Web

A survey was mailed to 293 referees from the review board of Food Science and Technology International with the following personal characteristics: ages: 35–45 (35%), 45–55 (37%), and 55–65 (27%); 93% PhD graduates; 69% male, 98% researchers, 82% teachers too, 85% review for other journals as well to assess reviewers’ attitudes or preferences in favor of or against masking their identity, and toward the electronic transmission of papers for review. The reviewers were mainly from Europe, North America, and South America. The questionnaire was anonymous and asked if respondents were in favor of an open review or masking of the reviewers, and if they agreed with the electronic transmission of the papers for their review (both from the point of view of author and reviewer). The response rate was 35% (103 respondents). The consistency between the answers as being authors or reviewers when asked by the peer review process was significant ( P<0.001) without significant differences in terms of gender or age. Seventy-five percent were in favor of masking reviewers, and 17% completely favored unblinded review. The consistency between the answers for paper transmission was significant ( P<0.001) without significant differences in terms of gender or age. Seventy-five percent were in favor of electronic transmission, 25% were against it. There was a significant association between the answers in favor of or against e-transmission and the age either as reviewers ( P=0.009) or as authors ( P= 0.031). The other associations between the system of review and gender or age were not significant. There was a preference among the participants for masking the reviewers, and a tendency to use the Web as the transmission medium because it is considered faster, easier, simpler, and more economic.

Download Full-text

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

10.20944/preprints201905.0098.v2 ◽

2019 ◽

Cited By ~ 1

Author(s):

Lonni Besançon ◽

Niklas Rönnberg ◽

Jonas Löwgren ◽

Jonathan Tennant ◽

Matthew Cooper

Keyword(s):

Human Computer Interaction ◽

Peer Review ◽

Review Process ◽

Peer Review Process ◽

Open Science ◽

Practical Implementation ◽

Science Practices ◽

Peer Reviewing ◽

Open Review

Download Full-text

Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors

10.1101/2021.07.28.454037 ◽

2021 ◽

Author(s):

Bridget C O'Brien ◽

Anthony R Artino ◽

Joseph A Costello ◽

Erik Driessen ◽

Lauren A Maggio

Keyword(s):

Peer Review ◽

Review Process ◽

Qualitative Content Analysis ◽

Peer Review Process ◽

Argument Quality ◽

Blind Review ◽

Coding Scheme ◽

Explicit Reference ◽

Education Journal ◽

Single Blind

Purpose: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer review process. Peer review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods: Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended "reject," we coded for alignment between reviewers' comments to the editor and to authors. We report descriptive statistics. Results: 49% of reviews contained comments to the editor (n=176). Most of these comments summarized the reviewers' impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 48 reviews recommending "reject," the majority of comments to the editor contained content that also appeared in comments to the authors (65%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (85%). Conclusion: Findings indicate variability in how reviewers use the confidential comments to editor section in online peer review systems, though generally the way they use them suggests integrity and transparency to authors.

Download Full-text