scholarly journals Double blind reviews in software engineering venues: Practicability, promises and perils

2016 ◽  
Author(s):  
Moritz Beller ◽  
Alberto Bacchelli
Keyword(s):  
Software Engineering ◽  
Scientific Method ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Blind Reviews ◽  
Hidden Biases ◽  
Benefits And Costs ◽  
Single Blind

The peer review process is central to the scientific method, the advancement and spread of research as well as crucial for individual careers. However, the single blind review process currently used in most Software Engineering (SE) venues is susceptible towards apparent and hidden biases, since reviewers know the identity of authors. In this paper, we perform a study on the benefits and costs that are associated with introducing double-blind reviews in SE venues. We surveyed the SE community’s opinion, interviewed experts on double-blind reviewing, and estimated the likelihood of reviewers being able to guess the authors. Our results indicate that double-blind reviewing could be introduced in large SE conferences at lower-than-generally believed costs and that the majority of the SE community is in favor of introducing it.

scholarly journals Double-blind review in software engineering venues

2016 ◽  
Author(s):  
Moritz Beller ◽  
Alberto Bacchelli
Keyword(s):  
Side Effects ◽  
Software Engineering ◽  
Scientific Method ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Hidden Biases ◽  
Benefits And Costs ◽  
Single Blind

The peer review process is central to the scientific method, the advancement and spread of research, as well as crucial for individual careers. However, the single-blind review mode currently used in most Software Engineering (SE) venues is susceptible to apparent and hidden biases, since reviewers know the identity of authors. We perform a study on the benefits and costs that are associated with introducing double- blind review in SE venues. We surveyed the SE community’s opinion and interviewed experts on double-blind reviewing. Our results indicate that the costs, mostly logistic challenges and side effects, outnumber its benefits and mostly regard difficulty for authors in blinding papers, for reviewers in understanding the increment with respect to previous work from the same authors, and for organizers to manage a complex transition. While the surveyed community largely consents on the costs of DBR, only less than one-third disagree with a switch to DBR for SE journals, all SE conferences, and, in particular, ICSE; the analysis of a survey with authors of submitted papers at ICSE 2016 run by the program chairs of that edition corroborates our result.

scholarly journals Double-blind review in software engineering venues

2016 ◽  
Author(s):  
Moritz Beller ◽  
Alberto Bacchelli
Keyword(s):  
Side Effects ◽  
Software Engineering ◽  
Scientific Method ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Hidden Biases ◽  
Benefits And Costs ◽  
Single Blind

The peer review process is central to the scientific method, the advancement and spread of research, as well as crucial for individual careers. However, the single-blind review mode currently used in most Software Engineering (SE) venues is susceptible to apparent and hidden biases, since reviewers know the identity of authors. We perform a study on the benefits and costs that are associated with introducing double- blind review in SE venues. We surveyed the SE community’s opinion and interviewed experts on double-blind reviewing. Our results indicate that the costs, mostly logistic challenges and side effects, outnumber its benefits and mostly regard difficulty for authors in blinding papers, for reviewers in understanding the increment with respect to previous work from the same authors, and for organizers to manage a complex transition. While the surveyed community largely consents on the costs of DBR, only less than one-third disagree with a switch to DBR for SE journals, all SE conferences, and, in particular, ICSE; the analysis of a survey with authors of submitted papers at ICSE 2016 run by the program chairs of that edition corroborates our result.

scholarly journals Peer review blinding practices of highly ranked dental journals: analysis and discussion

BDJ ◽  
2021 ◽  
Author(s):  
Clovis Mariano Faggion Jr
Keyword(s):  
Logistic Regression ◽  
Peer Review ◽  
Odds Ratio ◽  
Review Process ◽  
Peer Review Process ◽  
Linear Regression Analysis ◽  
Double Blind ◽  
Lower Odds ◽  
Blind Review ◽  
Single Blind

AbstractObjectives To evaluate the type of peer review blinding used in highly ranked dental journals and to discuss the influence of the blinding approaches on the peer review process.Methods All 91 dental journals classified by impact factor (IF) had their websites scrutinised for the type of peer review blinding used for submissions. If the information was not reported, the journals were contacted to obtain the information. Linear and logistic regression were applied to evaluate the association between type of peer review blinding and IF.Results The selected journals reported the following peer review blinding approaches: single-blind (N = 36, 39.6%), double-blind (N = 46, 50.5%), transparent (N = 2, 2.2%) and open (N = 1, 1.1%). Information from six (6.6%) journals was not available. A linear regression analysis demonstrated that journals with lower IFs were associated with double-blind review (p = 0.001). A logistic regression suggested lower odds of association between single-blind peer review and journals with IFs below a threshold of 2 (odds ratio 0.157, confidence interval 0.059 to 0.417, p <0.001).Conclusions The majority of highly ranked dental journals had single- and double-blind peer review; journals with higher IFs presented single-blind peer review and those with lower IFs reported double-blind peer review.

scholarly journals The cases for and against double-blind reviews

PeerJ ◽  
2019 ◽  
Vol 7 ◽  
pp. e6702 ◽  
Author(s):  
Amelia R. Cox ◽  
Robert Montgomerie
Keyword(s):  
Gender Bias ◽  
Behavioral Ecology ◽  
Peer Review Process ◽  
Impact Factors ◽  
Double Blind ◽  
Blind Review ◽  
Female Authors ◽  
Female Authorship ◽  
Blind Reviews ◽  
Single Blind

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers’ biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each other’s identity and gender. To test the efficacy of double-blind reviews in one behavioral ecology journal (Behavioral Ecology, BE), we assigned gender to every authorship of every paper published for 2010–2018 in that journal compared to four other journals with single-blind reviews but similar subject matter and impact factors. While female authorships comprised only 35% of the total in all journals, the double-blind journal (BE) did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (BE and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis) for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review no longer benefits female authors and we discuss the pros and cons of the double-blind reviewing process based on our findings.

The Case For and Against Double-blind Reviews

2018 ◽  
Author(s):  
Amelia R Cox ◽  
Robert Montgomerie
Keyword(s):  
Peer Review ◽  
Gender Bias ◽  
Behavioral Ecology ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Female Authors ◽  
Female Authorship ◽  
Blind Reviews ◽  
Single Blind

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers' unconscious biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each other's identities and genders. To test the efficacy of double-blind reviews, we assigned gender to every authorship of every paper published in 5 different journals with different peer review processes (double-blind vs. single blind) and subject matter (birds vs. behavioral ecology) from 2010-2018 (n = 4865 papers). While female authorships comprised only 35% of the total, the double-blind journal Behavioral Ecology did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (Behavioral Ecology and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis), for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review does not benefit female authors and may, in the long run, be detrimental.

scholarly journals Open, single-blind, double-blind: which peer review process do you prefer?

2014 ◽  
Vol 15 (1) ◽  
Author(s):  
Elizabeth C Moylan ◽  
Simon Harold ◽  
Ciaran O’Neill ◽  
Maria K Kowalczuk
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Single Blind

scholarly journals Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

PLoS ONE ◽  
2021 ◽  
Vol 16 (11) ◽  
pp. e0260558
Author(s):  
Bridget C. O’Brien ◽  
Anthony R. Artino ◽  
Joseph A. Costello ◽  
Erik Driessen ◽  
Lauren A. Maggio
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Qualitative Content Analysis ◽  
Peer Review Process ◽  
Argument Quality ◽  
Blind Review ◽  
Coding Scheme ◽  
Explicit Reference ◽  
Education Journal ◽  
Single Blind

Purpose Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. Results 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%). Conclusion Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors.

scholarly journals Peer review declaration

2022 ◽  
Vol 2148 (1) ◽  
pp. 011003
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Software Tool ◽  
Editorial Office ◽  
Technical Quality ◽  
Double Blind ◽  
Scientific Standards ◽  
Single Blind ◽  
Initial Check

All papers published in this volume of Journal of Physics: Conference Series have been peer reviewed through processes administered by the Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a proceedings journal published by IOP Publishing. • Type of peer review: Single-blind / Double-blind / Triple-blind / Open / Other (please describe) 1. ICPEM Editors perform an initial check of the manuscript’s suitability upon receipt, and use a software tool to finish the plagiarism analysis, manuscripts are out of conference topics will be rejected directly, generally, authors will receive the result within 3-5 working days in this round. 2. Only the manuscripts passed the initial checking can be submitted to reviewers, ICPEM Editorial Office will then organize the peer-review process performed by independent experts. Papers will be strictly and thoroughly peer-reviewed by experts and reviewers. 3. All regular papers are reviewed by at least two reviewers, but usually by three or more, and rated considering: Relevance, Originality, Technical Quality, Significance and Presentation of the submissions; There are four results: 1, Accept; 2, Accept after Minor Revisions; 3, Reconsider after Major Revisions; 4, Reject. 4. Authors have 2-3 weeks to make minor or major revisions after received the comments from reviewers. Usually, one round of major revisions is allowed. 5. Only the submission passed the peer review and accepted by reviewers will be included in the conference proceeding finally. • Conference submission management system: Online Email System • Number of submissions received: 141 • Number of submissions sent for review: 116 (25 papers out of the conference scope are rejected directly) • Number of submissions accepted: 69 • Acceptance Rate (Number of Submissions Accepted / Number of Submissions Received X 100): 49% • Average number of reviews per paper: 2-3 • Total number of reviewers involved: 164 • Any additional info on review process: • Contact person for queries: Name : Josh Sheng Affiliation: Hubei Zhongke Research Institute of Nature Science, China Email : [email protected]

scholarly journals Peer review declaration

2021 ◽  
Vol 890 (1) ◽  
pp. 011002
Keyword(s):  
Peer Review ◽  
English Language ◽  
Environmental Science ◽  
Review Process ◽  
Peer Review Process ◽  
Subject Area ◽  
Marine Science ◽  
Double Blind ◽  
Scientific Standards ◽  
Single Blind

Abstract All papers published in this volume of IOP Conference Series: Earth and Environmental Science has been peer reviewed through processes administered by the Editors. The review processes were conducted by expert referees to the professional and scientific standards expected of a proceedings journal published by IOP Publishing. • Type of peer review: Single-blind/Double-blind/Triple-blind/Open/Other (please describe) Double-blind: All papers came through the basic review which included an initial technical criteria check (paper field, structure of submission, adherence to the submission instructions, English language usage and the ethics of scientific writing including a check for the similarity rate). Any papers out of the scope or containing plagiarism were rejected directly. The initial technical criteria check by the editors. The accepted papers came through peer review process by two professional experts in the related subject area. After the peer review process was complete, the editors decide that the papers will be accepted for publication. • Conference submission management system: Email 2nd International Conference on Fisheries and Marine submission on https://unkhair.ac.id/ • Number of submissions received: 150 • Number of submissions sent for review: 88 • Number of submissions accepted: 73 • Acceptance Rate (Number of Submissions Accepted/Number of Submissions Received X 100): 48% • Average number of reviews per paper: 2 • Total number of reviewers involved: 35 • Any additional info on review process: • Contact person for queries: [email protected] Dr. Najamuddin Department of Marine Science, Faculty of Fisheries and Marine Science, Khairun University, Indonesia

scholarly journals Peer review declaration

2022 ◽  
Vol 2161 (1) ◽  
pp. 011002
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Assistant Professor ◽  
Committee Chair ◽  
Programme Committee ◽  
Blind Review ◽  
Institute Of Technology ◽  
Scientific Standards ◽  
Single Blind

All papers published in this volume of Journal of Physics: Conference Series have been peer reviewed through processes administered by the Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a proceedings journal published by IOP Publishing. • Type of peer review: Single-blind All the articles of AICECS 2021 followed the “Single-blind” peer review process, where the reviewers were aware of the authors’ identity but not vice-versa • Conference submission management system: EasyChair All the submission and communication to all the AICECS 2021 authors were through EasyChair (https://easychair.org/conferences/?conf=aicecs2021) • Number of submissions received: 149 • Number of submissions sent for review: 136 • Number of submissions accepted: 78 • Acceptance Rate: 52.3% (Number of Submissions Accepted/Number of Submissions Received X 100): (78/149) x 100 = 52.3%) • Average number of reviews per paper: 2 • Total number of reviewers involved: 106 Any additional info on review process: All the initial submissions went through a single-blind review, based on the subject experts (reviewers) and Technical Programme Committee Chair (TPC) and General Chair decision (accept or major revision or minor revision or reject) communicated to the authors through EasyChair. Based on the recommendation, the authors revised the articles and submitted their revised papers. The revised submission was verified by the TPC and General Chair for their final recommendation for the submission. Online similarity check has been carried out using Turnitin software at all the stages from submission to acceptance. Contact person for queries: Name : Dr. Tanweer Assistant Professor-Senior Scale, Department of E&C, General Chair, AICECS 2021, Manipal Institute of Technology, Manipal, Karnataka, India Email : [email protected]; [email protected]

Sign in / Sign up

Export Citation Format

Share Document