Double-Blind Reviews | ScienceGate

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers’ biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each other’s identity and gender. To test the efficacy of double-blind reviews in one behavioral ecology journal (Behavioral Ecology, BE), we assigned gender to every authorship of every paper published for 2010–2018 in that journal compared to four other journals with single-blind reviews but similar subject matter and impact factors. While female authorships comprised only 35% of the total in all journals, the double-blind journal (BE) did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (BE and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis) for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review no longer benefits female authors and we discuss the pros and cons of the double-blind reviewing process based on our findings.

Download Full-text

Double blind reviews in software engineering venues: Practicability, promises and perils

10.7287/peerj.preprints.1757v1 ◽

2016 ◽

Author(s):

Moritz Beller ◽

Alberto Bacchelli

Keyword(s):

Software Engineering ◽

Scientific Method ◽

Review Process ◽

Peer Review Process ◽

Double Blind ◽

Blind Review ◽

Blind Reviews ◽

Hidden Biases ◽

Benefits And Costs ◽

Single Blind

The peer review process is central to the scientific method, the advancement and spread of research as well as crucial for individual careers. However, the single blind review process currently used in most Software Engineering (SE) venues is susceptible towards apparent and hidden biases, since reviewers know the identity of authors. In this paper, we perform a study on the benefits and costs that are associated with introducing double-blind reviews in SE venues. We surveyed the SE community’s opinion, interviewed experts on double-blind reviewing, and estimated the likelihood of reviewers being able to guess the authors. Our results indicate that double-blind reviewing could be introduced in large SE conferences at lower-than-generally believed costs and that the majority of the SE community is in favor of introducing it.

Download Full-text

The Case For and Against Double-blind Reviews

10.1101/495465 ◽

2018 ◽

Author(s):

Amelia R Cox ◽

Robert Montgomerie

Keyword(s):

Peer Review ◽

Gender Bias ◽

Behavioral Ecology ◽

Peer Review Process ◽

Double Blind ◽

Blind Review ◽

Female Authors ◽

Female Authorship ◽

Blind Reviews ◽

Single Blind

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers' unconscious biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each other's identities and genders. To test the efficacy of double-blind reviews, we assigned gender to every authorship of every paper published in 5 different journals with different peer review processes (double-blind vs. single blind) and subject matter (birds vs. behavioral ecology) from 2010-2018 (n = 4865 papers). While female authorships comprised only 35% of the total, the double-blind journal Behavioral Ecology did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (Behavioral Ecology and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis), for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review does not benefit female authors and may, in the long run, be detrimental.

Download Full-text

Single- or double-blind review? A field study of system preference, reliability, bias, and validity

10.31234/osf.io/q2tkw ◽

2021 ◽

Author(s):

Timothy Joseph Pleskac ◽

Ellie Kyung ◽

Gretchen B. Chapman ◽

Oleg Urminsky

Keyword(s):

Field Study ◽

Scientific Work ◽

Annual Conference ◽

High Stakes ◽

Double Blind ◽

Anna Karenina ◽

Blind Review ◽

Subsequent Publication ◽

Blind Reviews ◽

Single Blind

Many scientists experience a practice-preference gap about peer review. Single-blind review---where authors' identities are revealed to reviewers---is often used for evaluation. Yet, double-blind review---where authors' identities are concealed---is seen as more fair. To understand this gap, we compared both systems in a high-stakes field study: submissions to the Society for Judgment and Decision Making’s annual conference, the leading international conference on this topic. Each submission received both review types. Reviewers were randomly assigned to the review system and submissions. Selected conference talks were evaluated for quality, popularity, and subsequent publication status. We assessed the two systems on reliability, bias, and validity. On reliability, while both systems had moderate reliability, agreement was higher on what constituted a poor submission than a strong one (Anna Karenina Principle). On bias, double-blind reviews showed a slight bias against submission by women (Matilda Effect), while single-blind reviews showed a preference for submissions with senior co-authors (Matthew Effect). On validity, neither system predicted talk quality or popularity, but both predicted publication status. Author characteristics did not consistently predict outcomes. Thus, we suggest the costs of single-blind review do not outweigh its benefits. Yet, double-blind review is also not a perfect solution. We propose an equitable approach for selecting scientific work may be an informed lottery: use double-blind review to identify submissions of merit, then randomly choose from this set.

Download Full-text