scholarly journals Gender bias in scholarly peer review

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Markus Helmer ◽  
Manuel Schottdorf ◽  
Andreas Neef ◽  
Demian Battaglia
Keyword(s):  
Peer Review ◽  
Gender Bias ◽  
Review Process ◽  
Peer Review Process ◽  
Public Information ◽  
Scholarly Publishing ◽  
Gender Preference ◽  
Peer Reviewers ◽  
Same Gender

Peer review is the cornerstone of scholarly publishing and it is essential that peer reviewers are appointed on the basis of their expertise alone. However, it is difficult to check for any bias in the peer-review process because the identity of peer reviewers generally remains confidential. Here, using public information about the identities of 9000 editors and 43000 reviewers from the Frontiers series of journals, we show that women are underrepresented in the peer-review process, that editors of both genders operate with substantial same-gender preference (homophily), and that the mechanisms of this homophily are gender-dependent. We also show that homophily will persist even if numerical parity between genders is reached, highlighting the need for increased efforts to combat subtler forms of gender bias in scholarly publishing.

scholarly journals Evaluating, Improving, and Appreciating Peer Review at IJTMB

2021 ◽  
Vol 14 (1) ◽  
pp. 1-3
Author(s):  
Ann Blair Kennedy, LMT, BCTMB, DrPH
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Scientific Publishing ◽  
Publication Process ◽  
Improve Efficiency ◽  
Peer Reviewers

  Peer review is a mainstay of scientific publishing and, while peer reviewers and scientists report satisfaction with the process, peer review has not been without criticism. Within this editorial, the peer review process at the IJTMB is defined and explained. Further, seven steps are identified by the editors as a way to improve efficiency of the peer review and publication process. Those seven steps are: 1) Ask authors to submit possible reviewers; 2) Ask reviewers to update profiles; 3) Ask reviewers to “refer a friend”; 4) Thank reviewers regularly; 5) Ask published authors to review for the Journal; 6) Reduce the length of time to accept peer review invitation; and 7) Reduce requested time to complete peer review. We believe these small requests and changes can have a big effect on the quality of reviews and speed in which manuscripts are published. This manuscript will present instructions for completing peer review profiles. Finally, we more formally recognize and thank peer reviewers from 2018–2020.

scholarly journals The effects of an editor serving as one of the reviewers during the peer-review process

F1000Research ◽  
2016 ◽  
Vol 5 ◽  
pp. 683 ◽  
Author(s):  
Marco Giordan ◽  
Attila Csikasz-Nagy ◽  
Andrew M. Collings ◽  
Federico Vaggi
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Final Decision ◽  
Scientific Journals ◽  
Scientific Publications ◽  
Peer Reviewers ◽  
Decision Type ◽  
Number Of Citations ◽  
Decision Times

BackgroundPublishing in scientific journals is one of the most important ways in which scientists disseminate research to their peers and to the wider public. Pre-publication peer review underpins this process, but peer review is subject to various criticisms and is under pressure from growth in the number of scientific publications.MethodsHere we examine an element of the editorial process ateLife, in which the Reviewing Editor usually serves as one of the referees, to see what effect this has on decision times, decision type, and the number of citations. We analysed a dataset of 8,905 research submissions toeLifesince June 2012, of which 2,750 were sent for peer review, using R and Python to perform the statistical analysis.ResultsThe Reviewing Editor serving as one of the peer reviewers results in faster decision times on average, with the time to final decision ten days faster for accepted submissions (n=1,405) and 5 days faster for papers that were rejected after peer review (n=1,099). There was no effect on whether submissions were accepted or rejected, and a very small (but significant) effect on citation rates for published articles where the Reviewing Editor served as one of the peer reviewers.ConclusionsAn important aspect ofeLife’s peer-review process is shown to be effective, given that decision times are faster when the Reviewing Editor serves as a reviewer. Other journals hoping to improve decision times could consider adopting a similar approach.

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2019 ◽  
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

scholarly journals High adoption rates in a new preprint service from Springer Nature and Research Square

2019 ◽  
Author(s):  
Damian Pattinson
Keyword(s):  
Peer Review ◽  
Real Time ◽  
Biomedical Research ◽  
Review Process ◽  
Positive Impact ◽  
Peer Review Process ◽  
Manuscript Reviewer ◽  
Peer Reviewers ◽  
Gates Foundation

In recent years, funders have increased their support for early sharing of biomedical research through the use of preprints. For most, such as the COAlitionS group of funders (ASAPbio 2019) and the Gates foundation, this takes the form of active encouragement, while for others, it is mandated. But despite these motivations, few authors are routinely depositing their work as a preprint before submitting to a journal. Some journals have started offering authors the option of posting their work early at the point at which it is submitted for review. These include PLOS, who offer a link to BiorXiv, the Cell journals, who offer SSRN posting through ‘Sneak Peak’, and Nature Communications, who offer posting to any preprint and a link from the journal page called ‘Under Consideration’. Uptake has ranged from 3% for the Nature pilot, to 18% for PLOS (The Official Plos Blog 2018). In order to encourage more researchers to post their work early, we have been offering authors who submit to BMC Series titles the opportunity to post their work as a preprint on Research Square, a new platform that lets authors share and improve their research. To encourage participation, authors are offered a greater amount of control and transparency over the peer review process if they opt in. First, they are given a detailed peer review timeline which updates in real time every time an event occurs on their manuscript (reviewer invited, reviewer accepts etc). Second, they are encouraged to share their preprint with colleagues, who are able to post comments on the paper. These comments are sent to the editor when they are making their decision. Third, authors can suggest potential peer reviewers, recommendations which are also passed onto the editor to vet and invite. Together, these incentives have had a positive impact on authors choosing to post a preprint. Among the journals that offer this service, the average opt-in rate is 40%. This translates to over 3,000 manuscripts (as of July 2019) that have been posted to Research Square since the launch of the service in October 2018. In this talk I will demonstrate the functionality of Research Square, and provide demographic and discipline data on which areas are most and least likely to post.

scholarly journals The effects of an editor serving as one of the reviewers during the peer-review process

F1000Research ◽  
2016 ◽  
Vol 5 ◽  
pp. 683 ◽  
Author(s):  
Marco Giordan ◽  
Attila Csikasz-Nagy ◽  
Andrew M. Collings ◽  
Federico Vaggi
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Final Decision ◽  
Scientific Journals ◽  
Scientific Publications ◽  
Peer Reviewers ◽  
Decision Type ◽  
Number Of Citations ◽  
Decision Times

BackgroundPublishing in scientific journals is one of the most important ways in which scientists disseminate research to their peers and to the wider public. Pre-publication peer review underpins this process, but peer review is subject to various criticisms and is under pressure from growth in the number of scientific publications.MethodsHere we examine an element of the editorial process ateLife, in which the Reviewing Editor usually serves as one of the referees, to see what effect this has on decision times, decision type, and the number of citations. We analysed a dataset of 8,905 research submissions toeLifesince June 2012, of which 2,747 were sent for peer review. This subset of 2747 papers was then analysed in detail.  ResultsThe Reviewing Editor serving as one of the peer reviewers results in faster decision times on average, with the time to final decision ten days faster for accepted submissions (n=1,405) and five days faster for papers that were rejected after peer review (n=1,099). Moreover, editors acting as reviewers had no effect on whether submissions were accepted or rejected, and a very small (but significant) effect on citation rates.ConclusionsAn important aspect ofeLife’s peer-review process is shown to be effective, given that decision times are faster when the Reviewing Editor serves as a reviewer. Other journals hoping to improve decision times could consider adopting a similar approach.

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2019 ◽  
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

Point and Click Data: An Assessment of Editorial Perceptions and Recommendations for the Peer-Review Process in the New Data Frontier

2017 ◽  
Vol 33 (1) ◽  
pp. 129-144 ◽  
Author(s):  
Jay C. Thibodeau ◽  
L. Tyler Williams ◽  
Annie L. Witte
Keyword(s):  
Peer Review ◽  
Data Streams ◽  
Best Practice ◽  
Review Process ◽  
Peer Review Process ◽  
Data Integrity ◽  
Data Availability ◽  
Peer Reviewers ◽  
Online Labor ◽  
New Research

ABSTRACT In the new research frontier of data availability, this study develops guidelines to aid accounting academicians as they seek to evidence data integrity proactively in the peer-review process. To that end, we explore data integrity issues associated with two emerging data streams that are gaining prominence in the accounting literature: online labor markets and social media sources. We provide rich detail surrounding academic thought about these data platforms through interview data collected from a sample of former senior journal editors and survey data collected from a sample of peer reviewers. We then propound a set of best practice considerations that are designed to mitigate the perceived risks identified by our assessment.

scholarly journals Low agreement among reviewers evaluating the same NIH grant applications

2018 ◽  
Vol 115 (12) ◽  
pp. 2952-2957 ◽  
Author(s):  
Elizabeth L. Pier ◽  
Markus Brauer ◽  
Amarette Filut ◽  
Anna Kaatz ◽  
Joshua Raclaw ◽  
...  
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
National Institutes Of Health ◽  
Success Rates ◽  
Grant Applications ◽  
Peer Reviewers ◽  
Funding Success ◽  
Numeric Rating

Obtaining grant funding from the National Institutes of Health (NIH) is increasingly competitive, as funding success rates have declined over the past decade. To allocate relatively scarce funds, scientific peer reviewers must differentiate the very best applications from comparatively weaker ones. Despite the importance of this determination, little research has explored how reviewers assign ratings to the applications they review and whether there is consistency in the reviewers’ evaluation of the same application. Replicating all aspects of the NIH peer-review process, we examined 43 individual reviewers’ ratings and written critiques of the same group of 25 NIH grant applications. Results showed no agreement among reviewers regarding the quality of the applications in either their qualitative or quantitative evaluations. Although all reviewers received the same instructions on how to rate applications and format their written critiques, we also found no agreement in how reviewers “translated” a given number of strengths and weaknesses into a numeric rating. It appeared that the outcome of the grant review depended more on the reviewer to whom the grant was assigned than the research proposed in the grant. This research replicates the NIH peer-review process to examine in detail the qualitative and quantitative judgments of different reviewers examining the same application, and our results have broad relevance for scientific grant peer review.

scholarly journals Does Gender Matter in Grant Peer Review?

2012 ◽  
Vol 220 (2) ◽  
pp. 121-129 ◽  
Author(s):  
Rüdiger Mutz ◽  
Lutz Bornmann ◽  
Hans-Dieter Daniel
Keyword(s):  
Peer Review ◽  
Gender Bias ◽  
Review Process ◽  
Peer Review Process ◽  
Board Of Trustees ◽  
Final Decision ◽  
Current State ◽  
Grant Applications ◽  
State Of Research ◽  
Austrian Science Fund

One of the most frequently voiced criticisms of the peer review process is gender bias. In this study we evaluated the grant peer review process (external reviewers’ ratings, and board of trustees’ final decision: approval or no approval for funding) at the Austrian Science Fund with respect to gender. The data consisted of 8,496 research proposals (census) across all disciplines from 1999 to 2009, which were rated on a scale from 1 to 100 (poor to excellent) by 18,357 external reviewers in 23,977 reviews. In line with the current state of research, we found that the final decision was not associated with applicant’s gender or with any correspondence between gender of applicants and reviewers. However, the decisions on the grant applications showed a robust female reviewer salience effect. The approval probability decreases (up to 10%), when there is parity or a majority of women in the group of reviewers. Our results confirm an overall gender null hypothesis for the peer review process of men’s and women’s grant applications in contrast to claims that women’s grants are systematically downrated.

Sign in / Sign up

Export Citation Format

Share Document