Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability.

2008 ◽  
Vol 63 (3) ◽  
pp. 160-168 ◽  
Author(s):  
Herbert W. Marsh ◽  
Upali W. Jayasinghe ◽  
Nigel W. Bond
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Grant Applications

scholarly journals Low agreement among reviewers evaluating the same NIH grant applications

2018 ◽  
Vol 115 (12) ◽  
pp. 2952-2957 ◽  
Author(s):  
Elizabeth L. Pier ◽  
Markus Brauer ◽  
Amarette Filut ◽  
Anna Kaatz ◽  
Joshua Raclaw ◽  
...  
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
National Institutes Of Health ◽  
Success Rates ◽  
Grant Applications ◽  
Peer Reviewers ◽  
Funding Success ◽  
Numeric Rating

Obtaining grant funding from the National Institutes of Health (NIH) is increasingly competitive, as funding success rates have declined over the past decade. To allocate relatively scarce funds, scientific peer reviewers must differentiate the very best applications from comparatively weaker ones. Despite the importance of this determination, little research has explored how reviewers assign ratings to the applications they review and whether there is consistency in the reviewers’ evaluation of the same application. Replicating all aspects of the NIH peer-review process, we examined 43 individual reviewers’ ratings and written critiques of the same group of 25 NIH grant applications. Results showed no agreement among reviewers regarding the quality of the applications in either their qualitative or quantitative evaluations. Although all reviewers received the same instructions on how to rate applications and format their written critiques, we also found no agreement in how reviewers “translated” a given number of strengths and weaknesses into a numeric rating. It appeared that the outcome of the grant review depended more on the reviewer to whom the grant was assigned than the research proposed in the grant. This research replicates the NIH peer-review process to examine in detail the qualitative and quantitative judgments of different reviewers examining the same application, and our results have broad relevance for scientific grant peer review.

scholarly journals Does Gender Matter in Grant Peer Review?

2012 ◽  
Vol 220 (2) ◽  
pp. 121-129 ◽  
Author(s):  
Rüdiger Mutz ◽  
Lutz Bornmann ◽  
Hans-Dieter Daniel
Keyword(s):  
Peer Review ◽  
Gender Bias ◽  
Review Process ◽  
Peer Review Process ◽  
Board Of Trustees ◽  
Final Decision ◽  
Current State ◽  
Grant Applications ◽  
State Of Research ◽  
Austrian Science Fund

One of the most frequently voiced criticisms of the peer review process is gender bias. In this study we evaluated the grant peer review process (external reviewers’ ratings, and board of trustees’ final decision: approval or no approval for funding) at the Austrian Science Fund with respect to gender. The data consisted of 8,496 research proposals (census) across all disciplines from 1999 to 2009, which were rated on a scale from 1 to 100 (poor to excellent) by 18,357 external reviewers in 23,977 reviews. In line with the current state of research, we found that the final decision was not associated with applicant’s gender or with any correspondence between gender of applicants and reviewers. However, the decisions on the grant applications showed a robust female reviewer salience effect. The approval probability decreases (up to 10%), when there is parity or a majority of women in the group of reviewers. Our results confirm an overall gender null hypothesis for the peer review process of men’s and women’s grant applications in contrast to claims that women’s grants are systematically downrated.

scholarly journals Quality Control through Peer Review Process in Scholarly Communication: Review of Related Literature

2017 ◽  
Vol 8 (3) ◽  
pp. 248
Author(s):  
Vishnu Kumar Gupta
Keyword(s):  
Quality Control ◽  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Scholarly Communication ◽  
Related Literature ◽  
Grant Applications ◽  
The Status ◽  
Peer Reviewers

<p>This review of related literature on the theme of peer review process in scholarly communication explains the status of research on periodicals, grant peer review and fellowships. The paper highlights the quality related issues of the scholarly communication and peer review process. Peer reviewers are invited to grant applications or assess fellowship or review manuscript in a peer review process undertake the responsibility for confirming top-level quality and standards in their concerned subject fields. <em></em></p>

scholarly journals A retrospective analysis of the peer review of more than 75,000 Marie Curie proposals between 2007 and 2018

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
David G Pina ◽  
Ivan Buljan ◽  
Darko Hren ◽  
Ana Marušić
Keyword(s):  
European Union ◽  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Evaluation Criteria ◽  
Marie Curie ◽  
The European Union ◽  
Grant Applications ◽  
Funding Agencies ◽  
Virtual Meetings

Most funding agencies rely on peer review to evaluate grant applications and proposals, but research into the use of this process by funding agencies has been limited. Here we explore if two changes to the organization of peer review for proposals submitted to various funding actions by the European Union has an influence on the outcome of the peer review process. Based on an analysis of more than 75,000 applications to three actions of the Marie Curie programme over a period of 12 years, we find that the changes – a reduction in the number of evaluation criteria used by reviewers and a move from in-person to virtual meetings – had little impact on the outcome of the peer review process. Our results indicate that other factors, such as the type of grant or area of research, have a larger impact on the outcome.

Getting Involved in the Peer Review Process

2008 ◽  
Author(s):  
Kenya Malcolm ◽  
Allison Groenendyk ◽  
Mary Cwik ◽  
Alisa Beyer
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process

Reference Conundrum: The Broad Definition of Peer-Review

2018 ◽  
Author(s):  
Cody Fullerton
Keyword(s):  
Peer Review ◽  
Gold Standard ◽  
Review Process ◽  
Peer Review Process ◽  
Academic Publishing ◽  
Inaccurate Information ◽  
The Core ◽  
Literature Reviews ◽  
Reference Interactions ◽  
Definition Of

For years, the gold-standard in academic publishing has been the peer-review process, and for the most part, peer-review remains a safeguard to authors publishing intentionally biased, misleading, and inaccurate information. Its purpose is to hold researchers accountable to the publishing standards of that field, including proper methodology, accurate literature reviews, etc. This presentation will establish the core tenants of peer-review, discuss if certain types of publications should be able to qualify as such, offer possible solutions, and discuss how this affects a librarian's reference interactions.

Sign in / Sign up

Export Citation Format

Share Document