Peer Review or Lottery? A Critical Analysis of Two Different Forms of Decision-making Mechanisms for Allocation of Research Grants

2019 ◽  
Vol 44 (6) ◽  
pp. 994-1019 ◽  
Author(s):  
Lambros Roumbanis
Keyword(s):  
Decision Making ◽  
Peer Review ◽  
Review Process ◽  
Selection Process ◽  
Peer Review Process ◽  
Dynamic Selection ◽  
Funding Agencies ◽  
Epistemic Diversity ◽  
Alternative Decision ◽  
Made In

At present, peer review is the most common method used by funding agencies to make decisions about resource allocation. But how reliable, efficient, and fair is it in practice? The ex ante evaluation of scientific novelty is a fundamentally uncertain endeavor; bias and chance are embedded in the final outcome. In the current study, I will examine some of the most central problems of peer review and highlight the possible benefits of using a lottery as an alternative decision-making mechanism. Lotteries are driven by chance, not reason. The argument made in the study is that the epistemic landscape could benefit in several respects by using a lottery, thus avoiding all types of bias, disagreement, and other limitations associated with the peer review process. Funding agencies could form a pool of funding applicants who have minimal qualification levels and then select randomly within that pool. The benefits of a lottery would not only be that it saves time and resources, but also that it contributes to a more dynamic selection process and increases the epistemic diversity, fairness, and impartiality within academia.

Creation of a palliative care advanced practice registered nurse peer-review process

2019 ◽  
Vol 25 (12) ◽  
pp. 596-602
Author(s):  
Patrice R. Fedel ◽  
Nicole E. Hembel ◽  
Lindsey M. Mueller
Keyword(s):  
Decision Making ◽  
Palliative Care ◽  
Peer Review ◽  
Professional Growth ◽  
Review Process ◽  
Clinical Decision Making ◽  
Peer Review Process ◽  
Clinical Decision ◽  
Advanced Practice ◽  
Case Review

The mark of a true profession is the ability to self-regulate. As such, advanced practice registered nurses (APRNs) are challenged by their professional organisations to participate in self-evaluation and peer review. Peer review is a method for evaluating the care provided by the APRN to both ensure quality nursing care and promote professional growth. Despite guidelines to participate in a formal peer-review process, there is little information within the nursing profession on how to accomplish peer review. A comprehensive literature review failed to provide a framework for peer review that is practice focused, fosters a learning environment and encompasses the thought process and clinical decision-making of the palliative care advanced practice nurse. A group of palliative care APRNs set out to create a process that encompassed the breadth of clinical decision-making in palliative care advanced nursing practice. Using the eight domains of palliative care, a narrative case review structure was created. The resulting process both assisted the APRNs in professional growth and provided timely feedback for the annual performance evaluation.

scholarly journals Evaluation of stakeholder views on peer review of NIHR applications for funding: a qualitative study

BMJ Open ◽  
2018 ◽  
Vol 8 (12) ◽  
pp. e022548 ◽  
Author(s):  
Sheila Turner ◽  
Abby Bull ◽  
Fay Chinnery ◽  
Jeremy Hinks ◽  
Nicola Mcardle ◽  
...  
Keyword(s):  
Peer Review ◽  
Research Funding ◽  
Review Process ◽  
Selection Process ◽  
Peer Review Process ◽  
Qualitative Interviews ◽  
Board Members ◽  
Process Improvements ◽  
Peer Reviewers ◽  
The Uk

ObjectivesInnovations resulting from research have both national and global impact, so selecting the most promising research studies to fund is crucial. Peer review of research funding applications is part of the selection process, and requires considerable resources. This study aimed to elicit stakeholder opinions about which factors contribute to and influence effective peer review of funding applications to the UK National Institute for Health Research (NIHR), and to identify possible minor improvements to current processes and any major changes or potential innovations to achieve a more efficient peer review process.DesignQualitative interviews with 30 stakeholders involved in the peer review process.ParticipantsParticipants were drawn from three NIHR coordinating centres and represented four types of stakeholders: board members with responsibility for making funding decisions, applicants, external peer reviewers and NIHR staff.MethodsAll interviews were conducted by telephone apart from three that were face to face with NIHR staff. Data were analysed using a thematic template method.ResultsThe responses from NIHR staff, board members and reviewers differed from those received from applicants. The first three groups focused on how well the process of peer review did or did not function. The applicants mentioned these points but in addition often reflected on how their personal application was assessed. Process improvements suggested included: developing a more proportionate review process; providing greater guidance, feedback, training, acknowledgement or incentives for peer reviewers; reducing the time commitment and amount of paperwork; and asking reviewers to comment on the importance, strengths and weaknesses of applications and flaws which are potentially ‘fixable’.ConclusionsOverall, participants were supportive of the need for peer review in evaluating applications for research funding. This study revealed which parts of the process are working well and are valued, and barriers, difficulties and potential areas for improvement and development.

scholarly journals A retrospective analysis of the peer review of more than 75,000 Marie Curie proposals between 2007 and 2018

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
David G Pina ◽  
Ivan Buljan ◽  
Darko Hren ◽  
Ana Marušić
Keyword(s):  
European Union ◽  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Evaluation Criteria ◽  
Marie Curie ◽  
The European Union ◽  
Grant Applications ◽  
Funding Agencies ◽  
Virtual Meetings

Most funding agencies rely on peer review to evaluate grant applications and proposals, but research into the use of this process by funding agencies has been limited. Here we explore if two changes to the organization of peer review for proposals submitted to various funding actions by the European Union has an influence on the outcome of the peer review process. Based on an analysis of more than 75,000 applications to three actions of the Marie Curie programme over a period of 12 years, we find that the changes – a reduction in the number of evaluation criteria used by reviewers and a move from in-person to virtual meetings – had little impact on the outcome of the peer review process. Our results indicate that other factors, such as the type of grant or area of research, have a larger impact on the outcome.

Getting Involved in the Peer Review Process

2008 ◽  
Author(s):  
Kenya Malcolm ◽  
Allison Groenendyk ◽  
Mary Cwik ◽  
Alisa Beyer
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process

Reference Conundrum: The Broad Definition of Peer-Review

2018 ◽  
Author(s):  
Cody Fullerton
Keyword(s):  
Peer Review ◽  
Gold Standard ◽  
Review Process ◽  
Peer Review Process ◽  
Academic Publishing ◽  
Inaccurate Information ◽  
The Core ◽  
Literature Reviews ◽  
Reference Interactions ◽  
Definition Of

For years, the gold-standard in academic publishing has been the peer-review process, and for the most part, peer-review remains a safeguard to authors publishing intentionally biased, misleading, and inaccurate information. Its purpose is to hold researchers accountable to the publishing standards of that field, including proper methodology, accurate literature reviews, etc. This presentation will establish the core tenants of peer-review, discuss if certain types of publications should be able to qualify as such, offer possible solutions, and discuss how this affects a librarian's reference interactions.

Sign in / Sign up

Export Citation Format

Share Document