scholarly journals Supporting robust, rigorous, and reliable reviewing as the cornerstone of our profession: Introducing a competency framework for peer review

2020 ◽  
Vol 13 (1) ◽  
pp. 1-27 ◽  
Author(s):  
Tine Köhler ◽  
M. Gloria González-Morales ◽  
George C. Banks ◽  
Ernest H. O’Boyle ◽  
Joseph A. Allen ◽  
...  
Keyword(s):  
Peer Review ◽  
Competency Framework ◽  
Academic Publishing ◽  
Formal Training ◽  
Peer Reviews ◽  
University Departments ◽  
Peer Reviewing ◽  
Definition Of ◽  
Quality Of Research

AbstractPeer review is a critical component toward facilitating a robust science in industrial and organizational (I-O) psychology. Peer review exists beyond academic publishing in organizations, university departments, grant agencies, classrooms, and many more work contexts. Reviewers are responsible for judging the quality of research conducted and submitted for evaluation. Furthermore, they are responsible for treating authors and their work with respect, in a supportive and developmental manner. Given its central role in our profession, it is curious that we do not have formalized review guidelines or standards and that most of us never receive formal training in peer reviewing. To support this endeavor, we are proposing a competency framework for peer review. The purpose of the competency framework is to provide a definition of excellent peer reviewing and guidelines to reviewers for which types of behaviors will lead to good peer reviews. By defining these competencies, we create clarity around expectations for peer review, standards for good peer reviews, and opportunities for training the behaviors required to deliver good peer reviews. We further discuss how the competency framework can be used to improve peer reviewing and suggest additional steps forward that involve suggestions for how stakeholders can get involved in fostering high-quality peer reviewing.

A Study of Medical Board Peer Reviews in Nevada

2010 ◽  
Vol 96 (1) ◽  
pp. 20-29
Author(s):  
Jerry C. Calvanese
Keyword(s):  
Peer Review ◽  
Board Members ◽  
Study Objective ◽  
Clark County ◽  
Peer Reviews ◽  
Medical Examiners ◽  
Washoe County ◽  
Total Average Cost ◽  
Total Average

ABSTRACT Study Objective: The purpose of this study was to obtain data on various characteristics of peer reviews. These reviews were performed for the Nevada State Board of Medical Examiners (NSBME) to assess physician licensees' negligence and/or incompetence. It was hoped that this data could help identify and define certain characteristics of peer reviews. Methods: This study examined two years of data collected on peer reviews. The complaints were initially screened by a medical reviewer and/or a committee composed of Board members to assess the need for a peer review. Data was then collected from the peer reviews performed. The data included costs, specialty of the peer reviewer, location of the peer reviewer, and timeliness of the peer reviews. Results: During the two-year study, 102 peer reviews were evaluated. Sixty-nine percent of the peer-reviewed complaints originated from civil malpractice cases and 15% originated from complaints made by patients. Eighty percent of the complaint physicians were located in Clark County and 12% were located in Washoe County. Sixty-one percent of the physicians who performed the peer reviews were located in Washoe County and 24% were located in Clark County. Twelve percent of the complaint physicians were in practice in the state for 5 years or less, 40% from 6 to 10 years, 20% from 11 to 15 years, 16% from 16 to 20 years, and 13% were in practice 21 years or more. Forty-seven percent of the complaint physicians had three or less total complaints filed with the Board, 10% had four to six complaints, 17% had 7 to 10 complaints, and 26% had 11 or more complaints. The overall quality of peer reviews was judged to be good or excellent in 96% of the reviews. A finding of malpractice was found in 42% of the reviews ordered by the medical reviewer and in 15% ordered by the Investigative Committees. There was a finding of malpractice in 38% of the overall total of peer reviews. The total average cost of a peer review was $791. In 47% of the peer reviews requested, materials were sent from the Board to the peer reviewer within 60 days of the original request and 33% took more than 120 days for the request to be sent. In 48% of the reviews, the total time for the peer review to be performed by the peer reviewer was less than 60 days. Twenty seven percent of the peer reviews took more than 120 days to be returned. Conclusion: Further data is needed to draw meaningful conclusions from certain peer review characteristics reported in this study. However, useful data was obtained regarding timeliness in sending out peer review materials, total times for the peer reviews, and costs.

Reference Conundrum: The Broad Definition of Peer-Review

2018 ◽  
Author(s):  
Cody Fullerton
Keyword(s):  
Peer Review ◽  
Gold Standard ◽  
Review Process ◽  
Peer Review Process ◽  
Academic Publishing ◽  
Inaccurate Information ◽  
The Core ◽  
Literature Reviews ◽  
Reference Interactions ◽  
Definition Of

For years, the gold-standard in academic publishing has been the peer-review process, and for the most part, peer-review remains a safeguard to authors publishing intentionally biased, misleading, and inaccurate information. Its purpose is to hold researchers accountable to the publishing standards of that field, including proper methodology, accurate literature reviews, etc. This presentation will establish the core tenants of peer-review, discuss if certain types of publications should be able to qualify as such, offer possible solutions, and discuss how this affects a librarian's reference interactions.

Implementing Teaching Portfolios and Peer Reviews in Tax Courses

1999 ◽  
Vol 21 (2) ◽  
pp. 95-107 ◽  
Author(s):  
Michael J. Calegari ◽  
Gregory G. Geisler ◽  
Ernest R. Larkins
Keyword(s):  
Peer Review ◽  
Public University ◽  
Written Feedback ◽  
Writing Assignments ◽  
Peer Reviews ◽  
Extant Literature ◽  
Research Publications ◽  
Teaching Portfolio ◽  
Course Syllabus

Extant literature suggests that the process of constructing a teaching portfolio can identify areas to improve, motivate positive changes, and elevate the importance of teaching in academe. This study describes the experience of the tax faculty at a public university in using teaching portfolios and peer reviews to improve the quality of the first two tax courses. The type of teaching portfolio used in this project consists of a course syllabus and a reflective statement that documents the rationale for all components of a course (i.e., lectures, projects, exams, writing assignments, presentations, etc.). The peer review aspect involves written feedback from a colleague on this teaching portfolio. Though research publications are usually subject to extensive peer review, teaching generally is not. Like research, however, teaching can be evaluated and ultimately improved through peer review. Thus, this study can provide valuable guidance to tax professors attempting to improve their courses.

Whose News Can You Trust?

2014 ◽  
pp. 324-352
Author(s):  
Rick Malleus
Keyword(s):  
Peer Review ◽  
Theoretical Framework ◽  
Online News ◽  
News Sources ◽  
Definition Of ◽  
Online News Sites

This chapter proposes a framework for analyzing the credibility of online news sites, allowing diaspora populations to evaluate the credibility of online news about their home countries. A definition of credibility is established as a theoretical framework for analysis, and a framework of seven elements is developed based on the following elements: accuracy, authority, believability, quality of message construction, peer review, comparison, and corroboration. Later, those elements are applied to a variety of online news sources available to the Zimbabwean diaspora that serves as a case study for explaining the framework. The chapter concludes with a discussion of the framework in relation to some contextual circumstances of diaspora populations and presents some limitations of the framework as diaspora populations might actually apply the different elements.

scholarly journals Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

BMJ Open ◽  
2020 ◽  
Vol 10 (6) ◽  
pp. e035604
Author(s):  
Cecilia Superchi ◽  
Darko Hren ◽  
David Blanco ◽  
Roser Rius ◽  
Alessandro Recchioni ◽  
...  
Keyword(s):  
Peer Review ◽  
Biomedical Research ◽  
Online Survey ◽  
Peer Review Process ◽  
Final Version ◽  
Report Quality ◽  
Study Results ◽  
Review Reports ◽  
Definition Of

ObjectiveTo develop a tool to assess the quality of peer-review reports in biomedical research.MethodsWe conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis.ResultsA total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments).ConclusionAssessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer-review process.

Group Peer Review

2018 ◽  
Vol 11 (4) ◽  
pp. 452-466 ◽  
Author(s):  
Gail Teachman ◽  
Martine C. Lévesque ◽  
Mark Tambe Keboa ◽  
Basem Adel Danish ◽  
Konstantinos Mastorakis ◽  
...  
Keyword(s):  
Qualitative Research ◽  
Research Training ◽  
Writing Strategies ◽  
Ongoing Debate ◽  
Postgraduate Students ◽  
Peer Reviews ◽  
Constructive Feedback ◽  
Peer Reviewing ◽  
First Time

Invitation to review a manuscript for publication marks an exciting milestone in graduate and postgraduate students’ training. Yet, peer reviewing is seldom explicitly taught. First-time reviewers approaching this task often lack mentorship, guidelines, and confidence. The ongoing debate about how to judge the quality of qualitative research can further complicate the task. In this article, we introduce an innovative model for conducting group peer reviews in the context of qualitative research training. After setting out the model's principles and process, we discuss its merits and reflect on our experiences as trainees and supervisor using the approach. In addition to providing opportunities for mentorship in appraisal methods, writing strategies, and approaches for framing constructive feedback, we suggest the model has the potential to advance trainees’ development as collegial peers and their overall learning as qualitative researchers. Finally, we discuss potential ways forward to extend the model in other contexts.

Political Science and the Gross Intellectual Product

1986 ◽  
Vol 21 (4) ◽  
pp. 396-405 ◽  
Author(s):  
Kenneth Minogue
Keyword(s):  
Political Science ◽  
Publication Rate ◽  
University Departments ◽  
Political Studies ◽  
The University ◽  
Intellectual Product ◽  
Quality Of Research

BRITISH POLITICAL SCIENTISTS HAVE RECENTLY AWAKENED from their pragmatic slumbers to discover a dragon loose among them: the dragon of evaluation. The University Grants Committee has just been grading the quality of research in university departments, and some have been found ‘below average’. (To be merely ‘average’ hardly seems much better.) How were these gradings arrived at? We do not know, but we do know that among the documents available to the committee was a paper by Ivor Crewe analysing the publications record of all British departments of politics over a six-year period from January 1978. (It will be published in Political Studies next year.) Professor Crewe's paper reveals, to no one's surprise, that there are immense differences in publication rate both between individuals and between different departments; his revelation will not enhance his popularity. But it is clear that these events are just the beginning. The dragon of evaluation is on the loose and there's no St George in sight.

scholarly journals Criteria for Peer Reviewing the Original Articles Submitted to a Biomedical Journal: Intellectual Honesty is the Best Policy.

2004 ◽  
Vol 43 (152) ◽  
pp. 103-110
Author(s):  
Bishnu Hari Paudel
Keyword(s):  
Peer Review ◽  
Biomedical Science ◽  
Quality Review ◽  
Peer Reviewing ◽  
Biomedical Publication ◽  
Timely Review ◽  
Peer Reviewers ◽  
Biomedical Publications ◽  
Review Criteria

Peer review - a process of assessing the quality of manuscripts submitted to a journal – is an establishednorm in biomedical publications. It is viewed as an extension of scientific process. The peer-reviewed researcharticles are considered trustworthy because they are believed to be unbiased and independent. The processof reviewing is a privilege and prestige. It is highly responsible, intellectually honest, and difficult job.Being expert in certain area of biomedical science is a prerequisite for reviewers. Young peer reviewerstrained in epidemiology or statistics produce high-quality review. The International Congresses on PeerReview in Biomedical Publication have shown many unresolved issues related to preparation or handling ofmanuscripts by a journal. Therefore, it is vital to identify authentic peer reviewers to ensure qualitypublication, thus, a set of peer review criteria is proposed for peer reviewing original articles. It is useful inquantifying (scoring) the manuscript quality. The proposed scoring system yields three categories ofmanuscripts: the first category is considered acceptable for publication after minor modification by editorialboard and/or reviewers, the second – requires rewriting and resubmission, and the third – rejected. Thesecriteria are preliminary guidelines, and require timely review. They are expected to sensitise peer reviewers,editors, contributors, and readers to move towards greater honesty and responsibility while working withmanuscripts. In summary, if the criteria are used they will facilitate editorial management of manuscripts,render more justice to authors and biomedical science, and improve publication quality.Key Words: Biomedical publication, peer review, peer review criteria, scoring of manuscripts, categories of manuscripts, journal of Nepal Medical Association.

Theory of Mediation as a Branch of Knowledge

2019 ◽  
pp. 63-71
Author(s):  
Михаил Козюк ◽  
Mikhail Kozyuk
Keyword(s):  
Social Science ◽  
Social Relations ◽  
Point Of View ◽  
Mediation Theory ◽  
Close Interaction ◽  
Social Technologies ◽  
First Pass ◽  
Definition Of ◽  
Quality Of Research

Today, the national social science features fast-paced studies on the theory of mediation. However, the branch status of this area is still obscure, which affects the quality of research. The paper explores various approaches to the definition of this status with a critical eye. Particular attention is paid to the thesis on the interdisciplinarity of the mediation theory. The author shows the consequences of classifying the mediation theory as an interdisciplinary branch. It is also troublesome, from the author’s point of view, to refer mediation purely to law. Mediation refers to phenomena that originate from social technologies ordering social relations and removing social conflicts. Only sociology and history can become the foundation of a new scientific branch. However, the mediation theory must first pass the stage of disciplinary constitutionalization, since its close interaction with many modern branches of knowledge would mean dissolution in them.

Sign in / Sign up

Export Citation Format

Share Document