Reviewers' perceptions of the peer review process for a medical education journal

Purpose Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. Results 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%). Conclusion Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors.

Download Full-text

Design and Implementation of a Peer-Reviewed Medical Education Video Competition: The Best of American Thoracic Society Video Lecture Series

Journal of Graduate Medical Education ◽

10.4300/jgme-d-19-00071.1 ◽

2019 ◽

Vol 11 (5) ◽

pp. 592-596 ◽

Cited By ~ 3

Author(s):

Nitin Seam ◽

Jeremy B. Richards ◽

Patricia A. Kritek ◽

Danai Khemasuwan ◽

Jennifer W. McCallister ◽

...

Keyword(s):

Medical Education ◽

Peer Review ◽

Health Care Providers ◽

Review Process ◽

Peer Review Process ◽

American Thoracic Society ◽

Care Providers ◽

Video Lecture ◽

Lecture Series ◽

Optimal Accuracy

ABSTRACT Background Video is an increasingly popular medium for consuming online content, and video-based education is effective for knowledge acquisition and development of technical skills. Despite the increased interest in and use of video in medical education, there remains a need to develop accurate and trusted collections of peer-reviewed videos for medical learners. Objective We developed the first professional society-based, open-access library of crowd-sourced and peer-reviewed educational videos for medical learners and health care providers. Methods A comprehensive peer-review process of medical education videos was designed, implemented, reviewed, and modified using a plan-do-study-act approach to ensure optimal accuracy and effective pedagogy, while emphasizing modern teaching methods and brevity. The number of submissions and views were tracked as metrics of interest and engagement of medical learners and educators. Results The Best of American Thoracic Society Video Lecture Series (BAVLS) was launched in 2016. Total video submissions for 2016, 2017, and 2018 were 26, 55, and 52, respectively. Revisions to the video peer-review process were made after each submission cycle. By 2017, the total views of BAVLS videos on www.thoracic.org and YouTube were 9100 and 17 499, respectively. By 2018, total views were 77 720 and 152 941, respectively. BAVLS has achieved global reach, with views from 89 countries. Conclusions The growth in submissions, content diversity, and viewership of BAVLS is a result of an intentional and evolving review process that emphasizes creativity and innovation in video-based pedagogy. BAVLS can serve as an example for developing institutional or society-based video platforms.

Download Full-text

Why Participate in Peer Review as a Journal Manuscript Reviewer: What’s in It for You?

Otolaryngology ◽

10.1177/0194599816669661 ◽

2016 ◽

Vol 156 (6) ◽

pp. 976-977 ◽

Cited By ~ 4

Author(s):

Kristen B. Pytynia

Keyword(s):

Medical Education ◽

Career Development ◽

Peer Review ◽

Subject Matter ◽

Continuing Medical Education ◽

Review Process ◽

Peer Review Process ◽

Scientific Journals ◽

Critical Component ◽

Education Credit

The peer review process for scientific journals relies on the efforts of volunteer reviewers. Reviewers are selected due to their expertise in their fields. With so many demands on professional time, the benefits of participating in peer review may not be obvious. However, reviewers benefit by exposure to the latest developments in their fields, facilitating their keeping up-to-date with the latest publications. Tenure committees look favorably on participation in peer review, and invitations to review underscore that the reviewer is a respected subject matter expert. Contacts made during the peer review process can lead to long-lasting collaboration. Continuing medical education credit can be obtained through various mechanisms. Overall, participating in peer review is an important part of career development and should be viewed as a critical component of advancement.

Download Full-text

Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors

10.1101/2021.07.28.454037 ◽

2021 ◽

Author(s):

Bridget C O'Brien ◽

Anthony R Artino ◽

Joseph A Costello ◽

Erik Driessen ◽

Lauren A Maggio

Keyword(s):

Peer Review ◽

Review Process ◽

Qualitative Content Analysis ◽

Peer Review Process ◽

Argument Quality ◽

Blind Review ◽

Coding Scheme ◽

Explicit Reference ◽

Education Journal ◽

Single Blind

Purpose: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer review process. Peer review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods: Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended "reject," we coded for alignment between reviewers' comments to the editor and to authors. We report descriptive statistics. Results: 49% of reviews contained comments to the editor (n=176). Most of these comments summarized the reviewers' impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 48 reviews recommending "reject," the majority of comments to the editor contained content that also appeared in comments to the authors (65%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (85%). Conclusion: Findings indicate variability in how reviewers use the confidential comments to editor section in online peer review systems, though generally the way they use them suggests integrity and transparency to authors.

Download Full-text