Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

ObjectiveTo develop a tool to assess the quality of peer-review reports in biomedical research.MethodsWe conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis.ResultsA total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1–5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer’s comments).ConclusionAssessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers’ work, and also as an outcome when evaluating interventions to improve the peer-review process.

Download Full-text

Peering into peer review: Good quality reviews of research articles require neither writing too much nor taking too long

Transinformação ◽

10.1590/2318-08892018000200006 ◽

2018 ◽

Vol 30 (2) ◽

pp. 209-218 ◽

Cited By ~ 1

Author(s):

Paula CABEZAS Del FIERRO ◽

Omar SABAJ MERUANE ◽

Germán VARAS ESPINOZA ◽

Valeria GONZÁLEZ HERRERA

Keyword(s):

Peer Review ◽

Review Process ◽

Average Length ◽

Peer Review Process ◽

Research Articles ◽

Review Time ◽

Feedback Quality ◽

Review Reports ◽

The Relationship

Abstract The value of scientific knowledge is highly dependent on the quality of the process used to produce it, namely, the quality of the peer-review process. This process is a pivotal part of science as it works both to legitimize and improve the work of the scientific community. In this context, the present study investigated the relationship between review time, length, and feedback quality of review reports in the peer-review process of research articles. For this purpose, the review time of 313 referee reports from three Chilean international journals were recorded. Feedback quality was determined estimating the rate of direct requests by the total number of comments in each report. Number of words was used to describe the average length in the sample. Results showed that average time and length have little variation across review reports, irrespective of their quality. Low quality reports tended to take longer to reach the editor, so neither time nor length were related to feedback quality. This suggests that referees mostly describe, criticize, or praise the content of the article instead of making useful and direct comments to help authors improve their manuscripts.

Download Full-text

What is open peer review? A systematic review

F1000Research ◽

10.12688/f1000research.11369.1 ◽

2017 ◽

Vol 6 ◽

pp. 588 ◽

Cited By ~ 55

Author(s):

Tony Ross-Hellauer

Keyword(s):

Systematic Review ◽

Peer Review ◽

Peer Review Process ◽

Open Science ◽

Subject Area ◽

Open Review ◽

Broad Subject ◽

Review Reports ◽

The Many ◽

Definition Of

Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions. While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are then systematically analysed to build a coherent typology of the many different innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers a total of 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature. Conclusions: Based on this work, I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the ethos of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

Download Full-text

A retrospective study investigating requests for self-citation during open peer review in a general medicine journal

10.1101/2020.06.09.20126904 ◽

2020 ◽

Author(s):

Erin Peebles ◽

Marissa Scandlyn ◽

Blair R. Hesp

Keyword(s):

Peer Review ◽

Review Process ◽

Peer Review Process ◽

General Medicine ◽

Chi Square ◽

Exact Test ◽

Medicine Journal ◽

Objective Feedback ◽

Review Reports

AbstractIntroductionPeer review is a volunteer process for improving the quality of publications by providing objective feedback to authors, but also presents an opportunity for reviewers to seek personal reward by requesting self-citations. Open peer review may reduce the prevalence of self-citation requests and encourage author rebuttal over accession. This study aimed to investigate the prevalence of self-citation requests and their inclusion in manuscripts in a journal with open peer review.MethodsRequests for additional references to be included during peer review for articles published between 1 January 2017 and 31 December 2018 in BMC Medicine were evaluated. Data extracted included total number of self-citations requested, self-citations that were included in the final published manuscript and manuscripts that included at least one self-citation, and compared with corresponding data on independent citations.ResultsIn total, 932 peer review reports from 373 manuscripts were analysed. At least one additional citation was requested in 25.9% (n=241) of reports. Self-citation requests were included in 44.4% of reports requesting additional citations (11.5% of all reports). Requests for self-citation were significantly more likely than independent citations to be incorporated in the published manuscript (65.1% vs 52.1%; chi-square p=0.003). At the manuscript level, when requested, self-citations were incorporated in 76.6% of manuscripts (n=72; 19.3% of all manuscripts) compared with 68.5% of manuscripts with independent citation requests (n=102; 27.3% of manuscripts). A significant interaction was observed between the presence of self-citation requests and the likelihood of any citation request being incorporated (100% incorporation in manuscripts with self-citation requests alone versus 62.7–72.2% with any independent citation request; Fisher’s exact test p<0.0005).ConclusionsRequests for self-citations during the peer review process are common. The transparency of open peer review may have the unexpected effect of encouraging authors to incorporate self-citation requests by disclosing peer reviewer identity.

Download Full-text

Why Academics Choose to Publish in a Mega-Journal

Journal of Education and Learning ◽

10.5539/jel.v6n4p348 ◽

2017 ◽

Vol 6 (4) ◽

pp. 348

Author(s):

Jovan Shopovski ◽

Dejan Marolov

Keyword(s):

Open Access ◽

Peer Review ◽

Online Survey ◽

Peer Review Process ◽

Scientific Journal ◽

Review Procedure ◽

Peer Review Procedure ◽

Significant Factors

With their broad scope, high publishing volume, a peer review process based on the scientific soundness of the content, and an open access model, mega journals have become an important part of scholarly publishing.The main aim of this paper is to determine the most important factor that influenced researchers’ decisions to submit their academic work to these type of journal. To this end, an online survey has been disseminated from November 2016 to August 2017, targeting the corresponding authors of the European Scientific Journal, ESJ. Data from 413 corresponding authors was collected.The focus was mainly on how they discover the journal and what led them to submit a paper to the journal. However, questions concerning their satisfaction with the peer review procedure were also part of the survey.The results have shown that a recommendation of a colleague is not only the main channel through which authors found out about the journal, but is also the major reason they decided to submit their paper to a mega-journal. Furthermore, the quality of the editorial board of the journal, the strong portfolio of papers and the open access concept are also significant factors in encouraging submission to a mega-journal. A majority of the respondents are satisfied with the communication and peer review procedure of the mega-journal, which might encourage new submissions in the future.

Download Full-text

What is open peer review? A systematic review

F1000Research ◽

10.12688/f1000research.11369.2 ◽

2017 ◽

Vol 6 ◽

pp. 588 ◽

Cited By ~ 34

Author(s):

Tony Ross-Hellauer

Keyword(s):

Systematic Review ◽

Peer Review ◽

Peer Review Process ◽

Open Science ◽

Subject Area ◽

Open Review ◽

Rich Information ◽

Broad Subject ◽

Review Reports ◽

Definition Of

Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions: I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

Download Full-text

Rewarding the quantity of peer review could harm biomedical research

Biochemia Medica ◽

10.11613/bm.2019.020201 ◽

2019 ◽

Vol 29 (2) ◽

pp. 200-205

Author(s):

Aceil Al-Khatib ◽

Jaime A. Teixeira da Silva

Keyword(s):

Peer Review ◽

Biomedical Research ◽

Poor Quality ◽

Unintended Consequences ◽

Present Evidence ◽

Perverse Incentive ◽

Review Reports ◽

Biomedical Publications

Voluntary peer review is generally provided by researchers as a duty or service to their disciplines. They commit their expertise, knowledge and time freely without expecting rewards or compensation. Peer review can be perceived as a reciprocal mission that aims to safeguard the quality of publications by helping authors improve their manuscripts. While voluntary peer review adds value to research, rewarding the quantity or the volume of peer review is likely to lure academics into providing poor quality peer review. Consequently, the quantity of peer review may increase, but at the expense of quality, which may lead to unintended consequences and might negatively affect the quality of biomedical publications. This paper aims to present evidence that while voluntary peer review may aid researchers, pressurized peer review may create a perverse incentive that negatively affects the integrity of the biomedical research record. We closely examine one of the proposed models for rewarding peer review based on the quantity of peer review reports. This article also argues that peer review should remain a voluntary mission, and should not be prompted by the need to attain tenure or promotion.

Download Full-text

COVID-19 PANDEMIC AWARENESS AND ITS IMPACT ON BANGLADESHI PEOPLE: A COMMUNITY BASED SURVEY

Universal Journal of Pharmaceutical Research ◽

10.22270/ujpr.v6i6.699 ◽

2022 ◽

Author(s):

Md. Monirul Islam ◽

Abu Zobayed ◽

Md. Imran Nur Manik ◽

Mohammad Asadujjaman

Keyword(s):

Social Media ◽

Peer Review ◽

Urban Areas ◽

Online Survey ◽

Daily Lives ◽

Initial Stage ◽

Unique System ◽

Physical Interactions ◽

Review Reports

Background: The present study has been carried out to find out the people’s perception about the COVID-19 pandemic and the effects of this pandemic on their daily lives. Methods: A Google form was used to create an online survey questionnaire distributed via personal relationships and social media. Data were collected using questionnaires consisting of socio-demographic status of participants, COVID-19 concepts and changes in the participants’ lifestyle. A total of 1088 respondents took part in the survey, and the results were evaluated using several statistical approaches. The results were presented in a variety of tables and graphs. Results: Males made up 53.3% of the participants, while females made up 46.7% who were from villages (29.87%), cities (53.77%), and semi-urban areas (16.36%). Most participants reported that they got information about the COVID-19 pandemic from social media, and they cited 'fever' as an indicator of the sickness. Although the amount of physical interactions with friends and family has not dropped much, working conditions and employment status have significantly altered. The majority of the participants claimed they have no concerns about the COVID-19. Conclusion: For up-to-date information regarding COVID-19, people in Bangladesh from all walks of life rely largely on social media. COVID-19 awareness isn't enough to inhibit the spreading of the virus from community to community. Employees' work schedules have also been drastically altered. To prevent transmission, we should all follow the safety rules strictly. Peer Review History: Received: 4 November 2021; Revised: 17 December; Accepted: 28 December, Available online: 15 January 2022 Academic Editor: Dr. Muhammad Zahid Iqbal, AIMST University, Malaysia, [email protected] UJPR follows the most transparent and toughest ‘Advanced OPEN peer review’ system. The identity of the authors and, reviewers will be known to each other. This transparent process will help to eradicate any possible malicious/purposeful interference by any person (publishing staff, reviewer, editor, author, etc) during peer review. As a result of this unique system, all reviewers will get their due recognition and respect, once their names are published in the papers. We expect that, by publishing peer review reports with published papers, will be helpful to many authors for drafting their article according to the specifications. Auhors will remove any error of their article and they will improve their article(s) according to the previous reports displayed with published article(s). The main purpose of it is ‘to improve the quality of a candidate manuscript’. Our reviewers check the ‘strength and weakness of a manuscript honestly’. There will increase in the perfection, and transparency. Received file: Reviewer's Comments: Average Peer review marks at initial stage: 5.5/10 Average Peer review marks at publication stage: 7.0/10 Reviewers: Dr. Eyassu Mathewos, School of public health, college of health sciences and medicine, Wolaita Soddo university- P.O. Box 158, Wolaita Soddo, Ethiopia. Dr. Barkat Ali Khan, Kampala International University , Uganda, [email protected] Similar Articles: COVID-19: PHARMACOLOGICAL AND THERAPEUTIC APPROACHES COVID-19 IN MEXICO: PREPARING FOR FUTURE PANDEMICS

Download Full-text

Reference Conundrum: The Broad Definition of Peer-Review

10.31229/osf.io/6c3dt ◽

2018 ◽

Author(s):

Cody Fullerton

Keyword(s):

Peer Review ◽

Gold Standard ◽

Review Process ◽

Peer Review Process ◽

Academic Publishing ◽

Inaccurate Information ◽

The Core ◽

Literature Reviews ◽

Reference Interactions ◽

Definition Of

For years, the gold-standard in academic publishing has been the peer-review process, and for the most part, peer-review remains a safeguard to authors publishing intentionally biased, misleading, and inaccurate information. Its purpose is to hold researchers accountable to the publishing standards of that field, including proper methodology, accurate literature reviews, etc. This presentation will establish the core tenants of peer-review, discuss if certain types of publications should be able to qualify as such, offer possible solutions, and discuss how this affects a librarian's reference interactions.

Download Full-text

The Quality of Statistical Reporting and Data Presentation in Predatory Dental Journals Was Lower Than in Non-Predatory Journals

Entropy ◽

10.3390/e23040468 ◽

2021 ◽

Vol 23 (4) ◽

pp. 468

Author(s):

Pentti Nieminen ◽

Sergio E. Uribe

Keyword(s):

Open Access ◽

Peer Review ◽

Scientific Journals ◽

Mean Values ◽

Data Presentation ◽

Predatory Journals ◽

Study Results ◽

The Mean ◽

Highly Cited

Proper peer review and quality of published articles are often regarded as signs of reliable scientific journals. The aim of this study was to compare whether the quality of statistical reporting and data presentation differs among articles published in ‘predatory dental journals’ and in other dental journals. We evaluated 50 articles published in ‘predatory open access (OA) journals’ and 100 clinical trials published in legitimate dental journals between 2019 and 2020. The quality of statistical reporting and data presentation of each paper was assessed on a scale from 0 (poor) to 10 (high). The mean (SD) quality score of the statistical reporting and data presentation was 2.5 (1.4) for the predatory OA journals, 4.8 (1.8) for the legitimate OA journals, and 5.6 (1.8) for the more visible dental journals. The mean values differed significantly (p < 0.001). The quality of statistical reporting of clinical studies published in predatory journals was found to be lower than in open access and highly cited journals. This difference in quality is a wake-up call to consume study results critically. Poor statistical reporting indicates wider general lower quality in publications where the authors and journals are less likely to be critiqued by peer review.

Download Full-text

Peer Review Process in Scholarly Communication and Scientific Publishing

Nepalese Journal of Development and Rural Studies ◽

10.3126/njdrs.v17i0.34947 ◽

2020 ◽

Vol 17 ◽

pp. 15-19

Author(s):

Bishnu Bahadur Khatri

Keyword(s):

Peer Review ◽

Review Process ◽

Peer Review Process ◽

Scholarly Communication ◽

Scientific Research ◽

Early Career ◽

Scientific Publishing ◽

The Public ◽

Quality Control Mechanism

Peer review in scholarly communication and scientific publishing, in one form or another, has always been regarded as crucial to the reputation and reliability of scientific research. In the growing interest of scholarly research and publication, this paper tries to discuss about peer review process and its different types to communicate the early career researchers and academics.This paper has used the published and unpublished documents for information collection. It reveals that peer review places the reviewer, with the author, at the heart of scientific publishing. It is the system used to assess the quality of scientific research before it is published. Therefore, it concludes that peer review is used to advancing and testing scientific knowledgeas a quality control mechanism forscientists, publishers and the public.

Download Full-text