scholarly journals Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study

BMC Medicine ◽  
2019 ◽  
Vol 17 (1) ◽  
Author(s):  
Anthony Chauvin ◽  
Philippe Ravaud ◽  
David Moher ◽  
David Schriger ◽  
Sally Hopewell ◽  
...  
Keyword(s):  
Peer Review ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Early Career ◽  
Diagnostic Study ◽  
Quality Of Reporting ◽  
Cross Sectional ◽  
Link Type ◽  
The Mean

Abstract Background The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process. Methods We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9). Results The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal’s usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88–1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82–89] versus 20% [16–24] and in identifying a switch in primary outcome 61% [44–77] versus 11% [3–26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57–65] versus 77% [74–81] and to identify a switch in primary outcome 77% [67–86] versus 98% [92–100]. Conclusions Trained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting. Trial registration Clinical.Trials.govNCT03119376 (Registered April, 18, 2017).

scholarly journals Peer Review Process in Scholarly Communication and Scientific Publishing

2020 ◽  
Vol 17 ◽  
pp. 15-19
Author(s):  
Bishnu Bahadur Khatri
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Scholarly Communication ◽  
Scientific Research ◽  
Early Career ◽  
Scientific Publishing ◽  
The Public ◽  
Quality Control Mechanism

Peer review in scholarly communication and scientific publishing, in one form or another, has always been regarded as crucial to the reputation and reliability of scientific research. In the growing interest of scholarly research and publication, this paper tries to discuss about peer review process and its different types to communicate the early career researchers and academics.This paper has used the published and unpublished documents for information collection. It reveals that peer review places the reviewer, with the author, at the heart of scientific publishing. It is the system used to assess the quality of scientific research before it is published. Therefore, it concludes that peer review is used to advancing and testing scientific knowledgeas a quality control mechanism forscientists, publishers and the public.

scholarly journals RETRACTION:  How blockchain-timestamped protocols could improve the trustworthiness of medical science

F1000Research ◽  
2017 ◽  
Vol 6 ◽  
pp. 805
Author(s):  
Greg Irving ◽  
John Holden
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Medical Science ◽  
Link Type

At the request of the authors Greg Irving and John Holden, the article titled “How blockchain-timestamped protocols could improve the trustworthiness of medical science” has been retracted from F1000Research. The authors have taken this decision after considering the methodological concerns raised by a peer reviewer during the post-publication open peer review process. As the methodology has been deemed to be unreliable, the article is now retracted. This applies to all three versions of the article: Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 1; referees: 2 approved]. F1000Research 2016, 5:222 (doi: 10.12688/f1000research.8114.1) Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 2; referees: 3 approved]. F1000Research 2016, 5:222 (doi: 10.12688/f1000research.8114.2) Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 3; referees: 3 approved, 1 not approved]. F1000Research 2017, 5:222 (doi: 10.12688/f1000research.8114.3).

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2019 ◽  
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

The Reliability of Manuscript Evaluation for the South African Journal of Psychology

1993 ◽  
Vol 23 (1) ◽  
pp. 43-48 ◽  
Author(s):  
Cornells Plug
Keyword(s):  
Peer Review ◽  
South African ◽  
Review Process ◽  
Peer Review Process ◽  
High Status ◽  
South African Journal ◽  
The South ◽  
Weighted Mean ◽  
The Mean ◽  
African Journal

The peer review process is widely used to evaluate manuscripts for publication in scientific journals. Yet the reliability of recommendations about the suitability of manuscripts has been found to be quite low: the weighted mean of the single-reviewer reliability for a variety of journals is only 0,29. This paper describes a study of the evaluation of manuscripts submitted to the South African Journal of Psychology (SAJP) from 1988 to 1990. Single-reviewer reliability was found to be 0,34, slightly (but not significantly) above the mean for high status overseas journals. An investigation of other aspects of the review process indicates that peer review is useful to both authors and editors despite its shortcomings. Most of the recommended improvements of peer review suggested recently were incorporated into the SAJP's review process several years ago.

scholarly journals Perspectives for early-career researchers on plagiarism and scientific integrity

FACETS ◽  
2020 ◽  
Vol 5 (1) ◽  
pp. 17-25
Author(s):  
Robert G. Young ◽  
T. Fatima Mitterboeck
Keyword(s):  
Graduate Students ◽  
Intellectual Property ◽  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Scientific Investigation ◽  
Early Career ◽  
Scientific Integrity ◽  
Predatory Journals ◽  
Early Career Researchers

Lapses in scientific integrity, such as plagiarism, persist in the scientific realm. To be successful and contributory, early-career researchers (ECRs), including graduate students, need to be able to effectively navigate the literature, peer-review process, and scientific research with integrity. Here we discuss different aspects of scientific integrity related to ECRs. Our discussion centres on the concepts of plagiarism and intellectual property, predatory journals, aspects of peer review, transparency in publishing, and false advanced accreditations. Negative elements within these topics may be especially damaging to ECRs, who may be less familiar with the research landscape. We highlight the need for ECRs to approach scientific investigation cautiously and thoughtfully to promote integrity through critical thinking.

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2019 ◽  
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

scholarly journals How to publish your manuscript successfully (with Wiley)

2021 ◽  
Author(s):  
Matteo Cavalleri
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Early Career ◽  
Training Module ◽  
Early Career Researchers

Part of the training module on publishing with the Council of Australian University Librarians, this 1h webinar provides tips to Early Career Researchers on how to write and accompany an article through the peer-review process. Not specific to Wiley journals, but why go elsewhere, really?

Shadow Program Committee Initiative

2021 ◽  
Vol 46 (4) ◽  
pp. 16-18
Author(s):  
Patanamon Thongtanunam ◽  
Ayushi Rastogi ◽  
Foutse Khomh ◽  
Serge Demeyer ◽  
Meiyappan Nagappan ◽  
...  
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Research Community ◽  
Program Committee ◽  
Early Career ◽  
New Faculty ◽  
Great Chance ◽  
Good Review ◽  
Initiative Program

The Shadow Program Committee (PC) is an initiative/program that provides an opportunity to Early-Career Researchers (ECRs), i.e., PhD students, postdocs, new faculty members, and industry practitioners, who have not been in a PC, to learn rst-hand about the peer-review process of the technical track at Software Engi- neering (SE) conferences. This program aims to train the next generation of PC members as well as to allow ECRs to be recog- nized and embedded in the research community. By participating in this program, ECRs will have a great chance i) to gain expe- rience about the reviewing process including the restrictions and ethical standards of the academic peer-review process; ii) to be mentored by senior researchers on how to write a good review; and iii) to create a network with other ECRs and senior researchers (i.e., Shadow PC advisors). The Shadow PC program was rst introduced to the SE research community at the Mining Software Repositories (MSR) confer- ence in 2021. The program was led by Patanamon Thongta- nunam and Ayushi Rastogi (Shadow PC Co-chairs) with support from Shadow PC Advisor Co-Chairs (Foutse Khomh and Serge Demeyer), PC Co-Chairs of the technical track (Meiyappan Na- gappan and Kelly Blincoe), and the General Chair of the con- ference, Gregorio Robles. To promote and facilitate the Shadow PC program at SE conferences in the future, this report provides details about the process and a re ection on the Shadow PC pro- gram during MSR2021. The presentation slides and video are also available online at https://youtu.be/ReUXwmtIEk8.

scholarly journals Perspectives on involvement in the peer-review process: surveys of patient and public reviewers at two journals

BMJ Open ◽  
2018 ◽  
Vol 8 (9) ◽  
pp. e023357 ◽  
Author(s):  
Sara Schroter ◽  
Amy Price ◽  
Ella Flemyng ◽  
Andrew Demaine ◽  
Jim Elliot ◽  
...  
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Research Process ◽  
Annual Acknowledgement ◽  
Biomedical Literature ◽  
Free Access ◽  
Cross Sectional Survey ◽  
Cross Sectional ◽  
User Friendly

ObjectiveIn 2014/2015,The BMJandResearch Involvement and Engagement(RIE) became the first journals to routinely include patients and the public in the peer review process of journal articles. This survey explores the perspectives and early experiences of these reviewers.DesignA cross-sectional survey.Setting and participantsPatient and public reviewers forThe BMJandRIEwho have been invited to review.ResultsThe response rate was 69% (157/227) for those who had previously reviewed and 31% (67/217) for those who had not yet reviewed. Reviewers described being motivated to review by the opportunity to include the patient voice in the research process, influence the quality of the biomedical literature and ensure it meets the needs of patients. Of the 157 who had reviewed, 127 (81%) would recommend being a reviewer to other patients and carers. 144 (92%) thought more journals should adopt patient and public review. Few reviewers (16/224, 7%) reported concerns about doing open review. Annual acknowledgement on the journals’ websites was welcomed as was free access to journal information. Participants were keen to have access to more online resources and training to improve their reviewing skills. Suggestions on how to improve the reviewing experience included: allowing more time to review; better and more frequent communication; a more user-friendly process; improving guidance on how to review including videos; improving the matching of papers to reviewers’ experience; providing more varied sample reviews and brief feedback on the usefulness of reviews; developing a sense of community among reviewers; and publicising of the contribution that patient and public review brings.ConclusionsPatient and public reviewers shared practical ideas to improve the reviewing experience and these will be reviewed to enhance the guidance and support given to them.

Sign in / Sign up

Export Citation Format

Share Document