scholarly journals Announcing a New Registered Report Article Type at the Journal of Speech, Language, and Hearing Research

2022 ◽  
pp. 1-4
Author(s):  
Holly L. Storkel ◽  
Frederick J. Gallun
Keyword(s):  
Peer Review ◽  
Publication Bias ◽  
Peer Review Process ◽  
Research Integrity ◽  
Research Practices ◽  
Scientific Publications ◽  
Hearing Research ◽  
Questionable Research Practices ◽  
Structural Problems ◽  
Article Type

Purpose: This editorial introduces the new registered reports article type for the Journal of Speech, Language, and Hearing Research . The goal of registered reports is to create a structural solution to address issues of publication bias toward results that are unexpected and sensational, questionable research practices that are used to produce novel results, and a peer-review process that occurs at the end of the research process when changes in fundamental design are difficult or impossible to implement. Conclusion: Registered reports can be a positive addition to scientific publications by addressing issues of publication bias, questionable research practices, and the late influence of peer review. This article type does so by requiring reviewers and authors to agree in advance that the experimental design is solid, the questions are interesting, and the results will be publishable regardless of the outcome. This procedure ensures that replication studies and null results make it into the published literature and that authors are not incentivized to alter their analyses based on the results that they obtain. Registered reports represent an ongoing commitment to research integrity and finding structural solutions to structural problems inherent in a research and publishing landscape in which publications are such a high-stakes aspect of individual and institutional success.

Publication bias and statistical power in gerontological psychology

2018 ◽  
Author(s):  
Christopher Brydges
Keyword(s):  
Publication Bias ◽  
Statistical Power ◽  
Research Practices ◽  
P Values ◽  
Questionable Research Practices ◽  
Gerontological Research ◽  
Article Type ◽  
Research Findings ◽  
Psychological Journal ◽  
Published Research

Objectives: Research has found evidence of publication bias, questionable research practices (QRPs), and low statistical power in published psychological journal articles. Isaacowitz’s (2018) editorial in the Journals of Gerontology Series B, Psychological Sciences called for investigation of these issues in gerontological research. The current study presents meta-research findings based on published research to explore if there is evidence of these practices in gerontological research. Method: 14,481 test statistics and p values were extracted from articles published in eight top gerontological psychology journals since 2000. Frequentist and Bayesian caliper tests were used to test for publication bias and QRPs (specifically, p-hacking and incorrect rounding of p values). A z-curve analysis was used to estimate average statistical power across studies.Results: Strong evidence of publication bias was observed, and average statistical power was approximately .70 – below the recommended .80 level. Evidence of p-hacking was mixed. Evidence of incorrect rounding of p values was inconclusive.Discussion: Gerontological research is not immune to publication bias, QRPs, and low statistical power. Researchers, journals, institutions, and funding bodies are encouraged to adopt open and transparent research practices, and using Registered Reports as an alternative article type to minimize publication bias and QRPs, and increase statistical power.

Skin in the Game: Personal Accountability and Journal Peer Review

2020 ◽  
Vol 15 (4) ◽  
pp. 330-338
Author(s):  
Donald F. Sacco ◽  
Samuel V. Bruton ◽  
Mitch Brown ◽  
Mary M. Medlin
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Motivational Factors ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Blind Review ◽  
Tentative Evidence ◽  
Open Review ◽  
Personal Accountability

Two preregistered studies explored the likelihood paper reviewers would request clarification from authors regarding potential questionable research practices (QRPs). Study 1 participants were instructed to imagine reviewing a journal manuscript as either a coauthor or peer reviewer and rate the extent to which they would request clarification from the author when encountering potential QRPs. Participants reported greater likelihood of requesting clarification when assigned to the coauthor relative to the peer reviewer role. Study 2 participants were assigned to either an anonymous or open-review condition and rated the extent to which they would seek clarification from an author regarding potential QRPs. Men (but not women) in the open review condition reported greater likelihood of seeking clarification about potential QRPs than men in the blind review condition. Results provide tentative evidence that motivational factors influence the peer review process, and suggestions are made for improving peer review practices.

Judges and defendants

2018 ◽  
Author(s):  
Gianfranco Pacchioni
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Social Cost ◽  
Peer Review Process ◽  
Scientific Journals ◽  
Scientific Publications ◽  
Contemporary Science ◽  
The Social ◽  
Pros And Cons ◽  
Scientific Publishers

This chapter explores how validation of new results works in science. It also looks at the peer-review process, both pros and cons, as well as scientific communication, scientific journals, and scientific publishers. We give an assessment of the total number of existing journals with peer review. Other topics discussed include the phenomenon of open access, predatory journals and their impact on contemporary science, and the market of scientific publications. Finally, we touch on degenerative phenomena, such as the market of co-authors, bogus papers, and irrelevant and wrong studies, as well as the problem and the social cost of irreproducible results.

scholarly journals Excess success in “Don’t count calorie labeling out: Calorie counts on the left side of menu items lead to lower calorie food choices”

2019 ◽  
Author(s):  
Gregory Francis ◽  
Evelina Thunell
Keyword(s):  
Success Rate ◽  
Publication Bias ◽  
Food Choices ◽  
Effect Sizes ◽  
Sample Sizes ◽  
Research Practices ◽  
Policy Makers ◽  
Questionable Research Practices ◽  
Calorie Labeling

Based on findings from six experiments, Dallas, Liu & Ubel (2019) concluded that placing calorie labels to the left of menu items influences consumers to choose lower calorie food options. Contrary to previously reported findings, they suggested that calorie labels do influence food choices, but only when placed to the left because they are in this case read first. If true, these findings have important implications for the design of menus and may help address the obesity pandemic. However, an analysis of the reported results indicates that they seem too good to be true. We show that if the effect sizes in Dallas et al. (2019) are representative of the populations, a replication of the six studies (with the same sample sizes) has a probability of only 0.014 of producing uniformly significant outcomes. Such a low success rate suggests that the original findings might be the result of questionable research practices or publication bias. We therefore caution readers and policy makers to be skeptical about the results and conclusions reported by Dallas et al. (2019).

scholarly journals The effects of an editor serving as one of the reviewers during the peer-review process

F1000Research ◽  
2016 ◽  
Vol 5 ◽  
pp. 683 ◽  
Author(s):  
Marco Giordan ◽  
Attila Csikasz-Nagy ◽  
Andrew M. Collings ◽  
Federico Vaggi
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Final Decision ◽  
Scientific Journals ◽  
Scientific Publications ◽  
Peer Reviewers ◽  
Decision Type ◽  
Number Of Citations ◽  
Decision Times

BackgroundPublishing in scientific journals is one of the most important ways in which scientists disseminate research to their peers and to the wider public. Pre-publication peer review underpins this process, but peer review is subject to various criticisms and is under pressure from growth in the number of scientific publications.MethodsHere we examine an element of the editorial process ateLife, in which the Reviewing Editor usually serves as one of the referees, to see what effect this has on decision times, decision type, and the number of citations. We analysed a dataset of 8,905 research submissions toeLifesince June 2012, of which 2,750 were sent for peer review, using R and Python to perform the statistical analysis.ResultsThe Reviewing Editor serving as one of the peer reviewers results in faster decision times on average, with the time to final decision ten days faster for accepted submissions (n=1,405) and 5 days faster for papers that were rejected after peer review (n=1,099). There was no effect on whether submissions were accepted or rejected, and a very small (but significant) effect on citation rates for published articles where the Reviewing Editor served as one of the peer reviewers.ConclusionsAn important aspect ofeLife’s peer-review process is shown to be effective, given that decision times are faster when the Reviewing Editor serves as a reviewer. Other journals hoping to improve decision times could consider adopting a similar approach.

scholarly journals Rethinking success, integrity, and culture in research (Part 2) — A multi-actor qualitative study on problems of science

2020 ◽  
Author(s):  
Noémie Aubert Bonn ◽  
Wim Pinxten
Keyword(s):  
Research Misconduct ◽  
Research Integrity ◽  
Structured Interviews ◽  
Community Members ◽  
Research Practices ◽  
Research Issues ◽  
Questionable Research Practices ◽  
The Rich ◽  
Research Students ◽  
The Impact

ABSTRACTBackgroundResearch misconduct and questionable research practices have been the subject of increasing attention in the past few years. But despite the rich body of research available, few empirical works provide the perspectives of non-researcher stakeholders.MethodsTo capture some of the forgotten voices, we conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science. We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.ResultsGiven the breadth of our results, we divided our findings in a two-paper series with the current paper focusing on the problems that affect the quality and integrity of science. We first discovered that perspectives on misconduct, including the core reasons for condemning misconduct, differed between individuals and actor groups. Beyond misconduct, interviewees also identified numerous problems which affect the integrity of research. Issues related to personalities and attitudes, lack of knowledge of good practices, and research climate were mentioned. Elements that were described as essential for success (in the associate paper) were often thought to accentuate the problems of research climates by disrupting research cultures and research environments. Even though everyone agreed that current research climates need to be addressed, no one felt responsible nor capable of initiating change. Instead, respondents revealed a circle of blame and mistrust between actor groups.ConclusionsOur findings resonate with recent debates, and extrapolate a few action points which might help advance the discussion. First, we must tackle how research is assessed. Second, approaches to promote better science should be revisited: not only should they directly address the impact of climates on research practices, but they should also redefine their objective to empower and support researchers rather than to capitalize on their compliance. Finally, inter-actor dialogues and shared decision making are crucial to building joint objectives for change.Trial registrationosf.io/33v3m

scholarly journals Peer Review in Law Journals

2021 ◽  
Vol 6 ◽  
Author(s):  
Jadranka Stojanovski ◽  
Elías Sanz-Casado ◽  
Tommaso Agnoloni ◽  
Ginevra Peruginelli
Keyword(s):  
Peer Review ◽  
Ethical Issues ◽  
Peer Review Process ◽  
Traditional Approach ◽  
Research Misconduct ◽  
Evaluation Criteria ◽  
Research Integrity ◽  
Publication Type ◽  
Chi Square

The field of law has retained its distinctiveness regarding peer review to this day, and reviews are often conducted without following standardized rules and principles. External and independent evaluation of submissions has recently become adopted by European law journals, and peer review procedures are still poorly defined, investigated, and attuned to the legal science publishing landscape. The aim of our study was to gain a better insight into current editorial policies on peer review in law journals by exploring editorial documents (instructions, guidelines, policies) issued by 119 Croatian, Italian, and Spanish law journals. We relied on automatic content analysis of 135 publicly available documents collected from the journal websites to analyze the basic features of the peer review processes, manuscript evaluation criteria, and related ethical issues using WordStat8. Differences in covered topics between the countries were compared using the chi-square test. Our findings reveal that most law journals have adopted a traditional approach, in which the editorial board manages mostly anonymized peer review (104, 77%) engaging independent/external reviewers (65, 48%). Submissions are evaluated according to their originality and relevance (113, 84%), quality of writing and presentation (94, 70%), comprehensiveness of literature references (93, 69%), and adequacy of methods (57, 42%). The main ethical issues related to peer review addressed by these journals are reviewer’s competing interests (42, 31%), plagiarism (35, 26%), and biases (30, 22%). We observed statistically significant differences between countries in mentioning key concepts such as “Peer review ethics”, “Reviewer”, “Transparency of identities”, “Publication type”, and “Research misconduct”. Spanish journals favor reviewers’ “Independence” and “Competence” and “Anonymized” peer review process. Also, some manuscript types popular in one country are rarely mentioned in other countries. Even though peer review is equally conventional in all three countries, high transparency in Croatian law journals, respect for research integrity in Spanish ones, and diversity and inclusion in Italian are promising indicators of future development.

scholarly journals Online Journals: the Path Towards Visibility and Quality

1970 ◽  
Vol 3 ◽  
pp. 175-184
Author(s):  
Julie Walker
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Editorial Office ◽  
Online Journal ◽  
Scientific Publications ◽  
Journal Editors ◽  
Virtuous Cycle ◽  
Journal Visibility

Increasing the visibility of a journal is the key to increasing quality. The International Network for the Availability of Scientific Publications works with journal editors in the global South to publish their journals online and to increase the efficiency of the peer review process. Editors are trained in using the Open Journals System software and in online journal management and strategy so they have the tools and knowledge needed to initiate a ‘virtuous cycle' in which visibility leads to an increase in the number and quality of submissions and in turn, increased citations and impact. In order to maximise this increase in quality, it must be supported by strong editorial office processes and management. This article describes some of the issues and strategies faced by the editors INASP works with, placing a particular emphasis on Nepal Journals Online. Key words: INASP; Open Journals System; Journals Online Projects; Nepal Journals Online; journal visibility; peer review DOI: 10.3126/dsaj.v3i0.2786 Dhaulagiri Journal of Sociology and Anthropology Vol.3 2009 175-184

scholarly journals The effects of an editor serving as one of the reviewers during the peer-review process

F1000Research ◽  
2016 ◽  
Vol 5 ◽  
pp. 683 ◽  
Author(s):  
Marco Giordan ◽  
Attila Csikasz-Nagy ◽  
Andrew M. Collings ◽  
Federico Vaggi
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Final Decision ◽  
Scientific Journals ◽  
Scientific Publications ◽  
Peer Reviewers ◽  
Decision Type ◽  
Number Of Citations ◽  
Decision Times

BackgroundPublishing in scientific journals is one of the most important ways in which scientists disseminate research to their peers and to the wider public. Pre-publication peer review underpins this process, but peer review is subject to various criticisms and is under pressure from growth in the number of scientific publications.MethodsHere we examine an element of the editorial process ateLife, in which the Reviewing Editor usually serves as one of the referees, to see what effect this has on decision times, decision type, and the number of citations. We analysed a dataset of 8,905 research submissions toeLifesince June 2012, of which 2,747 were sent for peer review. This subset of 2747 papers was then analysed in detail.  ResultsThe Reviewing Editor serving as one of the peer reviewers results in faster decision times on average, with the time to final decision ten days faster for accepted submissions (n=1,405) and five days faster for papers that were rejected after peer review (n=1,099). Moreover, editors acting as reviewers had no effect on whether submissions were accepted or rejected, and a very small (but significant) effect on citation rates.ConclusionsAn important aspect ofeLife’s peer-review process is shown to be effective, given that decision times are faster when the Reviewing Editor serves as a reviewer. Other journals hoping to improve decision times could consider adopting a similar approach.

Questionable Research Practices (QRPs) and Their Devastating Scientific Effects

2021 ◽  
pp. 56-90
Author(s):  
R. Barker Bausell
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Institutional Research ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Research Participants ◽  
Positive Results ◽  
Exhaustive List

The linchpin of both publication bias and irreproducibility involves an exhaustive list of more than a score of individually avoidable questionable research practices (QRPs) supplemented by 10 inane institutional research practices. While these untoward effects on the production of false-positive results are unsettling, a far more entertaining (in a masochistic sort of way) pair of now famous iconoclastic experiments conducted by Simmons, Nelson, and Simonsohn are presented in which, with the help of only a few well-chosen QRPs, research participants can actually become older after simply listening to a Beatle’s song. In addition, surveys designed to estimate the prevalence of these and other QRPs in the published literatures are also described.

Sign in / Sign up

Export Citation Format

Share Document