Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

2012 ◽  
Vol 23 (5) ◽  
pp. 524-532 ◽  
Author(s):  
Leslie K. John ◽  
George Loewenstein ◽  
Drazen Prelec
Keyword(s):  
Scientific Misconduct ◽  
Estimation Methods ◽  
Truth Telling ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Media Attention ◽  
Elicitation Format ◽  
The Impact

Cases of clear scientific misconduct have received significant media attention recently, but less flagrantly questionable research practices may be more prevalent and, ultimately, more damaging to the academic enterprise. Using an anonymous elicitation format supplemented by incentives for honest reporting, we surveyed over 2,000 psychologists about their involvement in questionable research practices. The impact of truth-telling incentives on self-admissions of questionable research practices was positive, and this impact was greater for practices that respondents judged to be less defensible. Combining three different estimation methods, we found that the percentage of respondents who have engaged in questionable practices was surprisingly high. This finding suggests that some questionable practices may constitute the prevailing research norm.

scholarly journals Rethinking success, integrity, and culture in research (Part 2) — A multi-actor qualitative study on problems of science

2020 ◽  
Author(s):  
Noémie Aubert Bonn ◽  
Wim Pinxten
Keyword(s):  
Research Misconduct ◽  
Research Integrity ◽  
Structured Interviews ◽  
Community Members ◽  
Research Practices ◽  
Research Issues ◽  
Questionable Research Practices ◽  
The Rich ◽  
Research Students ◽  
The Impact

ABSTRACTBackgroundResearch misconduct and questionable research practices have been the subject of increasing attention in the past few years. But despite the rich body of research available, few empirical works provide the perspectives of non-researcher stakeholders.MethodsTo capture some of the forgotten voices, we conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science. We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.ResultsGiven the breadth of our results, we divided our findings in a two-paper series with the current paper focusing on the problems that affect the quality and integrity of science. We first discovered that perspectives on misconduct, including the core reasons for condemning misconduct, differed between individuals and actor groups. Beyond misconduct, interviewees also identified numerous problems which affect the integrity of research. Issues related to personalities and attitudes, lack of knowledge of good practices, and research climate were mentioned. Elements that were described as essential for success (in the associate paper) were often thought to accentuate the problems of research climates by disrupting research cultures and research environments. Even though everyone agreed that current research climates need to be addressed, no one felt responsible nor capable of initiating change. Instead, respondents revealed a circle of blame and mistrust between actor groups.ConclusionsOur findings resonate with recent debates, and extrapolate a few action points which might help advance the discussion. First, we must tackle how research is assessed. Second, approaches to promote better science should be revisited: not only should they directly address the impact of climates on research practices, but they should also redefine their objective to empower and support researchers rather than to capitalize on their compliance. Finally, inter-actor dialogues and shared decision making are crucial to building joint objectives for change.Trial registrationosf.io/33v3m

scholarly journals Ethical Shades of Gray: Questionable Research Practices in Health Professions Education

2018 ◽  
Author(s):  
Anthony R. Artino ◽  
Erik W. Driessen ◽  
Lauren A. Maggio
Keyword(s):  
Scientific Misconduct ◽  
Health Professions ◽  
The United States ◽  
Health Professions Education ◽  
Survey Study ◽  
Practice Research ◽  
Scientific Integrity ◽  
Public Confidence ◽  
Research Practices ◽  
Questionable Research Practices

AbstractPurposeTo maintain scientific integrity and engender public confidence, research must be conducted responsibly. Whereas scientific misconduct, like data fabrication, is clearly irresponsible and unethical, other behaviors—often referred to as questionable research practices (QRPs)—exploit the ethical shades of gray that color acceptable practice. This study aimed to measure the frequency of self-reported QRPs in a diverse, international sample of health professions education (HPE) researchers.MethodIn 2017, the authors conducted an anonymous, cross-sectional survey study. The web-based survey contained 43 QRP items that asked respondents to rate how often they had engaged in various forms of scientific misconduct. The items were adapted from two previously published surveys.ResultsIn total, 590 HPE researchers took the survey. The mean age was 46 years (SD=11.6), and the majority of participants were from the United States (26.4%), Europe (23.2%), and Canada (15.3%). The three most frequently reported QRPs were adding authors to a paper who did not qualify for authorship (60.6%), citing articles that were not read (49.5%), and selectively citing papers to please editors or reviewers (49.4%). Additionally, respondents reported misrepresenting a participant’s words (6.7%), plagiarizing (5.5%), inappropriately modifying results (5.3%), deleting data without disclosure (3.4%), and fabricating data (2.4%). Overall, 533 (90.3%) respondents reported at least one QRP.ConclusionsNotwithstanding the methodological limitations of survey research, these findings indicate that a substantial proportion of HPE researchers report a range of QRPs. In light of these results, reforms are needed to improve the credibility and integrity of the HPE research enterprise.“Researchers should practice research responsibly. Unfortunately, some do not.” –Nicholas H. Steneck, 20061

scholarly journals Status of our Science

2017 ◽  
Author(s):  
Matt Motyl ◽  
Alexander Pantelis Demos ◽  
Timothy S Carsel ◽  
Brittany Elyse Hanson ◽  
JP Prims ◽  
...  
Keyword(s):  
Current Debate ◽  
Scientific Quality ◽  
Great Length ◽  
Research Practices ◽  
Personality Psychology ◽  
Positive Direction ◽  
Questionable Research Practices ◽  
Current State ◽  
The Impact

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the prevalence of questionable research practices (QRPs) and the replicability of particular findings, the impact of the current discussion on research practices is unknown. The current studies examine whether and how practices have changed, if at all, over the last 10 years. In Study 1, we surveyed 1,166 social and personality psychologists about how the current debate has affected their perceptions of their own and the field’s research practices. In Study 2, we coded the research practices and critical test statistics from social and personality psychology articles published in 2003-2004 and 2013-2014. Together, these studies suggest that (1) perceptions of the current state of the field are more pessimistic than optimistic; (2) the discussion has increased researchers’ intentions to avoid QRPs and adopt proposed best practices, (3) the estimated replicability of research published in 2003-2004 may not be as bad as many feared, and (4) research published in 2013-2014 shows some improvement over research published in 2003-2004, a result that suggests the field is evolving in a positive direction.

scholarly journals The pandemic threatens the Registered Reports system as well as human lives

2020 ◽  
Author(s):  
Kyoshiro Sasaki ◽  
Yuki Yamada
Keyword(s):  
Statistical Analysis ◽  
Experimental Design ◽  
Data Collection ◽  
Sample Size ◽  
Life Experience ◽  
Real Life ◽  
Research Practices ◽  
Questionable Research Practices ◽  
System P ◽  
The Impact

A Registered Reports system is key to preventing questionable research practices. Under this system, manuscripts, including their detailed protocols (i.e., hypothesis, experimental design, sample size, and methods of statistical analysis), are reviewed prior to data collection. If a protocol manuscript is accepted, publication of the full manuscript including the results and discussion is guaranteed in principle regardless of whether the collected data support the registered hypothesis. However, this assurance of publication might be broken under the impact of the COVID-19 pandemic. The present paper reports the first author’s real-life experience related to the collapse of the assurance of publication in the Registered Reports system and discusses the disbenefits of this collapse. Furthermore, we propose the implementation of a journal section specific to protocol manuscripts as a solution to the crisis of the Registered Reports system.

scholarly journals Philosophy of Scientific Malpractice

SATS ◽  
2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Hanne Andersen
Keyword(s):  
Research Ethics ◽  
Philosophy Of Science ◽  
Scientific Knowledge ◽  
Scientific Misconduct ◽  
Current Work ◽  
Responsible Conduct Of Research ◽  
Research Practices ◽  
New Approach ◽  
Questionable Research Practices ◽  
Responsible Conduct

Abstract This paper presents current work in philosophy of science in practice that focusses on practices that are detrimental to the production of scientific knowledge. The paper argues that philosophy of scientific malpractice both provides an epistemological complement to research ethics in understanding scientific misconduct and questionable research practices, and provides a new approach to how training in responsible conduct of research can be implemented.

Quantifying, and correcting for, the impact of questionable research practices on false discovery rates in psychological science

2020 ◽  
Author(s):  
Dwight Kravitz ◽  
Stephen Mitroff
Keyword(s):  
Large Scale ◽  
Unintended Consequences ◽  
Research Practices ◽  
False Discovery Rates ◽  
Questionable Research Practices ◽  
Real Effects ◽  
False Discovery ◽  
Replication Crisis ◽  
The Impact ◽  
Discovery Rates

Large-scale replication failures have shaken confidence in the social sciences, psychology in particular. Most researchers acknowledge the problem, yet there is widespread debate about the causes and solutions. Using “big data,” the current project demonstrates that unintended consequences of three common questionable research practices (retaining pilot data, adding data after checking for significance, and not publishing null findings) can explain the lion’s share of the replication failures. A massive dataset was randomized to create a true null effect between two conditions, and then these three practices were applied. They produced false discovery rates far greater than 5% (the generally accepted rate), and were strong enough to obscure, or even reverse, the direction of real effects. These demonstrations suggest that much of the replication crisis might be explained by simple, misguided experimental choices. This approach also produces empirically-based corrections to account for these practices when they are unavoidable, providing a viable path forward.

Sign in / Sign up

Export Citation Format

Share Document