scholarly journals What Do Participants Think of Our Research Practices? An Examination of Behavioral Psychology Participants’ Preferences

2021 ◽  
Author(s):  
Julia G. Bottesini ◽  
Mijke Rhemtulla ◽  
Simine Vazire
Keyword(s):  
Open Access Publishing ◽  
Selective Reporting ◽  
Minimal Risk ◽  
Research Practices ◽  
Scientific Fraud ◽  
Questionable Research Practices ◽  
Research Participants ◽  
Behavioral Psychology ◽  
Is Value

What research practices should be considered acceptable? Historically, scientists have set the standards for what constitutes acceptable research practices. However, there is value in considering non-scientists’ perspectives, including research participants’. 1,873 participants from MTurk and university subject pools were surveyed after their participation in one of eight minimal-risk studies. We asked participants how they would feel if common research practices were applied to their data: p-hacking/cherry-picking results, selective reporting of studies, Hypothesizing After Results are Known (HARKing), committing fraud, conducting direct replications, sharing data, sharing methods, and open access publishing. An overwhelming majority of psychology research participants think questionable research practices (e.g., p-hacking, HARKing) are unacceptable (68.3--81.3%), and were supportive of practices to increase transparency and replicability (71.4--80.1%). A surprising number of participants expressed positive or neutral views toward scientific fraud, raising concerns about the quality of our data. We grapple with this concern and interpret our results in light of the limitations of our study. Despite ambiguity in our results, we argue that there is evidence (from our study and others’) that researchers may be violating participants’ expectations and should be transparent with participants about how their data will be used.

scholarly journals Status of our Science

2017 ◽  
Author(s):  
Matt Motyl ◽  
Alexander Pantelis Demos ◽  
Timothy S Carsel ◽  
Brittany Elyse Hanson ◽  
JP Prims ◽  
...  
Keyword(s):  
Current Debate ◽  
Scientific Quality ◽  
Great Length ◽  
Research Practices ◽  
Personality Psychology ◽  
Positive Direction ◽  
Questionable Research Practices ◽  
Current State ◽  
The Impact

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the prevalence of questionable research practices (QRPs) and the replicability of particular findings, the impact of the current discussion on research practices is unknown. The current studies examine whether and how practices have changed, if at all, over the last 10 years. In Study 1, we surveyed 1,166 social and personality psychologists about how the current debate has affected their perceptions of their own and the field’s research practices. In Study 2, we coded the research practices and critical test statistics from social and personality psychology articles published in 2003-2004 and 2013-2014. Together, these studies suggest that (1) perceptions of the current state of the field are more pessimistic than optimistic; (2) the discussion has increased researchers’ intentions to avoid QRPs and adopt proposed best practices, (3) the estimated replicability of research published in 2003-2004 may not be as bad as many feared, and (4) research published in 2013-2014 shows some improvement over research published in 2003-2004, a result that suggests the field is evolving in a positive direction.

scholarly journals Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices

2019 ◽  
Author(s):  
Aaron R Caldwell ◽  
Andrew David Vigotsky ◽  
Greg Nuckols ◽  
Ian Boardley ◽  
Julia Schmidt ◽  
...  
Keyword(s):  
Data Collection ◽  
Science Research ◽  
Exercise Science ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Data Collection And Analysis ◽  
Primary Means ◽  
Sport And Exercise ◽  
Stage 1

The primary means for disseminating sport and exercise science research is currently through journal articles. However, not all studies, especially those with null findings, make it to formal publication. This publication bias towards positive findings may contribute to questionable research practices. Preregistration is a solution to prevent the publication of distorted evidence resulting from this system. This process asks authors to register their hypotheses and methods before data collection on a publicly available repository or by submitting a Registered Report. In the Registered Reports format, authors submit a Stage 1 manuscript to a participating journal that includes an introduction, methods, and any pilot data indicating the exploratory or confirmatory nature of the study. After a Stage 1 peer review, the manuscript can then be offered in-principle acceptance, rejected, or sent back for revisions to improve the quality of the study. If accepted, the project is guaranteed publication, assuming the authors follow the data collection and analysis protocol. After data collection, authors re-submit a Stage 2 manuscript that includes the results and discussion, and the study is evaluated on clarity and conformity with the planned analysis. In its final form, Registered Reports appear almost identical to a typical publication, but give readers confidence that the hypotheses and main analyses are less susceptible to bias from questionable research practices. From this perspective, we argue that inclusion of Registered Reports by researchers and journals will improve the transparency, replicability, and trust in sport and exercise science research.

Implications of the Open Science Era for Educational Psychology Research Syntheses

2020 ◽  
Author(s):  
Carly D Robinson
Keyword(s):  
Educational Psychology ◽  
Time Frame ◽  
Open Science ◽  
Selective Reporting ◽  
Psychology Research ◽  
Special Issue ◽  
Questionable Research Practices ◽  
Comprehensive Information ◽  
Open Materials

Extensive debate and criticism of potentially common, yet questionable research practices that lead to biased findings within social and health sciences has emerged over the last decade. These challenges likely apply to educational psychology, though the field has been slow to address them. This article discusses current research norms, strategic solutions proposed under the broad rubric of “Open Science”, and the implications of both for the way research syntheses in educational psychology are conducted and the quality of the information they produce. Strategies such as preregistration, open materials and data, and registered reports stand to address significant threats to the validity of research synthesis. These include challenges associated with publication, dissemination, and selective reporting biases, comprehensive information retrieval, and opportunities to execute unique analytic approaches. A final issue is the development of parallel solutions that address biases specific to the decision making of researchers conducting and evaluating research syntheses. PLEASE DO NOT CITE YET:This article is part of a forthcoming journal Special Issue on Open Science in Education and currently under review. Carly Robinson is NOT the correct author, so please do not cite this article until it is updated with the correct authors' names. If you are interested in citing this work please either (a) check back at this url later -- we anticipated that the correct authors' names will be included no later than February 2021, or (b) contact Carly Robinson ([email protected]) directly to see if the paper might be cited on an earlier time frame.

Questionable Research Practices (QRPs) and Their Devastating Scientific Effects

2021 ◽  
pp. 56-90
Author(s):  
R. Barker Bausell
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Institutional Research ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Research Participants ◽  
Positive Results ◽  
Exhaustive List

The linchpin of both publication bias and irreproducibility involves an exhaustive list of more than a score of individually avoidable questionable research practices (QRPs) supplemented by 10 inane institutional research practices. While these untoward effects on the production of false-positive results are unsettling, a far more entertaining (in a masochistic sort of way) pair of now famous iconoclastic experiments conducted by Simmons, Nelson, and Simonsohn are presented in which, with the help of only a few well-chosen QRPs, research participants can actually become older after simply listening to a Beatle’s song. In addition, surveys designed to estimate the prevalence of these and other QRPs in the published literatures are also described.

Unfolding the Black Box of Questionable Research Practices: Where Is the Line Between Acceptable and Unacceptable Practices?

2020 ◽  
Vol 30 (3) ◽  
pp. 335-360
Author(s):  
Christian Linder ◽  
Siavash Farahbakhsh
Keyword(s):  
Communicative Action ◽  
Scientific Research ◽  
Black Box ◽  
Extensive Literature ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Scientific Standards ◽  
Normative Perspective

ABSTRACTDespite the extensive literature on what questionable research practices (QRPs) are and how to measure them, the normative underpinnings of such practices have remained less explored. QRPs often fall into a grey area of justifiable and unjustifiable practices. Where to precisely draw the line between such practices challenges individual scholars and this harms science. We investigate QRPs from a normative perspective using the theory of communicative action. We highlight the role of the collective in assessing individual behaviours. Our contribution is a framework that allows identification of when particular actions cross over from acceptable to unacceptable practice. Thus, this article provides grounds for developing scientific standards to raise the quality of scientific research.

scholarly journals Pre-registration and Registered Reports: A primer from UKRN

2020 ◽  
Author(s):  
Suzanne Stewart ◽  
Eike Mark Rinke ◽  
Ronan McGarrigle ◽  
Dermot Lynott ◽  
Carole Lunny ◽  
...  
Keyword(s):  
Selective Reporting ◽  
Research Practices ◽  
Questionable Research Practices

Help reduce questionable research practices, and prevent selective reporting.

scholarly journals The Ethical Use of Fit Indices in Structural Equation Modeling: Recommendations for Psychologists

2021 ◽  
Vol 12 ◽  
Author(s):  
Bryant M. Stone
Keyword(s):  
Structural Equation Modeling ◽  
Structural Equation ◽  
Structural Equation Models ◽  
Ethical Dilemmas ◽  
Current Paper ◽  
Equation Modeling ◽  
Selective Reporting ◽  
Fit Indices ◽  
Research Practices ◽  
Questionable Research Practices

Fit indices provide helpful information for researchers to assess the fit of their structural equation models to their data. However, like many statistics and methods, researchers can misuse fit indices, which suggest the potential for questionable research practices that might arise during the analytic and interpretative processes. In the current paper, the author highlights two critical ethical dilemmas regarding the use of fit indices, which are (1) the selective reporting of fit indices and (2) using fit indices to justify poorly-fitting models. The author highlights the dilemmas and provides potential solutions for researchers and journals to follow to reduce these questionable research practices.

scholarly journals Psychology and Law, Meet Open Science

2021 ◽  
Author(s):  
Bradley David McAuliff ◽  
Melanie B. Fessinger ◽  
Anthony Perillo ◽  
Jennifer Torkildson Perillo
Keyword(s):  
Open Access ◽  
False Positive ◽  
Psychological Research ◽  
Open Science ◽  
Open Access Publishing ◽  
Next Generation ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Psychology And Law ◽  
Open Materials

As the field of psychology and law begins to embrace more transparent and accessible science, many questions arise about what open science actually is and how to do it. In this chapter, we contextualize this reform by examining fundamental concerns about psychological research—irreproducibility and replication failures, false-positive errors, and questionable research practices—that threaten its validity and credibility. Next, we turn to psychology’s response by reviewing the concept of open science and explaining how to implement specific practices—preregistration, registered reports, open materials/data/code, and open access publishing—designed to make research more transparent and accessible. We conclude by weighing the implications of open science for the field of psychology and law, specifically with respect to how we conduct and evaluate research, as well as how we train the next generation of psychological scientists and share scientific findings in applied settings.

scholarly journals Survey of Australian STEMM Early Career Researchers: job insecurity and questionable research practices are major structural concerns

2020 ◽  
Author(s):  
Katherine Christian ◽  
Carolyn Johnstone ◽  
Jo-ann Larkins ◽  
Wendy Wright ◽  
Michael R. Doran
Keyword(s):  
Job Insecurity ◽  
Intention To Leave ◽  
Early Career ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Community Needs ◽  
Early Career Researchers ◽  
Engineering Mathematics ◽  
Quality Of Research

AbstractWe sought to understand the pressures on Early Career Researchers (ECR) in the Science, Technology, Engineering, Mathematics, & Medicine (STEMM) disciplines, collecting data from 658 ECRs working in Australia. Respondents indicated a “love of science”, but most also indicated an intention to leave their position. Decisions were primarily motivated by job insecurity (52%), while grievances included poor supervision (60%), bullying or harassment (34%), inequitable hiring practices (39%) and poor support for families (9.6%). A concerning rate of “questionable research practices” by colleagues (34.1% to 41.1%) was reported to have impacted ECR career advancement. Our study links recent reports that characterise the health of the research industry, providing direct insight from ECRs on job insecurity, workplace culture challenges, and the logical rise of questionable research practices. Internationally, nationally and institutionally the research community needs to improve job security (care for our people) and the quality of research data (our product).

scholarly journals Spanish Scientists’ Opinion about Science and Researcher Behavior

2021 ◽  
Vol 24 ◽  
Author(s):  
Dolores Frias-Navarro ◽  
Marcos Pascual-Soler ◽  
José Perezgonzalez ◽  
Héctor Monterde-i-Bort ◽  
Juan Pascual-Llobell
Keyword(s):  
Impact Factor ◽  
Work Climate ◽  
High Impact ◽  
High Impact Factor ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Academic Competition

Abstract We surveyed 348 Psychology and Education researchers within Spain, on issues such as their perception of a crisis in Science, their confidence in the quality of published results, and the use of questionable research practices (QRP). Their perceptions regarding pressure to publish and academic competition were also collected. The results indicate that a large proportion of the sample of Spanish academics think there is a crisis in Science, mainly due to a lack of economic investment, and doubts the quality of published findings. They also feel strong pressure to publish in high impact factor journals and a highly competitive work climate.

Sign in / Sign up

Export Citation Format

Share Document