Unfolding the Black Box of Questionable Research Practices: Where Is the Line Between Acceptable and Unacceptable Practices?

2020 ◽  
Vol 30 (3) ◽  
pp. 335-360
Author(s):  
Christian Linder ◽  
Siavash Farahbakhsh
Keyword(s):  
Communicative Action ◽  
Scientific Research ◽  
Black Box ◽  
Extensive Literature ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Scientific Standards ◽  
Normative Perspective

ABSTRACTDespite the extensive literature on what questionable research practices (QRPs) are and how to measure them, the normative underpinnings of such practices have remained less explored. QRPs often fall into a grey area of justifiable and unjustifiable practices. Where to precisely draw the line between such practices challenges individual scholars and this harms science. We investigate QRPs from a normative perspective using the theory of communicative action. We highlight the role of the collective in assessing individual behaviours. Our contribution is a framework that allows identification of when particular actions cross over from acceptable to unacceptable practice. Thus, this article provides grounds for developing scientific standards to raise the quality of scientific research.

A reconnaissance of tropical resources during Revolutionary years: the role of the Paris Museum d'Histoire Naturelle

1991 ◽  
Vol 18 (3) ◽  
pp. 333-362 ◽  
Author(s):  
MADELEINE LY-TIO-FANE
Keyword(s):  
South Africa ◽  
Natural History ◽  
Scientific Research ◽  
Extensive Literature ◽  
War Of Independence ◽  
Leading Role ◽  
Scientific Principles ◽  
Set Up ◽  
Joseph Ii

SUMMARY The recent extensive literature on exploration and the resulting scientific advances has failed to highlight the contribution of Austrian enterprise to the study of natural history. The leading role of Joseph II among the neutral powers which assumed the carrying trade of the belligerents during the American War of Independence, furthered the development of collections for the Schönbrunn Park and Gardens which had been set up on scientific principles by his parents. On the conclusion of peace, Joseph entrusted to Professor Maerter a world-encompassing mission in the course of which the Chief Gardener Franz Boos and his assistant Georg Scholl travelled to South Africa to collect plants and animals. Boos pursued the mission to Isle de France and Bourbon (Mauritius and Reunion), conveyed by the then unknown Nicolas Baudin. He worked at the Jardin du Roi, Pamplemousses, with Nicolas Cere, or at Palma with Joseph Francois Charpentier de Cossigny. The linkage of Austrian and French horticultural expertise created a situation fraught with opportunities which were to lead Baudin to the forefront of exploration and scientific research as the century closed in the upheaval of the Revolutionary Wars.

scholarly journals Ethical Consistency and Experience: An Attempt to Influence Researcher Attitudes Toward Questionable Research Practices Through Reading Prompts

2019 ◽  
Vol 15 (3) ◽  
pp. 216-226 ◽  
Author(s):  
Samuel V. Bruton ◽  
Mitch Brown ◽  
Donald F. Sacco
Keyword(s):  
Best Practices ◽  
Research Ethics ◽  
Scientific Research ◽  
Psychological Interventions ◽  
Research Practices ◽  
Widespread Occurrence ◽  
Questionable Research Practices ◽  
The Past

Over the past couple of decades, the apparent widespread occurrence of Questionable Research Practices (QRPs) in scientific research has been widely discussed in the research ethics literature as a source of concern. Various ways of reducing their use have been proposed and implemented, ranging from improved training and incentives for adopting best practices to systematic reforms. This article reports on the results of two studies that investigated the efficacy of simple, psychological interventions aimed at changing researcher attitudes toward QRPs. While the interventions did not significantly modify researchers’ reactions to QRPs, they showed differential efficacy depending on scientists’ experience, suggesting complexities in researcher psychology and the ethics of QRPs that merit further study.

scholarly journals Status of our Science

2017 ◽  
Author(s):  
Matt Motyl ◽  
Alexander Pantelis Demos ◽  
Timothy S Carsel ◽  
Brittany Elyse Hanson ◽  
JP Prims ◽  
...  
Keyword(s):  
Current Debate ◽  
Scientific Quality ◽  
Great Length ◽  
Research Practices ◽  
Personality Psychology ◽  
Positive Direction ◽  
Questionable Research Practices ◽  
Current State ◽  
The Impact

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the prevalence of questionable research practices (QRPs) and the replicability of particular findings, the impact of the current discussion on research practices is unknown. The current studies examine whether and how practices have changed, if at all, over the last 10 years. In Study 1, we surveyed 1,166 social and personality psychologists about how the current debate has affected their perceptions of their own and the field’s research practices. In Study 2, we coded the research practices and critical test statistics from social and personality psychology articles published in 2003-2004 and 2013-2014. Together, these studies suggest that (1) perceptions of the current state of the field are more pessimistic than optimistic; (2) the discussion has increased researchers’ intentions to avoid QRPs and adopt proposed best practices, (3) the estimated replicability of research published in 2003-2004 may not be as bad as many feared, and (4) research published in 2013-2014 shows some improvement over research published in 2003-2004, a result that suggests the field is evolving in a positive direction.

scholarly journals Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices

2019 ◽  
Author(s):  
Aaron R Caldwell ◽  
Andrew David Vigotsky ◽  
Greg Nuckols ◽  
Ian Boardley ◽  
Julia Schmidt ◽  
...  
Keyword(s):  
Data Collection ◽  
Science Research ◽  
Exercise Science ◽  
Research Practices ◽  
Questionable Research Practices ◽  
Data Collection And Analysis ◽  
Primary Means ◽  
Sport And Exercise ◽  
Stage 1

The primary means for disseminating sport and exercise science research is currently through journal articles. However, not all studies, especially those with null findings, make it to formal publication. This publication bias towards positive findings may contribute to questionable research practices. Preregistration is a solution to prevent the publication of distorted evidence resulting from this system. This process asks authors to register their hypotheses and methods before data collection on a publicly available repository or by submitting a Registered Report. In the Registered Reports format, authors submit a Stage 1 manuscript to a participating journal that includes an introduction, methods, and any pilot data indicating the exploratory or confirmatory nature of the study. After a Stage 1 peer review, the manuscript can then be offered in-principle acceptance, rejected, or sent back for revisions to improve the quality of the study. If accepted, the project is guaranteed publication, assuming the authors follow the data collection and analysis protocol. After data collection, authors re-submit a Stage 2 manuscript that includes the results and discussion, and the study is evaluated on clarity and conformity with the planned analysis. In its final form, Registered Reports appear almost identical to a typical publication, but give readers confidence that the hypotheses and main analyses are less susceptible to bias from questionable research practices. From this perspective, we argue that inclusion of Registered Reports by researchers and journals will improve the transparency, replicability, and trust in sport and exercise science research.

Component-based Analysis of Dynamic Search Performance

2022 ◽  
Vol 40 (3) ◽  
pp. 1-47
Author(s):  
Ameer Albahem ◽  
Damiano Spina ◽  
Falk Scholer ◽  
Lawrence Cavedon
Keyword(s):  
Information Needs ◽  
Black Box ◽  
User Feedback ◽  
Effect Sizes ◽  
Search Performance ◽  
Evaluation Scores ◽  
Dynamic Search ◽  
Dynamic Domain

In many search scenarios, such as exploratory, comparative, or survey-oriented search, users interact with dynamic search systems to satisfy multi-aspect information needs. These systems utilize different dynamic approaches that exploit various user feedback granularity types. Although studies have provided insights about the role of many components of these systems, they used black-box and isolated experimental setups. Therefore, the effects of these components or their interactions are still not well understood. We address this by following a methodology based on Analysis of Variance (ANOVA). We built a Grid Of Points that consists of systems based on different ways to instantiate three components: initial rankers, dynamic rerankers, and user feedback granularity. Using evaluation scores based on the TREC Dynamic Domain collections, we built several ANOVA models to estimate the effects. We found that (i) although all components significantly affect search effectiveness, the initial ranker has the largest effective size, (ii) the effect sizes of these components vary based on the length of the search session and the used effectiveness metric, and (iii) initial rankers and dynamic rerankers have more prominent effects than user feedback granularity. To improve effectiveness, we recommend improving the quality of initial rankers and dynamic rerankers. This does not require eliciting detailed user feedback, which might be expensive or invasive.

scholarly journals Verifiability in computer-aided research: the role of digital scientific notations at the human-computer interface

2018 ◽  
Vol 4 ◽  
pp. e158 ◽  
Author(s):  
Konrad Hinsen
Keyword(s):  
Scientific Information ◽  
Scientific Research ◽  
Theoretical Models ◽  
Black Box ◽  
Point Of View ◽  
Computer Interface ◽  
Human Computer Interface ◽  
Scientific Software ◽  
Computer Aided

Most of today’s scientific research relies on computers and software for processing scientific information. Examples of such computer-aided research are the analysis of experimental data or the simulation of phenomena based on theoretical models. With the rapid increase of computational power, scientific software has integrated more and more complex scientific knowledge in a black-box fashion. As a consequence, its users do not know, and do not even have a chance of finding out, which assumptions and approximations their computations are based on. This black-box nature of scientific software has made the verification of much computer-aided research close to impossible. The present work starts with an analysis of this situation from the point of view of human-computer interaction in scientific research. It identifies the key role of digital scientific notations at the human-computer interface, reviews the most popular ones in use today, and describes a proof-of-concept implementation of Leibniz, a language designed as a verifiable digital scientific notation for models formulated as mathematical equations.

Decision-Making in the Prescription of Orthotics and Prosthetics for Partial-Foot Amputees

2019 ◽  
Author(s):  
Pratima Saravanan ◽  
Charity Hipple ◽  
Jingxin Wang ◽  
Christopher McComb ◽  
Jessica Menold
Keyword(s):  
Decision Making ◽  
Low Income ◽  
Low Income Countries ◽  
Extensive Literature ◽  
Proposed Model ◽  
Decision Making Model ◽  
Future Work ◽  
Automated Methods

Abstract Prosthetists face a daunting number of decisions that directly affect an amputee’s ability to walk and indirectly affect the overall quality of life of that amputee. In addition, the lack of resources in low-income countries provides a barrier to receive care after an amputation, and approximately 80% of amputees in low-income countries lack appropriate prosthetic care. In this research, we are motivated to understand what factors affect the decision-making strategies of prosthetists and podiatrists when prescribing prosthetics and orthotics to partial foot amputees. This work establishes a decision-making framework as a step towards automated methods that may reduce the complexities and decision-making burden of prosthetic prescription, ultimately increasing the efficiency of prosthetic prescription in low-resourced areas. A decision-making model is proposed based on an extensive literature review of over 100 papers. The proposed model is compared to qualitative data regarding decision-making strategies during prosthetic or orthotic prescription collected from nine prosthetists, surgeons, and other healthcare professionals directly involved in amputee care. Changes to the proposed model are described and future work exploring the role of automated methods to support decision-making in the context of prosthetics is discussed.

scholarly journals What Do Participants Think of Our Research Practices? An Examination of Behavioral Psychology Participants’ Preferences

2021 ◽  
Author(s):  
Julia G. Bottesini ◽  
Mijke Rhemtulla ◽  
Simine Vazire
Keyword(s):  
Open Access Publishing ◽  
Selective Reporting ◽  
Minimal Risk ◽  
Research Practices ◽  
Scientific Fraud ◽  
Questionable Research Practices ◽  
Research Participants ◽  
Behavioral Psychology ◽  
Is Value

What research practices should be considered acceptable? Historically, scientists have set the standards for what constitutes acceptable research practices. However, there is value in considering non-scientists’ perspectives, including research participants’. 1,873 participants from MTurk and university subject pools were surveyed after their participation in one of eight minimal-risk studies. We asked participants how they would feel if common research practices were applied to their data: p-hacking/cherry-picking results, selective reporting of studies, Hypothesizing After Results are Known (HARKing), committing fraud, conducting direct replications, sharing data, sharing methods, and open access publishing. An overwhelming majority of psychology research participants think questionable research practices (e.g., p-hacking, HARKing) are unacceptable (68.3--81.3%), and were supportive of practices to increase transparency and replicability (71.4--80.1%). A surprising number of participants expressed positive or neutral views toward scientific fraud, raising concerns about the quality of our data. We grapple with this concern and interpret our results in light of the limitations of our study. Despite ambiguity in our results, we argue that there is evidence (from our study and others’) that researchers may be violating participants’ expectations and should be transparent with participants about how their data will be used.

Sign in / Sign up

Export Citation Format

Share Document