Experimental deception: Science, performance, and reproducibility

Experimental deception has not been seriously examined in terms of its impact on reproducible science. I demonstrate, using data from the Open Science Collaboration’s Reproducibility Project (2015), that experiments involving deception have a higher probability of not replicating and have smaller effect sizes compared to experiments that do not have deception procedures. This trend is possibly due to missing information about the context and performance of agents in the studies in which the original effects were generated, leading to either compromised internal validity, or an incomplete specification and control of variables in replication studies. Of special interest are the mechanisms by which deceptions are implemented and how these present challenges for the efficient transmission of critical information from experimenter to participant. I rehearse possible frameworks that might form the basis of a future research program on experimental deception and make some recommendations as to how such a program might be initiated.

Deception in Management Accounting Experimental Research: “A Tricky Issue” Revisited

ABSTRACT Management accounting (MA) scholars generally accept that our subject matter requires a multidisciplinary approach. Broadly speaking, there are two main views from different base disciplines about experimental deception: “deception if necessary” (social psychology) and “deception should be banned” (experimental economics). We aim to develop a common understanding within the MA research community about what constitutes deceptive research practice. We review arguments supporting the two main views and analyze the transfer of their norms into MA research. We develop a framework that evaluates the need for and potential consequences of using deception. Our analysis implies careful consideration of the decision to employ deception and case-by-case editorial review of experiments employing deception are necessary. In the long run, the MA research community may consider if an explicit policy on the role of deception in MA research is warranted or whether a case-by-case approach, as advocated by us as an interim measure, is sufficient.

[Working Paper] Deception predicts replication failure in OSC Reproducibility Project

Describes preliminary analysis using the data from the Open Science Collaboration's Reproducibility Project (2015). After coding the studies as to whether deception was a feature of their procedures, it is shown that replications that involved deception had an 89% probability of not being replicated, as opposed to the 63% failure rate of studies that did not involve deception. This result will be part of a larger project exploring the nature of experimental deception as a scientific technique.