Analytic Review as a Solution to the Misreporting of Statistical Results in Psychological Science
We propose analytic review as a solution to the problem of misreporting statistical results in psychological science. Analytic review requires authors submitting manuscripts for publication to also submit the data file and syntax used during analyses. Regular reviewers or statistical experts then review reported analyses, in order to verify that the analyses reported were actually conducted, and that the statistical values are accurately reported. We begin by describing the problem of misreporting in psychology, and then introduce the basic analytic review process. We then highlight both primary and secondary benefits of adopting analytic review, and describe different permutations of the analytic review system, each with its own strengths and limitations. We conclude by attempting to dispel three anticipated concerns about analytic review, namely: analytic review will increase the workload placed on scholars, analytic review will infringe on the traditional peer-review process, and analytic review will hurt the image of the discipline of psychology. Although implementing analytic review will add one more step to the bureaucratic publication process, we believe it can be implemented in an efficient manner that would greatly assist in decreasing the frequency and impact of misreporting, while also providing secondary benefits in other domains of scientific integrity.