Modeling the Wason Selection Task: A Response to Ragni and Johnson-Laird (2020)

Kellen and Klauer (this issue) discussed a number of limitations in Ragni, Kola, and Johnson-Laird’s (2018) evaluation of theories of the Wason Selection Task. In a reply, Ragni and Johnson-Laird (this issue) raise a number of counter-arguments along with new arguments against the Inference-Guessing Theory. We argue that much of their reply hinges on a number of conceptual and methodological confusions as well as a mischaracterization of some of our main points.

Comparing Theories of Selection Tasks: A Critical Assessment of Boundaries and Benchmarks

The Wason selection task is one of the most prominent paradigms in the psychology of reasoning, with hundreds of published investigations in the last fifty odd years. Butdespite its central role in reasoning research, there has been little to no attempt tomake sense of the data in way that allows us to discard potential theoretical accounts. In fact, theories have been allowed to proliferate without any comprehensive evaluation of their relative performance. In an attempt to address this problem, Ragni, Kola, and Johnson-Laird (2018) reported a meta-analysis of 228 experiments using the Wason selection task. This data corpus was used to evaluate sixteen different theories on the basis of three predictions: 1) the occurrence of canonical selections, 2) dependencies in selections, and 3) the effect of counter-example salience. Ragni et al. argued that all three effects cull the number of candidate theories down to only two, which are subsequently compared in a model-selection analysis. The present comment argues against the diagnostic value attributed to some of these predictions. Moreover, we revisit Ragni et al.’s model-selection analysis and show that the model they propose is non-identifiable and often fails to account for the data. Altogether, the problems discussed here suggest that we are still far from a much-needed theoretical winnowing.

Perspectives on Correctness in Probabilistic Inference from Psychology

Research into decision making has enabled us to appreciate that the notion of correctness is multifaceted. Different normative framework for correctness can lead to different insights about correct behavior. We illustrate the shifts for correctness insights with two tasks, the Wason selection task and the conjunction fallacy task; these tasks have had key roles in the development of logical reasoning and decision making research respectively. The Wason selection task arguably has played an important part in the transition from understanding correctness using classical logic to classical probability theory (and information theory). The conjunction fallacy has enabled a similar shift from baseline classical probability theory to quantum probability. The focus of this overview is the latter, as it represents a novel way for understanding probabilistic inference in psychology. We conclude with some of the current challenges concerning the application of quantum probability theory in psychology in general and specifically for the problem of understanding correctness in decision making.