Paying a cost to reduce uncertainty can be adaptive, because better informed decision-makers can align their preferences to opportunities. However, birds and mammals display an appetite for information that they cannot use to functionally alter behaviour or its outcomes. We explore two putative motivational mechanisms for this paradoxical behaviour. The information hypothesis proposes that reducing uncertainty is reinforcing per se, consistent with the concept of curiosity: a motivation to know, in the absence of instrumental benefits. In contrast, the conditioned reinforcement hypothesis sees information-seeking as a consequence of asymmetries in secondarily acquired reinforcement: responding increments caused by post-choice stimuli announcing positive outcomes (S+) exceed decrements caused by stimuli signalling absence of reward (S−). We contrast these hypotheses experimentally. Rats chose between two equally profitable options delivering food probabilistically after a fixed delay. In the informative option (Info), the outcome (food/no food) was signalled immediately after choice, whereas in the non-informative option (NoInfo) outcomes were uncertain until the delay lapsed. Subjects preferred (Info) when (1) outcomes were signalled by salient auditory cues, (2) only the absence of reward was signalled, and (3) only reward was signalled, though acquisition was slower when rewards were not explicitly signalled. Our results show that a salient good news signal is not required as a conditioned reinforcer to generate paradoxical preferences. Terminal preferences support the information hypothesis but the slower acquisition of (Info) preference when S+ is not present is consistent with the conditioning account. We conclude that both uncertainty reduction and conditioned reinforcement influence choice.
Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning