A Normative Account of Confirmation Bias During Reinforcement Learning

Reinforcement learning involves updating estimates of the value of states and actions on the basis of experience. Previous work has shown that in humans, reinforcement learning exhibits a confirmatory bias: when the value of a chosen option is being updated, estimates are revised more radically following positive than negative reward prediction errors, but the converse is observed when updating the unchosen option value estimate. Here, we simulate performance on a multi-arm bandit task to examine the consequences of a confirmatory bias for reward harvesting. We report a paradoxical finding: that confirmatory biases allow the agent to maximize reward relative to an unbiased updating rule. This principle holds over a wide range of experimental settings and is most influential when decisions are corrupted by noise. We show that this occurs because on average, confirmatory biases lead to overestimating the value of more valuable bandits and underestimating the value of less valuable bandits, rendering decisions overall more robust in the face of noise. Our results show how apparently suboptimal learning rules can in fact be reward maximizing if decisions are made with finite computational precision.

Replies to Commentators

This chapter responds to the commentaries by Kolodny, Brooks, and Stilz by elaborating on and adding to points made in the first three chapters. In connection with Chapter 1, it addresses various aspects of the requirement to communicate respect, including the collective character of the required communication, the effectiveness and appropriateness of law as its form, the need for equal participation in crafting it, and the possibility of individual dissent from it. It also considers whether the communicative conception offers a plausible normative account of the motivations underlying democratic movements. In connection with Chapters 2 and 3, it expands on the democratic character of common law and defends the claim that states may pursue discretionary interests, arguing that this pursuit is compatible not only with specific requirements of justice but also with liberalism.

Burdens of Proof

This chapter provides a brief introduction to the scholarly conversation concerning burdens of persuasion. An adequate account of burdens must first explain what case-related facts the burden draws upon to produce outcomes. I review a variety of answers to this question, including probability threshold, likelihood ratio, belief function, weight-of-evidence, explanatory, and story-based approaches. I then identify several key questions that theories must answer with respect to inputs and show that the best answer on any given question must depend on whether the theory is advanced as a psychological, doctrinal, or normative account. The remainder of the chapter considers varying methods of transforming these inputs into case outcomes, including fixed thresholds, variable thresholds, multi-stepped, and continuous approaches. With respect to these choices, the problem of describing current practices is much easier, but the normative debates are harder to resolve.

Defining Stereotypes and Stereotyping

How Stereotypes Deceives Us aims to illuminate the conditions under which stereotypes and stereotyping lead to misperceptions and misjudgements, but what exactly are stereotypes and what is stereotyping? This chapter defends the definitions of stereotypes and stereotyping that are adopted throughout this book. In particular, this chapter defends a non-normative conception of stereotyping, according to which stereotypes can be accurate or inaccurate, and stereotyping can be distorting or non-distorting. Existing arguments in favour of the non-normative account are critically evaluated before a pragmatic argument is presented and defended. It is argued that stereotypes should be defined as comparative social attitudes that make distinctions between social groups. Reasons are given for accepting that social attitudes other than beliefs, including implicit attitudes, should count as stereotypes.

How is political corruption wrong?

This chapter offers a normative account of the threat political corruption poses to institutional well-functioning. When political corruption occurs against the background of legitimate or nearly just institutions, it is inherently wrong because it constitutes a wrongful form of interaction between officeholders. The idea of interactive injustice is used to qualify this kind of relational wrong. The way officeholders treat each other in their institutional interactions should be governed by a regulative principle of office accountability. When officeholders fail office accountability by acting in a corrupt way or by participating in corrupt institutional practices, they alter ipso facto the normative order of just interactions constitutive of their institution. This alteration indicates how political corruption is inherently unjust as a violation of the duty of office accountability, even in the absence of identifiable consequences. This normative view of political corruption is distinguished from other views based on impartiality or political equality.

Solidarity, Trust, and Christian Faith in the Doctor–Patient Relationship

In this article, we first give a normative account of the doctor–patient relationship as: oriented to the good of the patient’s health; motivated by a vocational commitment; and characterized by solidarity and trust. We then look at the difference that Christianity can, and we believe, should, make to that relationship, so understood. In doing so, we consolidate and expand upon some claims we have made in a forthcoming book, Ethics and the Healing Profession (Curlin and Tollefsen, 2021).1