Automated Risk Assessment in the Criminal Justice Process

This chapter addresses recent concerns about “algorithmic bias,” specifically in the context of the criminal justice process. Starting from a recent controversy about the use of “automated risk assessment tools” in criminal sentencing and parole hearings, where evidence suggests that such tools effectively discriminate against minority defendants, this chapter argues that the problem here has nothing in particular to do with algorithm-assisted reasoning, nor is it in any clear sense a case of epistemic bias. Rather, given the data set that we are given to work with, there is reason to think that no improvement to our epistemic routines would deliver significantly better results. Instead, the bias is effectively encoded into the data set itself, via a long history of institutionalized racism. This suggests a different diagnosis of the problem: in deeply divided societies, there may just be no way to simultaneously satisfy our moral ideals and our epistemic ideals.

From Psychology to Philosophy

Calls to develop a framework for ‘non-ideal epistemology’ have recently gained traction in philosophical discourse, but little detail has yet been offered as to what this might involve. This chapter aims to remedy this shortcoming, both as a broader theoretical development and with specific view toward the epistemology of prejudice. Specifically,this chapter develops the notion of non-ideal epistemology along two dimensions. Along one dimension, constraints arising from distinctive capacity limitations of the human mind (“endogenous non-ideality”) are considered. In another dimension, constraints arising from specific limitations on the information environments that epistemic agents are forced to operate within (“exogenous non-ideality”) are considered. Taking a non-ideal approach to epistemology does not, however, mean giving up on epistemic normativity altogether: to the contrary, this chapter argues that non-ideal epistemology provides the only way for such norms to provide a genuine critical grip on human cognition at all.

The Psychology of Stereotypes

This book has defined prejudice as involving a certain class of negative stereotypes. In chapter 2, this definition is elaborated further in light of developments in social and cognitive psychology. The human mind is an information-processing mechanism operating in real-time. Its defining problem is that of developing effective algorithms to cope with a threat of information overload. These algorithms distinctively involve compression of information, resulting in a predictable loss of fidelity. But even cognitive processing strategies involving significant filtering and compression can be cognitively optimal, relative to our contingent, “non-ideal” cognitive starting points. The basic cognitive expression of this fact is what psychologists call ‘categorization.’ Stereotyping is essentially just categorization applied to the domain of social cognition. As such, no epistemological aspersions can be cast on prejudice simply in virtue of being grounded in stereotypes. The chapter ends by explaining how this research relates to the currently popular concept of ‘implicit bias.’

Introduction

This introductory chapter seeks a preliminary clarification of what prejudice is and why it is a cause of persistent normative concern. It then distinguishes between two normative vocabularies in which that concern can be framed: the moral and the epistemic. When we consider prejudice from a moral point of view, we are concerned with the harms suffered by people who are targeted by prejudiced beliefs, and the moral responsibilities incurred by those who hold these beliefs. When we consider prejudice from an epistemological point of view we are concerned with the cognitive processes by which people come to hold these beliefs. This book is primarily focused on the epistemology of prejudice: the first order of the day is to explain why we should not hope to ground our account of the moral wrongs flowing from prejudice in an account of the epistemic wrongs committed by those who hold these beliefs.

Common Ground

This chapter outlines a way to study the social dynamics of prejudice even in the absence of “prejudiced believers.” Stereotypes often serve to provide us with “social scripts.” We often comply with these scripts even though we don’t endorse their content, simply because we have reason to believe that others endorse them, and because they are typically backed up by sanctions. But others may be in the same situation. Accordingly, we could find ourselves in situations where nobody endorses the stereotypes encoded in our social scripts, even as these scripts continue to govern our mutual interactions. Reverting to notions of “collective” or “shared” epistemic responsibility provides no real traction on these kinds of situations, nor any novel perspectives on remedial action. As a case study, this chapter offers the paradox of “perceived electability,” where voters fail to support a preferred minority candidate because they believe others will not vote for her.

A Better Approach

But maybe the relation between morality and epistemology runs in the opposite direction, and it is the range of our epistemic responsibility which constrains the range of our moral responsibility. Initially, this may seem like an unwelcome thought. To the contrary, this chapter argues that it can serve to empower victims of prejudice seeking redress. Moral responsibility is a multi-dimensional concept: while it is plausible that ascriptions of moral blame track ascriptions of epistemic responsibility, other forms of moral liability do not. Drawing on insights from tort law and discrimination law, this chapter argues that victims’ claims to have been wronged in no way depends on their ability to demonstrate that their victimizers were positioned to know that their actions were wrong. This frees victims of prejudice from the substantial burden of having to show that prejudiced believers are—universally, or in any specific case—epistemically irrational in believing as they do.

The Epistemology of Prejudice Maintenance

But what about prejudice maintenance? Questions of acquisition aside, how could anyone be epistemically rational in retaining their prejudiced belief over time in the face of the significant amounts of contrary evidence confronting them in their everyday lives? This chapter argues that much of this evidence can be easily absorbed by prejudiced believers, in keeping with our best canons of epistemic rationality. Drawing on recent work on generic generalizations, the chapter argues that prejudiced beliefs are in no sense falsified by single contrary instances, or even larger swaths of them. Even when subjects are rationally required to recognize instances as providing contrary evidence, the correct response may simply be to reduce one’s credence in the relevant proposition. The result may be that they are somewhat less prejudiced than before. But they are still prejudiced, even as, by hypothesis, they have responded correctly to their evidence.

Moral Constraints on Belief?

The assumption that moral normativity and epistemic normativity run on separate tracks has recently come under pressure from developments such as “moral encroachment” and “doxastic morality.” Motivating these developments is the idea that in morally charged scenarios—for instance where we stand to impart unwarranted harms on others by forming certain beliefs about them—our epistemic requirements change: beliefs that would be justified by the evidence in a morally inert scenario may no longer be justified once the “moral stakes” are taken into account. In this sense, morality can act as a constraint on rational belief formation. This chapter argues that none of these approaches can carry out the task set for them. Specifically, both founder on the fact that moral and non-moral reasoning are often deeply entangled: even if we agreed about the moral principles, our assessment of who falls under the principles would depend on our further, non-moral beliefs.

The Epistemology of Prejudice Acquisition

The standard analysis of the epistemology of prejudice often assumes that we might simply define prejudice as a type of belief acquired or maintained without regard to one’s evidence, and therefore as involving some kind of breakdown of epistemic rationality. We now move to assess this assumption on its merits. Chapter 4 considers the problem in light of the acquisition of prejudiced belief. It argues that canons of inductive inference as well as considerations from the epistemology of testimony strongly support the view that individuals can come to acquire prejudiced belief without compromising their epistemic rationality. In fact, given the information environments they find themselves in, these might well be the beliefs that they should form, epistemically speaking, in the simple sense these are the beliefs that are best supported by their evidence. There is no conceptual barrier to understanding how people could be epistemically justified in acquiring prejudiced beliefs.

Prejudice from an Epistemological Point of View

The book’s opening chapter begins by providing a working definition of prejudice in terms of negatively charged stereotypes targeting some group of people, and derivatively, the individuals who comprise this group. It then turns to situating this approach in the larger landscape of contemporary epistemological theory. The study of prejudiced belief falls within the ambit of social epistemology. It should also, it is argued, be considered as a form of situated, applied epistemology. As such, it is recognizably a contribution to “non-ideal epistemology” (a notion to be further elaborated in chapter 3): non-ideal epistemology aims to provide normative guidelines for decision-making under uncertainty. Currently popular “externalist” approaches to epistemology are of no help here. But at the same time, non-ideal epistemology is also not “internalist,” since it routinely holds what we are responsible to a broader subset of the total evidence than is currently in our possession.