Modeling and Reasoning with Preferences and Ethical Priorities in AI Systems

As AI systems make decisions that affect our lives, we must ensure that these systems operate according to the same constraints, guidelines, and ethical principles that a human would follow. Humans make many complex decisions that rely on their subjective preferences, but their decisions are usually also constrained by these ethical priorities. Hence it is essential to equip AI systems with the tools to evaluate whether or not preferences are compatible with these other priorities. In computer science, the Conditional Preference networks (CP-nets), which graphically represent conditional and qualitative preference relations, can be used to model, combine, and compare subjective preferences and ethical priorities. This chapter proposes that one can use CP-nets to measure the distance between an agent’s subjective preference and the ethical principles of the agent’s community in order to ensure that the decisions of an AI system are aligned with a given set of ethical priorities.

Rational versus Intuitive Outcomes of Reasoning with Preferences: Argumentation Perspective

Reasoning with preference information is a common human activity. As modelling human reasoning is one of the main objectives of AI, reasoning with preferences is an important topic in various fields of AI, such as Knowledge Representation and Reasoning (KR). Argumentation is one particular branch of KR that concerns, among other tasks, modelling common-sense reasoning with preferences. A key issue there, is the lack of consensus on how to deal with preferences. Witnessing this is a multitude of proposals on how to formalise reasoning with preferences in argumentative terms. As a commonality, however, formalisms of argumentation with preferences tend to fulfil various criteria of `"rational" reasoning, notwithstanding the fact that human reasoning is often not `"rational", yet seemingly `"intuitive". In this paper, we study how several formalisms of argumentation with preferences model human intuition behind a particular common-sense reasoning problem. More specifically, we present a common-sense scenario of reasoning with rules and preferences, complemented with a survey of decisions made by human respondents that indicates an "intuitive" solution, and analyse how this problem is tackled in argumentation. We conclude that most approaches to argumentation with preferences afford a ``"rational" solution to the problem, and discuss one recent formalism that yields the "intuitive" solution instead. We argue that our results call for advancements in the area of argumentation with preferences in particular, as well as for further studies of reasoning with preferences in AI at large.

“Reasoning with Preferences?”

Rationality requires certain things of you. It requires you not to have contradictory beliefs or intentions, not to intend something you believe to be impossible, to believe what obviously follows from something you believe, and so on. Its requirements can be expressed using schemata such as:Modus ponens. Rationality requires of N that, if N believes p and N believe that if p then q, then N believes q.Necessary means. Rationality requires of N that, if N intends that e, and if N believes that e will be so only if m is so, and if N believes m will be so only if she intends that m, then N intends that m.Krasia. Rationality requires of N that, if N believes she ought to F, and if N believes she will F only if she intends to F, then N intends to F.

Reasoning with preferences?

Rationality requires certain things of you. It requires you not to have contradictory beliefs or intentions, not to intend something you believe to be impossible, to believe what obviously follows from something you believe, and so on. Its requirements can be expressed using schemata such as: