AbstractIn this article, I will advocate caution against a formalization of ethics by showing that it may produce and perpetuate unjustified power imbalances, disadvantaging those without a proper command of the formalisms, and those not in a position to decide on the formalisms’ use. My focus rests mostly on ethics formalized for the purpose of implementing ethical evaluations in computer science–artificial intelligence, in particular—but partly also extends to the project of applying mathematical rigor to moral argumentation with no direct intention to automate moral deliberation. Formal ethics of the latter kind can, however, also be seen as a facilitator of automated ethical evaluation. I will argue that either form of formal ethics presents an obstacle to inclusive and fair processes for arriving at a society-wide moral consensus. This impediment to inclusive moral deliberation may prevent a significant portion of society from acquiring a deeper understanding of moral issues. However, I will defend the view that such understanding supports genuine and sustained moral progress. From this, it follows that formal ethics is not per se supportive of moral progress. I will illustrate these arguments by practical examples of manifest asymmetric relationships of power primarily from the domain of autonomous vehicles as well as on more visionary concepts, such as artificial moral advisors. As a result, I will show that in these particular proposed use-cases of formal ethics, machine ethics risks to run contrary to their proponents’ proclaimed promises of increasing the rigor of moral deliberation and even improving human morality on the whole. Instead, I will propose that inclusive discourse about automating ethical evaluations, e.g., in autonomous vehicles, should be conducted with unrelenting transparency about the limitations of implementations of ethics. As an outlook, I will briefly discuss uses formal ethics that are more likely to avoid discrepancies between the ideal of inclusion and the challenge from power asymmetries.Please check and confirm that the authors and their respective affiliations have been correctly identified and amend if necessary.I confirm.Author names: Please confirm if the author names are presented accurately and in the correct sequence (given name, middle name/initial, family name). I confirm. Kindly check and confirm the country name for the affiliation [1] is correct.I confirm.
Human decision-making biases in the moral dilemmas of autonomous vehicles