Beyond RoboDebt

Automated decision support systems with high stake decision processes are frequently controversial. The Online Compliance Intervention (herewith “OCI” or “RoboDebt”) is a system of compliance implemented with the intention to facilitate automatic issuance of statutory debt notices to individuals, taking a receipt of welfare payments and exceeding their entitlement. The system appears to employ rudimentary data scraping and expert systems to determine whether notices should be validly issued. However, many individuals that take receipt of debt notices assert that they were issued in error. The commentary on the system has resulted in a lot of conflation of the system with other system types and caused many to question the role of decision of support systems in public administration given the potentially deleterious impacts of such systems for the most vulnerable. The authors employ a taxonomy of Robotic Process Automation (RPA) issues, to review the OCI and RPA more generally. This paper identifies potential problems of bias, inconsistency, procedural fairness, and overall systematic error. This research also considers a series of RoboDebt specific issues regarding contractor arrangements and the potential impact of the system for Australia's Indigenous population. The authors offer a set of recommendations based on the observed challenges, emphasizing the importance of moderation, independent algorithmic audits, and ongoing reviews. Most notably, this paper emphasizes the need for greater transparency and a broadening of criteria to determine vulnerability that encompasses, temporal, geographic, and technological considerations.