Developing Conversational Agents for Use in Criminal Investigations

The adoption of artificial intelligence (AI) systems in environments that involve high risk and high consequence decision-making is severely hampered by critical design issues. These issues include system transparency and brittleness, where transparency relates to (i) the explainability of results and (ii) the ability of a user to inspect and verify system goals and constraints; and brittleness, (iii) the ability of a system to adapt to new user demands. Transparency is a particular concern for criminal intelligence analysis, where there are significant ethical and trust issues that arise when algorithmic and system processes are not adequately understood by a user. This prevents adoption of potentially useful technologies in policing environments. In this article, we present a novel approach to designing a conversational agent (CA) AI system for intelligence analysis that tackles these issues. We discuss the results and implications of three different studies; a Cognitive Task Analysis to understand analyst thinking when retrieving information in an investigation, Emergent Themes Analysis to understand the explanation needs of different system components, and an interactive experiment with a prototype conversational agent. Our prototype conversational agent, named Pan, demonstrates transparency provision and mitigates brittleness by evolving new CA intentions. We encode interactions with the CA with human factors principles for situation recognition and use interactive visual analytics to support analyst reasoning. Our approach enables complex AI systems, such as Pan, to be used in sensitive environments, and our research has broader application than the use case discussed.

Download Full-text

Providing a foundation for interpretable autonomous agents through elicitation and modeling of criminal investigation pathways

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1071181320641057 ◽

2020 ◽

Vol 64 (1) ◽

pp. 239-243

Author(s):

Sam Hepenstal ◽

Leishi Zhang ◽

Neesha Kodogoda ◽

B.L. William Wong

Keyword(s):

Artificial Intelligence ◽

Task Analysis ◽

Autonomous Agents ◽

Cognitive Task ◽

Cognitive Task Analysis ◽

Criminal Investigation ◽

Event Tree ◽

Analysis Method ◽

Ethical Concerns ◽

Criminal Intelligence

Criminal investigations are guided by repetitive and time-consuming information retrieval tasks, often with high risk and high consequence. If Artificial intelligence (AI) systems can automate lines of inquiry, it could reduce the burden on analysts and allow them to focus their efforts on analysis. However, there is a critical need for algorithmic transparency to address ethical concerns. In this paper, we use data gathered from Cognitive Task Analysis (CTA) interviews of criminal intelligence analysts and perform a novel analysis method to elicit question networks. We show how these networks form an event tree, where events are consolidated by capturing analyst intentions. The event tree is simplified with a Dynamic Chain Event Graph (DCEG) that provides a foundation for transparent autonomous investigations.

Download Full-text

How analysts think: A preliminary study of human needs and demands for AI-based conversational agents

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1071181319631333 ◽

2019 ◽

Vol 63 (1) ◽

pp. 178-182 ◽

Cited By ~ 1

Author(s):

Sam Hepenstal ◽

B.L. William Wong ◽

Leishi Zhang ◽

Neesha Kodogoda

Keyword(s):

Preliminary Analysis ◽

Intelligence Analysis ◽

Conversational Agent ◽

Conversational Agents ◽

Human Needs ◽

Preliminary Study ◽

Depth Interviews

For conversational agents to provide benefit to intelligence analysis they need to be able to recognise and respond to the analysts intentions. Furthermore, they must provide transparency to their algorithms and be able to adapt to new situations and lines of inquiry. We present a preliminary analysis as a first step towards developing conversational agents for intelligence analysis: that of understanding and modeling analyst intentions so they can be recognised by conversational agents. We describe in-depth interviews conducted with experienced intelligence analysts and implications for designing conversational agent intentions.

Download Full-text

Characterizing the intelligence analysis process through a longitudinal field study: Implications for visual analytics

Information Visualization ◽

10.1177/1473871612468877 ◽

2012 ◽

Vol 13 (2) ◽

pp. 134-158 ◽

Cited By ~ 4

Author(s):

Youn-ah Kang ◽

John Stasko

Keyword(s):

Field Study ◽

Visual Analytics ◽

Task Analysis ◽

System Development ◽

Intelligence Analysis ◽

Work Processes ◽

Target Domain ◽

Primary Target ◽

Analysis Process ◽

And Task

While intelligence analysis has been a primary target domain for visual analytics system development, relatively little user and task analysis has been conducted within this area. Our research community’s understanding of the work processes and practices of intelligence analysts is not deep enough to adequately address their needs. Without a better understanding of the analysts and their problems, we cannot build visual analytics systems that integrate well with their work processes and truly provide benefit to them. In order to close this knowledge gap, we conducted a longitudinal, observational field study of intelligence analysts in training within the intelligence program at Mercyhurst College. We observed three teams of analysts, each working on an intelligence problem for a 10-week period. Based on the findings of the study, we describe and characterize processes and methods of intelligence analysis that we observed, make clarifications regarding the processes and practices, and suggest design implications for visual analytics systems for intelligence analysis.

Download Full-text