Chapter 9 - Theory of Mind and Joint Attention - The Handbook on Socially Interactive Agents

The present chapter provides an overview from the perspective of social cognitive neuroscience (SCN) regarding theory of mind (ToM) and joint attention (JA) as crucial mechanisms of social cognition and discusses how these mechanisms have been investigated in social interaction with artificial agents. In the final sections, the chapter reviews computational models of ToM and JA in social robots (SRs) and intelligent virtual agents (IVAs) and discusses the current challenges and future directions.

Voice-enabled Intelligent Virtual Agents for People with Amnesia: A Systematic Review (Preprint)

BACKGROUND Older adults often have increasing memory problems, and worldwide about 50 million people have dementia. This syndrome gradually affects a patient over a period of 10-20 years. Intelligent virtual agents may support people suffering from memory problems. OBJECTIVE To identify the state of the art of experimental studies with virtual agents on a screen capable of verbal dialogues with older adults with memory problems. METHODS Conduct a systematic search into selected databases PubMed, SCOPUS, Microsoft Academic, Google Scholar, Web of Science and CrossRef on Virtual Agent and Memory Problems on papers that describe such experiments. Search criteria were (“Virtual Agent” OR “Virtual Assistant” OR “Virtual Human” OR “Conversational Agent” OR “Virtual Coach” OR Chatbot) AND (Dementia OR Alzheimer OR Amnesia OR “Mild Cognitive Impairment”). Risk of bias has been evaluated using the QualSyst tool that scores 14 study quality items. Eligible studies are reported in a table including country, study design type, target sample size, controls, study aims, experiment population, intervention details, results and an image of the agent. RESULTS Nine studies were included. The average number of participants in the studies was 18 (SD=12). The verbal interactions were generally short. The human utterance consisted in 8 out of 9 studies out of short words or phrases that were predefined in the agent’s speech recognition algorithm. The average study quality score was .68 (SD=.08) on a scale 0-1.The number of experimental studies on talking virtual agents that support people with memory problems is still small. The details on the verbal interaction are limited, which make it difficult to assess the quality of that interaction and the possible effect of confounding parameters. Further research is needed with extended and prolonged dialogues. CONCLUSIONS The number of experimental studies on talking virtual agents that support people with memory problems is still small. The details on the verbal interaction are limited, which make it difficult to assess the quality of that interaction and the possible effect of confounding parameters. Further research is needed with extended and prolonged dialogues.

Non-native speaker perception of Intelligent Virtual Agents in two languages: the impact of amount and type of grammatical mistakes

Having a mixed-cultural membership becomes increasingly common in our modern society. It is thus beneficial in several ways to create Intelligent Virtual Agents (IVAs) that reflect a mixed-cultural background as well, e.g., for educational settings. For research with such IVAs, it is essential that they are classified as non-native by members of a target culture. In this paper, we focus on variations of IVAs’ speech to create the impression of non-native speakers that are identified as such by speakers of two different mother tongues. In particular, we investigate grammatical mistakes and identify thresholds beyond which the agents is clearly categorised as a non-native speaker. Therefore, we conducted two experiments: one for native speakers of German, and one for native speakers of English. Results of the German study indicate that beyond 10% of word order mistakes and 25% of infinitive mistakes German-speaking IVAs are perceived as non-native speakers. Results of the English study indicate that beyond 50% of omission mistakes and 50% of infinitive mistakes English-speaking IVAs are perceived as non-native speakers. We believe these thresholds constitute helpful guidelines for computational approaches of non-native speaker generation, simplifying research with IVAs in mixed-cultural settings.

Virtual Reality in Object Location

Currently, virtual reality (VR) is a computer technology that is growing in terms of developments and discoveries. Virtual reality has been introduced in different areas due to the growing interest it has caused in people. The development of applications with virtual reality is increasingly varied, covering activities, tasks, or processes of everyday life in the fields of industry, education, medicine, tourism, art, entertainment, design, and modeling of objects, among others. This chapter will focus on describing the latest advances and developments in virtual reality within the scope of representing reality in the process of locating objects. With the support of virtual environments and intelligent virtual agents, the author has managed to develop a computational model that generates indications in natural language, for the location of objects considering spatial and cognitive aspects of the users.

An Exploratory Study on the Effect of Artificial Intelligence-Enabled Technology on Customer Experiences in the Banking Sector

This study investigates how artificial intelligent technology in the banking sector has affected consumers’ overall experience. It focuses on how consumers’ personal digital transformation has affected digital banking development and how this further affects consumer’s expectations and experience. It assesses how banks use Artificial Intelligent Virtual Agents such as Chatbots to transform how consumers use their banking facilities. Lastly, this study investigates the scope of neobanks in the banking sector. The study found that digital transformations have led to an increase in consumers’ expectations from their banks. Whilst banks revolutionise their customer service offerings through virtual agents, customers are not engaging with these at an expected rate. Findings revealed that Neobanks are not operating at their expected traction due to consumer knowledge gaps, occasioned by a lack of advertised information to customers from their banks.

An Exploratory Study on the Effect of Artificial Intelligence-Enabled Technology on Customer Experiences in the Banking Sector

This study investigates how artificial intelligent technology in the banking sector has affected consumers’ overall experience. It focuses on how consumers’ personal digital transformation has affected digital banking development and how this further affects consumer’s expectations and experience. It assesses how banks use Artificial Intelligent Virtual Agents such as Chatbots to transform how consumers use their banking facilities. Lastly, this study investigates the scope of neobanks in the banking sector. The study found that digital transformations have led to an increase in consumers’ expectations from their banks. Whilst banks revolutionise their customer service offerings through virtual agents, customers are not engaging with these at an expected rate. Findings revealed that Neobanks are not operating at their expected traction due to consumer knowledge gaps, occasioned by a lack of advertised information to customers from their banks.

Towards Awareness of Human Relational Strategies in Virtual Agents

As Intelligent Virtual Agents (IVAs) increase in adoption and further emulate human personalities, we are interested in how humans apply relational strategies to them compared to other humans in a service environment. Human-computer data from three live customer service IVAs was collected, and annotators marked all text that was deemed unnecessary to the determination of user intention as well as the presence of multiple intents. After merging the selections of multiple annotators, a second round of annotation determined the classes of relational language present in the unnecessary sections such as Greetings, Backstory, Justification, Gratitude, Rants, or Expressing Emotions. We compare the usage of such language in human-human service interactions. We show that removal of this language from task-based inputs has a positive effect by both an increase in confidence and improvement in responses, as evaluated by humans, demonstrating the need for IVAs to anticipate relational language injection. This work provides a methodology to identify relational segments and a baseline of human performance in this task as well as laying the groundwork for IVAs to reciprocate relational strategies in order to improve their believeability.