A review of Virtual Assistants' characteristics: recommendations for designing an optimal human-machine cooperation

Designed to improve human-machine interactions, virtual agents, and particularly virtual assistants (VAs), are spreading in our daily lives. Presenting a very wide variety of characteristics, studies generally report their own agent with its own characteristics and objective. So we can wonder if some of these characteristics are a consensus for VAs in general. Within this work, we aim to identify the agents' characteristics that should be considered when designing a virtual assistant promoting the best communication and cooperation between man and machine. We review the aspects of representation of the agent (embodied or not) and its ability to interact with the human being whether by speech or gestures, but also by displaying personality traits. This overview makes some focuses on virtual assistance of any kind embarked on vehicles.

Social Presence as a Moderator of the Effect of Agent Behavior on Emotional Experience in Social Interactions in Virtual Reality

Background: Exposure therapy involves exposure to feared stimuli and is considered to be the gold-standard treatment for anxiety disorders. While its application in Virtual Reality (VR) has been very successful for phobic disorders, the effects of exposure to virtual social stimuli in Social Anxiety Disorder are heterogeneous. This difference has been linked to demands on realism and presence, particularly social presence, as a pre-requisite in evoking emotional experiences in virtual social interactions. So far, however, the influence of social presence on emotional experience in social interactions with virtual agents remains unknown.Objective: We investigated the relationship between realism and social presence and the moderating effect of social presence on the relationship between agent behavior and experienced emotions in virtual social interaction.Methods: Healthy participants (N = 51) faced virtual agents showing supportive and dismissive behaviors in two virtual environments (short interactions and oral presentations). At first, participants performed five blocks of short one-on-one interactions with virtual agents (two male and two female agents per block). Secondly, participants gave five presentations in front of an audience of 16 agents. In each scenario, agent behavior was a within subjects factor, resulting in one block of neutral, two blocks of negative, and two blocks of positive agent behavior. Ratings of agent behavior (valence and realism), experience (valence and arousal), and presence (physical and social) were collected after every block. Moderator effects were investigated using mixed linear models with random intercepts. Correlations were analyzed via repeated measures correlations.Results: Ratings of valence of agent behaviors showed reliable relationships with experienced valence and less reliable relationships with experienced arousal. These relationships were moderated by social presence in the presentation scenario. Results for the interaction scenario were weaker but potentially promising for experimental studies. Variations in social presence and realism over time were correlated but social presence proved a more reliable moderator.Conclusion: Our findings emphasize the role of social presence for emotional experience in response to specific agent behaviors in virtual social interactions. While these findings should be replicated with experimental designs and in clinical samples, variability in social presence might account for heterogeneity in efficacy of virtual exposure to treat social anxiety disorder.

Tacit Knowledge Elicitation Process for Industry 4.0

Manufacturers migrate their processes to Industry 4.0 that includes new technologies for improving productivity and efficiency of operations. One of the issues is capturing, recreating, and documenting the tacit knowledge of the aging workers. However, there are no systematic procedures to incorporate this knowledge into Enterprise Resource Planning systems for maintaining a competitive advantage. This paper describes a solution proposal for a tacit knowledge elici-tation process for capturing operational best practices of experienced workers in industrial domains for on-site training. We use Concept Maps mining and domain ontologies to discover and integrate information into a Knowledge Graph in a role game played by human and virtual agents. Ethical and societal concerns are discussed as well.

Acceptability of Virtual Characters as a Social Skills Trainer: Usability Study (Preprint)

BACKGROUND Social skills training by human trainers is a well-established method to obtain appropriate social interaction skills and strengthen social self-efficacy. Our previous works automated social skills training by developing a virtual agent that teaches social skills through interaction. This study attempts to investigate the effect of virtual agent design on automated social skills training. However previous works have not investigated virtual agent design for virtual social skills trainers. OBJECTIVE The three main purposes of this research are summarized: to investigate virtual agent appearance for automated SST, to investigate the relationship between acceptability and other measures (likeability, acceptability, realism, and familiarity), and to investigate the relationship between likeability and an individual’s characteristics (gender, age, and autistic traits). METHODS We prepared images and videos of a virtual agent, and 1,218 crowdsourced workers rated the virtual agents through a questionnaire. In designing personalized virtual agents, we investigated the acceptability, likeability, and other impressions of the virtual agents and their relationship to the individuals’ characteristics. RESULTS As a result, we found the difference between the virtual agents in all measures (P < 0.001). A female anime-type virtual agent was rated as the most likeable. We also confirmed that participants’ gender, age, and autistic traits are related to the ratings. CONCLUSIONS We confirmed the effect of virtual agent design on automated social skills training. Our findings are important in designing the appearance of an agent for use in personalized automated social skills training.

Social buffering of human fear is shaped by gender, social concern, and the presence of real vs virtual agents

The presence of a partner can attenuate physiological fear responses, a phenomenon known as social buffering. However, not all individuals are equally sociable. Here we investigated whether social buffering of fear is shaped by sensitivity to social anxiety (social concern) and whether these effects are different in females and males. We collected skin conductance responses (SCRs) and affect ratings of female and male participants when they experienced aversive and neutral sounds alone (alone treatment) or in the presence of an unknown person of the same gender (social treatment). Individual differences in social concern were assessed based on a well-established questionnaire. Our results showed that social concern had a stronger effect on social buffering in females than in males. The lower females scored on social concern, the stronger the SCRs reduction in the social compared to the alone treatment. The effect of social concern on social buffering of fear in females disappeared if participants were paired with a virtual agent instead of a real person. Together, these results showed that social buffering of human fear is shaped by gender and social concern. In females, the presence of virtual agents can buffer fear, irrespective of individual differences in social concern. These findings specify factors that shape the social modulation of human fear, and thus might be relevant for the treatment of anxiety disorders.

A Unified Dialogue Management Strategy for Multi-intent Dialogue Conversations in Multiple Languages

Building Virtual Agents capable of carrying out complex queries of the user involving multiple intents of a domain is quite a challenge, because it demands that the agent manages several subtasks simultaneously. This article presents a universal Deep Reinforcement Learning framework that can synthesize dialogue managers capable of working in a task-oriented dialogue system encompassing various intents pertaining to a domain. The conversation between agent and user is broken down into hierarchies, to segregate subtasks pertinent to different intents. The concept of Hierarchical Reinforcement Learning, particularly options , is used to learn policies in different hierarchies that operates in distinct time steps to fulfill the user query successfully. The dialogue manager comprises top-level intent meta-policy to select among subtasks or options and a low-level controller policy to pick primitive actions to communicate with the user to complete the subtask provided to it by the top-level policy in varying intents of a domain. The proposed dialogue management module has been trained in a way such that it can be reused for any language for which it has been developed with little to no supervision. The developed system has been demonstrated for “Air Travel” and “Restaurant” domain in English and Hindi languages. Empirical results determine the robustness and efficacy of the learned dialogue policy as it outperforms several baselines and a state-of-the-art system.

Virtual Reality Applications in Medicine During the COVID-19 Pandemic: PRISMA Systematic Review (Preprint)

BACKGROUND Since the COVID-19 outbreak, the adoption rate of virtual reality in medicine has seen a massive rise. Many hospitals and medical universities rushed to implement virtual reality to remotely provide medical treatment or medical education and training. OBJECTIVE This systematic review aimed to describe the literature on virtual reality applications during the COVID-19 crisis to treat mental and physical health conditions and for medical education and training. METHODS A systematic search of the literature was made following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. It was pre-registered in the International Platform of Registered Systematic Review and Meta-analysis Protocols (INPLASY)— INPLASY202190108. The search databases were PsycINFO, Web of Science, and Medline. The search string was: [(“virtual reality”)] AND [(“COVID-19”)]. RESULTS N=44 studies met inclusion criteria during the period 2020 – 2021. CONCLUSIONS Findings show the benefit of virtual reality for treating several mental health conditions during the COVID-19 pandemic, including stress, anxiety, and depression, and for cognitive rehabilitation. Besides, VR was useful to promote physical exercise and for the management of chronic pain. As regards education and training, virtual reality resulted an effective learning tool during the COVID-19 pandemic in many medical areas such as nursing, pediatry,cardiology, and urology. The majority of the retrieved studies recruited young adults. Studies showed the usefulness of VR for the treatment of health problems and for medical education and training both in the format with high immersion (i.e., immersive VR) and in that with a low level of immersion (i.e., desktop VR). Various VR systems (i.e., PC-based, mobile, standalone) and contents (i.e., 360° videos and photos, virtual environments, VR games, embodied virtual agents) showed positive results. Finally, VR has been used successfully in both face-to-face and remote trials.