Social Interaction With Agents and Avatars in Immersive Virtual Environments: A Survey

Immersive virtual reality technologies are used in a wide range of fields such as training, education, health, and research. Many of these applications include virtual humans that are classified into avatars and agents. An overview of the applications and the advantages of immersive virtual reality and virtual humans is presented in this survey, as well as the basic concepts and terminology. To be effective, many virtual reality applications require that the users perceive and react socially to the virtual humans in a realistic manner. Numerous studies show that people can react socially to virtual humans; however, this is not always the case. This survey provides an overview of the main findings regarding the factors affecting the social interaction with virtual humans within immersive virtual environments. Finally, this survey highlights the need for further research that can lead to a better understanding of human–virtual human interaction.

Realistic Virtual Humans for Cultural Heritage Applications

Virtual Humans are becoming a commodity in computing technology and lately have been utilized in the context of interactive presentations in Virtual Cultural Heritage environments and exhibitions. To this end, this research work underlines the importance of aligning and fine-tuning Virtual Humans’ appearance to their roles and highlights the importance of affective components. Building realistic Virtual Humans was traditionally a great challenge requiring a professional motion capturing studio and heavy resources in 3D animation and design. In this paper, a workflow for their implementation is presented, based on current technological trends in wearable mocap systems and advancements in software technology for their implementation, animation, and visualization. The workflow starts from motion recording and segmentation to avatar implementation, retargeting, animation, lip synchronization, face morphing, and integration to a virtual or physical environment. The testing of the workflow occurs in a use case for the Mastic Museum of Chios and the implementation is validated both in a 3D virtual environment accessed through Virtual Reality and on-site at the museum through an Augmented Reality application. The findings, support the initial hypothesis through a formative evaluation, and lessons learned are transformed into a set of guidelines to support the replication of this work.

Practical Language Processing for Virtual Humans

NPCEditor is a system for building a natural language processing component for virtual humans capable of engaging a user in spoken dialog on a limited domain. It uses a statistical language classification technology for mapping from user's text input to system responses. NPCEditor provides a user-friendly editor for creating effective virtual humans quickly. It has been deployed as a part of various virtual human systems in several applications.

Affordable But Not Cheap: A Case Study of the Effects of Two 3D-Reconstruction Methods of Virtual Humans

Realistic and lifelike 3D-reconstruction of virtual humans has various exciting and important use cases. Our and others’ appearances have notable effects on ourselves and our interaction partners in virtual environments, e.g., on acceptance, preference, trust, believability, behavior (the Proteus effect), and more. Today, multiple approaches for the 3D-reconstruction of virtual humans exist. They significantly vary in terms of the degree of achievable realism, the technical complexities, and finally, the overall reconstruction costs involved. This article compares two 3D-reconstruction approaches with very different hardware requirements. The high-cost solution uses a typical complex and elaborated camera rig consisting of 94 digital single-lens reflex (DSLR) cameras. The recently developed low-cost solution uses a smartphone camera to create videos that capture multiple views of a person. Both methods use photogrammetric reconstruction and template fitting with the same template model and differ in their adaptation to the method-specific input material. Each method generates high-quality virtual humans ready to be processed, animated, and rendered by standard XR simulation and game engines such as Unreal or Unity. We compare the results of the two 3D-reconstruction methods in an immersive virtual environment against each other in a user study. Our results indicate that the virtual humans from the low-cost approach are perceived similarly to those from the high-cost approach regarding the perceived similarity to the original, human-likeness, beauty, and uncanniness, despite significant differences in the objectively measured quality. The perceived feeling of change of the own body was higher for the low-cost virtual humans. Quality differences were perceived more strongly for one’s own body than for other virtual humans.

Social Presence Outside the Augmented Reality Field of View

Augmented reality headsets in use today have a large area in which the real world can be seen, but virtual content cannot be displayed. Users perceptions of content in this area is not well understood. This work studies participants perception of a virtual character in this area by grounding this question in relevant theories of perception and performing a study using both behavioral and self-report measures. We find that virtual characters within the augmented periphery receive lower social presence scores, but we do notfind a difference in task performance. These findings inform application design and encourage future work in theories of AR perception and perception of virtual humans.

Social Responses to Virtual Humans: The Effect of Human-Like Characteristics

As a virtual human is provided with more human-like characteristics, will it elicit stronger social responses from people? Two experiments were conducted to address these questions. The first experiment investigated whether virtual humans can evoke a social facilitation response and how strong that response is when people are given different cognitive tasks that vary in difficulty. The second experiment investigated whether people apply politeness norms to virtual humans. Participants were tutored either by a human tutor or a virtual human tutor that varied in features and then evaluated the tutor’s performance. Results indicate that virtual humans can produce social facilitation not only with facial appearance but also with voice. In addition, performance in the presence of voice synced facial appearance seems to elicit stronger social facilitation than in the presence of voice only or face only. Similar findings were observed with the politeness norm experiment. Participants who evaluated their tutor directly reported the tutor’s performance more favorably than participants who evaluated their tutor indirectly. This valence toward the voice synced facial appearance had no statistical difference compared to the valence toward the human tutor condition. The results suggest that designers of virtual humans should be mindful about the social nature of virtual humans.