A High Fidelity Mixed Reality System for Remote Collaboration

<p>This thesis presents a novel system for enabling remote collaboration within a mixed reality environment. Since the increase of virtual and augmented reality headsets, there has been increased interest in improving remote collaboration. Systems have been proposed to use 3D geometry or 360° video for providing remotely collaborating users with a view of the local, real-world environment. However, many systems provide limited interactions in the local environment and target using coupled views of all users, rather than simulating face-to-face interactions, or use virtual environments for the remote user, losing visual realism. The presented system enables a user situated in a remote location to join a local user to collaborate on a task. An omni-directional camera is streamed to the remote user in real-time to provide a live view of the local space. The 360° video is also used to provide believable lighting when compositing virtual objects into the real-world. Remote users are displayed to local users as an abstracted avatar to provide basic body gestures and social presence. Voice chat is also provided for verbal communication. The system has been evaluated for technical performance and user experience. The evaluation found the performance of the system was suitable for real-time collaboration. Remote and local users were also found to have similar satisfaction with the system, experiencing high levels of presence, social presence and tele-presence. Shared cinematic and remote presentations are suggested as possible applications to guide further development of the system.</p>

A High Fidelity Mixed Reality System for Remote Collaboration

<p>This thesis presents a novel system for enabling remote collaboration within a mixed reality environment. Since the increase of virtual and augmented reality headsets, there has been increased interest in improving remote collaboration. Systems have been proposed to use 3D geometry or 360° video for providing remotely collaborating users with a view of the local, real-world environment. However, many systems provide limited interactions in the local environment and target using coupled views of all users, rather than simulating face-to-face interactions, or use virtual environments for the remote user, losing visual realism. The presented system enables a user situated in a remote location to join a local user to collaborate on a task. An omni-directional camera is streamed to the remote user in real-time to provide a live view of the local space. The 360° video is also used to provide believable lighting when compositing virtual objects into the real-world. Remote users are displayed to local users as an abstracted avatar to provide basic body gestures and social presence. Voice chat is also provided for verbal communication. The system has been evaluated for technical performance and user experience. The evaluation found the performance of the system was suitable for real-time collaboration. Remote and local users were also found to have similar satisfaction with the system, experiencing high levels of presence, social presence and tele-presence. Shared cinematic and remote presentations are suggested as possible applications to guide further development of the system.</p>

Immersive Virtual Reality Experience of Historical Events Using Haptics and Locomotion Simulation

Virtual reality (VR) and 3D modeling technologies have become increasingly powerful tools for multiple fields, such as education, architecture, and cultural heritage. Museums are no longer places for only placing and exhibiting collections and artworks. They use such technologies to offer a new way of communicating art and history with their visitors. In this paper, we present the initial results of a proposed workflow towards highlighting and interpreting a historic event with the use of an immersive and interactive VR experience and the utilization of multiple senses of the user. Using a treadmill for navigating and haptic gloves for interacting with the environment, combined with the detailed 3D models, deepens the sense of immersion. The results of our study show that engaging multiple senses and visual manipulation in an immersive 3D environment can effectively enhance the perception of visual realism and evoke a stronger sense of presence, amplifying the educational and informative experience in a museum.

Influence of Realistic Virtual Environments and Humanlike Avatars on Patients With Social Phobia

The practice of treating phobias with Virtual Reality-based therapies is a well-established field. Understanding the level of realism required by the therapy to be most effective is an essential matter of study. This research aims to explore the effects of visual realism on the emotional response in subjects with social phobia when exposed to VR-based applications. Social phobias are triggered by the presence of other people, which translated into virtual environments, refers to avatars. Our hypothesis is that patients with social phobia experience different emotional response to humanlike avatars compared to people without social phobia. To try the hypothesis, a prototype-based survey is conducted. Three types of avatars are implemented with different levels of human likeness: low, medium, and high. The analysis of the collected data suggests that for people with social phobias the anxiety is lowest for avatars with high levels of human likeness. This result is in direct contrast with the uncanny valley effect theory. The research explores how we should design virtual environments to make them more effective in the treatment of phobias. Moreover, the research produces new knowledge about the perception of humanlike avatars in virtual reality.

Impact of View-Dependent Image-Based Effects on Perception of Visual Realism and Presence in Virtual Reality Environments Created Using Multi-Camera Systems

Several recent works have presented image-based methods for creating high-fidelity immersive virtual environments from photographs of real-world scenes. In this paper, we provide a user-centered evaluation of such methods by way of a user study investigating their impact on viewers’ perception of visual realism and sense of presence. In particular, we focus on two specific elements commonly introduced by image-based approaches. First, we investigate the extent to which using dedicated image-based rendering algorithms to render the scene with view-dependent effects (such as specular highlights) causes users to perceive it as being more realistic. Second, we study whether making the scene fade out beyond a fixed volume in 3D space significantly reduces participants’ feeling of being there, examining different sizes for this viewing volume. To provide details on the virtual environment used in the study, we also describe how we recreated a museum gallery for room-scale virtual reality using a custom-built multi-camera rig. The results of our study show that using image-based rendering to render view-dependent effects can effectively enhance the perception of visual realism and elicit a stronger sense of presence, even when it implies constraining the viewing volume to a small range of motion.

Towards Indistinguishable Augmented Reality

Adding virtual information that is indistinguishable from reality has been a long-awaited goal in Augmented Reality (AR). While already demonstrated in the 1960s, only recently have Optical See-Through Head-Mounted Displays (OST-HMDs) seen a reemergence, partially thanks to large investments from industry, and are now considered to be the ultimate hardware for augmenting our visual perception. In this article, we provide a thorough review of state-of-the-art OST-HMD-related techniques that are relevant to realize the aim of an AR interface almost indistinguishable from reality. In this work, we have an initial look at human perception to define requirements and goals for implementing such an interface. We follow up by identifying three key challenges for building an OST-HMD-based AR interface that is indistinguishable from reality: spatial realism, temporal realism, and visual realism. We discuss existing works that aim to overcome these challenges while also reflecting against the goal set by human perception. Finally, we give an outlook into promising research directions and expectations for the years to come.

Seeing and envisioning: camera-based practice, democratic praxis, and socially engaged arts in Toronto

This project explores the varied ways cameras have become integrated into contemporary socially engaged arts practices. The emergent, participatory, and inclusive characteristics of these diverse practices are increasingly common in contemporary art and culture, with cooperative processes, community activism, formal experimentation, and public involvement being regarded, now more than ever, as legitimate strategies for developing artistic form and content. This project considers the innovative uses of cameras in these practices, arguing that such uses are not simply convenient or instrumental, but are often critical mediations between visual realism and cultural expressivity. The dissertation begins to address a gap in research on material practices in the cultural production of art by elaborating a theory of socially engaged camera arts. Drawn from ethnographic research in the Toronto community arts/socially engaged arts ecology, this theory begins to describe how camera practices seem to be moving beyond traditional image production practices in order to support and even help envision broader repertoires of practice in processes of social and cultural action. The dissertation develops three interrelated theoretical frames - expansion, organization and pedagogy - to insist on the key place of socially engaged camera arts, and camera arts in general, in the iterative, activist-led revitalization of community cultural infrastructures.

Seeing and envisioning: camera-based practice, democratic praxis, and socially engaged arts in Toronto

This project explores the varied ways cameras have become integrated into contemporary socially engaged arts practices. The emergent, participatory, and inclusive characteristics of these diverse practices are increasingly common in contemporary art and culture, with cooperative processes, community activism, formal experimentation, and public involvement being regarded, now more than ever, as legitimate strategies for developing artistic form and content. This project considers the innovative uses of cameras in these practices, arguing that such uses are not simply convenient or instrumental, but are often critical mediations between visual realism and cultural expressivity. The dissertation begins to address a gap in research on material practices in the cultural production of art by elaborating a theory of socially engaged camera arts. Drawn from ethnographic research in the Toronto community arts/socially engaged arts ecology, this theory begins to describe how camera practices seem to be moving beyond traditional image production practices in order to support and even help envision broader repertoires of practice in processes of social and cultural action. The dissertation develops three interrelated theoretical frames - expansion, organization and pedagogy - to insist on the key place of socially engaged camera arts, and camera arts in general, in the iterative, activist-led revitalization of community cultural infrastructures.