Effects of the Size of Mixed-Reality Person Representations on Stress and Presence in Telecommunication

We study how mixed reality (MR) telepresence can enhance long-distance human interaction and how altering 3D representations of a remote person can be used to modulate stress and anxiety during social interactions. To do so, we developed an MR telepresence system employing commodity depth sensors and Microsoft’s Hololens. A textured, polygonal 3D model of a person was reconstructed in real time and transmitted over network for rendering in remote location using HoloLens. In this study, we used mock job interview paradigm to induce stress in human–subjects interacting with an interviewer presented as an MR hologram. Participants were exposed to three different types of real-time reconstructed virtual holograms of the interviewer, a natural-sized 3D reconstruction (NR), a miniature 3D reconstruction (SR) and a 2D-display representation (LCD). Participants reported their subjective experience through questionnaires, while their biophysical responses were recorded. We found that the size of 3D representation of a remote interviewer had a significant effect on participants’ stress levels and their sense of presence. The questionnaire data showed that NR condition induced more stress and presence than SR condition and was significantly different from LCD condition. We also found consistent patterns in the biophysical data.

Download Full-text

Real-time mixed-reality telepresence via 3D reconstruction with HoloLens and commodity depth sensors

Proceedings of the 19th ACM International Conference on Multimodal Interaction - ICMI 2017 ◽

10.1145/3136755.3143031 ◽

2017 ◽

Cited By ~ 7

Author(s):

Michal Joachimczak ◽

Juan Liu ◽

Hiroshi Ando

Keyword(s):

3D Reconstruction ◽

Real Time ◽

Mixed Reality ◽

Depth Sensors

Download Full-text

Real-time, High-resolution Depth Upsampling on Embedded Accelerators

ACM Transactions on Embedded Computing Systems ◽

10.1145/3436878 ◽

2021 ◽

Vol 20 (3) ◽

pp. 1-22

Author(s):

David Langerman ◽

Alan George

Keyword(s):

High Resolution ◽

Low Power ◽

Real Time ◽

Mixed Reality ◽

Graphics Processing Unit ◽

Processing Unit ◽

Reconfigurable Logic ◽

Depth Sensors ◽

Time Requirements ◽

Graphics Processing

High-resolution, low-latency apps in computer vision are ubiquitous in today’s world of mixed-reality devices. These innovations provide a platform that can leverage the improving technology of depth sensors and embedded accelerators to enable higher-resolution, lower-latency processing for 3D scenes using depth-upsampling algorithms. This research demonstrates that filter-based upsampling algorithms are feasible for mixed-reality apps using low-power hardware accelerators. The authors parallelized and evaluated a depth-upsampling algorithm on two different devices: a reconfigurable-logic FPGA embedded within a low-power SoC; and a fixed-logic embedded graphics processing unit. We demonstrate that both accelerators can meet the real-time requirements of 11 ms latency for mixed-reality apps. 1

Download Full-text

Poster: Real time hand pose recognition with depth sensors for mixed reality interfaces

2013 IEEE Symposium on 3D User Interfaces (3DUI) ◽

10.1109/3dui.2013.6550233 ◽

2013 ◽

Author(s):

Byungkyu Kang ◽

Mathieu Rodrigue ◽

Tobias Hollerer ◽

Hwasup Lim

Keyword(s):

Real Time ◽

Mixed Reality ◽

Pose Recognition ◽

Depth Sensors ◽

Hand Pose

Download Full-text

A Robust Real-Time 3D Reconstruction Method for Mixed Reality Telepresence

International Journal of Innovative Computing ◽

10.11113/ijic.v10n2.265 ◽

2020 ◽

Vol 10 (2) ◽

Author(s):

Fazliaty Edora Fadzli ◽

Ajune Wanis Ismail

Keyword(s):

3D Reconstruction ◽

Real Time ◽

Real World ◽

Mixed Reality ◽

Real Life ◽

Reconstruction Method ◽

Reconstruction Methods ◽

Virtual Element ◽

World Environment ◽

Full Body

Mixed Reality (MR) is a technology which enable to bring a virtual element into the real-world environment. MR intends to improve reality on the virtual world immerse onto real-world space. Occasionally the MR has been improved as the display technologies advanced progressively. In MR collaborative interface context, the local and remote user work together on collaborative task while sense the immersive environment in the cooperative application. User telepresence is an immersive telepresence, where the reconstruction of a human appears in a real-life. Up till now, producing full telepresence of the life-size human body may require a high transmission bandwidth of the internet. Therefore, this paper explores on a robust real-time 3D reconstruction method for MR telepresence. This paper discusses the previous works on the reconstruction method of a full-body human and the existing research works that have proposed the reconstruction methods for telepresence. Besides the 3D reconstruction method, this paper also enlightens our recent finding on the MR framework to transport a full-body human from a local location to a remote location. The MR telepresence will be discussed, as well as the robust 3D reconstruction method which has been implemented with user telepresence feature where the user experiences an accurate 3D representation of a remote person. The paper ends with the discussion and results, MR telepresence with robust 3D reconstruction method to execute user telepresence.

Download Full-text

Real-Time 3D Reconstruction for Occlusion-Aware Interactions in Mixed Reality

Advances in Visual Computing - Lecture Notes in Computer Science ◽

10.1007/978-3-642-10331-5_45 ◽

2009 ◽

pp. 480-489 ◽

Cited By ~ 3

Author(s):

Alexander Ladikos ◽

Nassir Navab

Keyword(s):

3D Reconstruction ◽

Real Time ◽

Mixed Reality

Download Full-text

45. Education at distance: Broadcasting ECG rounds to Southeastern Ontario (BESO Project). An innovative approach for teaching electrocardiography

Clinical & Investigative Medicine ◽

10.25011/cim.v30i4.2805 ◽

2007 ◽

Vol 30 (4) ◽

pp. 51 ◽

Cited By ~ 2

Author(s):

A. Baranchuk ◽

G. Dagnone ◽

P. Fowler ◽

M. N. Harrison ◽

L. Lisnevskaia ◽

...

Keyword(s):

Health Care ◽

Real Time ◽

Digital Signal ◽

Teaching Methodology ◽

Care Providers ◽

Long Distance ◽

Medicine Clerkship ◽

Time Problem ◽

The Impact ◽

Ecg Interpretation

Electrocardiography (ECG) interpretation is an essential skill for physicians as well as for many other health care professionals. Continuing education is necessary to maintain these skills. The process of teaching and learning ECG interpretation is complex and involves both deductive mechanisms and recognition of patterns for different clinical situations (“pattern recognition”). The successful methodologies of interactive sessions and real time problem based learning have never been evaluated with a long distance education model. To evaluate the efficacy of broadcasting ECG rounds to different hospitals in the Southeastern Ontario region; to perform qualitative research to determine the impact of this methodology in developing and maintaining skills in ECG interpretation. ECG rounds are held weekly at Kingston General Hospital and will be transmitted live to Napanee, Belleville, Oshawa, Peterborough and Brockville. The teaching methodology is based on real ECG cases. The audience is invited to analyze the ECG case and the coordinator will introduce comments to guide the case through the proper algorithm. Final interpretation will be achieved emphasizing the deductive process and the relevance of each case. An evaluation will be filled out by each participant at the end of each session. Videoconferencing works through a vast array of internet LANs, WANs, ISDN phone lines, routers, switches, firewalls and Codecs (Coder/Decoder) and bridges. A videoconference Codec takes the analog audio and video signal codes and compresses it into a digital signal and transmits that digital signal to another Codec where the signal is decompressed and retranslated back into analog video and audio. This compression and decompression allows large amounts of data to be transferred across a network at close to real time (384 kbps with 30 frames of video per second). Videoconferencing communication works on voice activation so whichever site is speaking has the floor and is seen by all the participating sites. A continuous presence mode allows each site to have the same visual and audio involvement as the host site. A bridged multipoint can connect between 8 and 12 sites simultaneously. This innovative methodology for teaching ECG will facilitate access to developing and maintaining skills in ECG interpretation for a large number of health care providers. Bertsch TF, Callas PW, Rubin A. Effectiveness of lectures attended via interactive video conferencing versus in-person in preparing third-year internal medicine clerkship students for clinical practice examinations. Teach Learn Med 2007; 19(1):4-8. Yellowlees PM, Hogarth M, Hilty DM. The importance of distributed broadband networks to academic biomedical research and education programs. Acad Psychaitry 2006;30:451-455

Download Full-text

Development of environment design support mixed reality system capable of environment estimation using deep learning

Impact ◽

10.21820/23987073.2020.2.9 ◽

2020 ◽

Vol 2020 (2) ◽

pp. 9-11

Author(s):

Tomohiro Fukuda

Keyword(s):

Deep Learning ◽

Real Time ◽

Computer Games ◽

Construction Projects ◽

Mixed Reality ◽

Semantic Segmentation ◽

Environment Design ◽

Aviation Training ◽

Architecture And Design ◽

World Environment

Mixed reality (MR) is rapidly becoming a vital tool, not just in gaming, but also in education, medicine, construction and environmental management. The term refers to systems in which computer-generated content is superimposed over objects in a real-world environment across one or more sensory modalities. Although most of us have heard of the use of MR in computer games, it also has applications in military and aviation training, as well as tourism, healthcare and more. In addition, it has the potential for use in architecture and design, where buildings can be superimposed in existing locations to render 3D generations of plans. However, one major challenge that remains in MR development is the issue of real-time occlusion. This refers to hiding 3D virtual objects behind real articles. Dr Tomohiro Fukuda, who is based at the Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering at Osaka University in Japan, is an expert in this field. Researchers, led by Dr Tomohiro Fukuda, are tackling the issue of occlusion in MR. They are currently developing a MR system that realises real-time occlusion by harnessing deep learning to achieve an outdoor landscape design simulation using a semantic segmentation technique. This methodology can be used to automatically estimate the visual environment prior to and after construction projects.

Download Full-text

Near-Perfect Automation: Investigating Performance, Trust, and Visual Attention Allocation

Human Factors The Journal of the Human Factors and Ergonomics Society ◽

10.1177/00187208211032889 ◽

2021 ◽

pp. 001872082110328

Author(s):

Cyrus K. Foroughi ◽

Shannon Devlin ◽

Richard Pak ◽

Noelle L. Brown ◽

Ciara Sibley ◽

...

Keyword(s):

Visual Attention ◽

False Alarm ◽

Real Time ◽

Subjective Experience ◽

Automated System ◽

Automated Systems ◽

High Stakes ◽

Individual Level ◽

Scan Pattern ◽

Subjective Trust

Objective Assess performance, trust, and visual attention during the monitoring of a near-perfect automated system. Background Research rarely attempts to assess performance, trust, and visual attention in near-perfect automated systems even though they will be relied on in high-stakes environments. Methods Seventy-three participants completed a 40-min supervisory control task where they monitored three search feeds. All search feeds were 100% reliable with the exception of two automation failures: one miss and one false alarm. Eye-tracking and subjective trust data were collected. Results Thirty-four percent of participants correctly identified the automation miss, and 67% correctly identified the automation false alarm. Subjective trust increased when participants did not detect the automation failures and decreased when they did. Participants who detected the false alarm had a more complex scan pattern in the 2 min centered around the automation failure compared with those who did not. Additionally, those who detected the failures had longer dwell times in and transitioned to the center sensor feed significantly more often. Conclusion Not only does this work highlight the limitations of the human when monitoring near-perfect automated systems, it begins to quantify the subjective experience and attentional cost of the human. It further emphasizes the need to (1) reevaluate the role of the operator in future high-stakes environments and (2) understand the human on an individual level and actively design for the given individual when working with near-perfect automated systems. Application Multiple operator-level measures should be collected in real-time in order to monitor an operator’s state and leverage real-time, individualized assistance.

Download Full-text