scholarly journals Holographic patient tracking after bed movement for augmented reality neuronavigation using a head-mounted display

2021 ◽  
Author(s):  
T. Fick ◽  
J.A.M. van Doormaal ◽  
E.W. Hoving ◽  
L. Regli ◽  
T.P.C. van Doormaal
Keyword(s):  
Augmented Reality ◽  
Anatomical Landmarks ◽  
Registration Accuracy ◽  
Patient Tracking ◽  
First Report ◽  
Head Mounted Display ◽  
Reference Array ◽  
Custom Made ◽  
Before And After ◽  
Neuronavigation System

Abstract Background Holographic neuronavigation has several potential advantages compared to conventional neuronavigation systems. We present the first report of a holographic neuronavigation system with patient-to-image registration and patient tracking with a reference array using an augmented reality head-mounted display (AR-HMD). Methods Three patients undergoing an intracranial neurosurgical procedure were included in this pilot study. The relevant anatomy was first segmented in 3D and then uploaded as holographic scene in our custom neuronavigation software. Registration was performed using point-based matching using anatomical landmarks. We measured the fiducial registration error (FRE) as the outcome measure for registration accuracy. A custom-made reference array with QR codes was integrated in the neurosurgical setup and used for patient tracking after bed movement. Results Six registrations were performed with a mean FRE of 8.5 mm. Patient tracking was achieved with no visual difference between the registration before and after movement. Conclusions This first report shows a proof of principle of intraoperative patient tracking using a standalone holographic neuronavigation system. The navigation accuracy should be further optimized to be clinically applicable. However, it is likely that this technology will be incorporated in future neurosurgical workflows because the system improves spatial anatomical understanding for the surgeon.

scholarly journals Laparoscopic augmented reality registration for oncological resection site repair

2021 ◽  
Author(s):  
Fabian Joeres ◽  
Tonia Mielke ◽  
Christian Hansen
Keyword(s):  
Augmented Reality ◽  
Time Pressure ◽  
Tumour Resection ◽  
Anatomical Landmarks ◽  
Proof Of Concept ◽  
Registration Accuracy ◽  
Target Registration Error ◽  
Concept Evaluation ◽  
Navigation Support ◽  
Challenges And Opportunities

Abstract Purpose Resection site repair during laparoscopic oncological surgery (e.g. laparoscopic partial nephrectomy) poses some unique challenges and opportunities for augmented reality (AR) navigation support. This work introduces an AR registration workflow that addresses the time pressure that is present during resection site repair. Methods We propose a two-step registration process: the AR content is registered as accurately as possible prior to the tumour resection (the primary registration). This accurate registration is used to apply artificial fiducials to the physical organ and the virtual model. After the resection, these fiducials can be used for rapid re-registration (the secondary registration). We tested this pipeline in a simulated-use study with $$N=18$$ N = 18 participants. We compared the registration accuracy and speed for our method and for landmark-based registration as a reference. Results Acquisition of and, thereby, registration with the artificial fiducials were significantly faster than the initial use of anatomical landmarks. Our method also had a trend to be more accurate in cases in which the primary registration was successful. The accuracy loss between the elaborate primary registration and the rapid secondary registration could be quantified with a mean target registration error increase of 2.35 mm. Conclusion This work introduces a registration pipeline for AR navigation support during laparoscopic resection site repair and provides a successful proof-of-concept evaluation thereof. Our results indicate that the concept is better suited than landmark-based registration during this phase, but further work is required to demonstrate clinical suitability and applicability.

scholarly journals Ambiguity-Free Optical–Inertial Tracking for Augmented Reality Headsets

Sensors ◽  
2020 ◽  
Vol 20 (5) ◽  
pp. 1444 ◽  
Author(s):  
Fabrizio Cutolo ◽  
Virginia Mamone ◽  
Nicola Carbonaro ◽  
Vincenzo Ferrari ◽  
Alessandro Tognetti
Keyword(s):  
Augmented Reality ◽  
Optical Tracking ◽  
Tracking Data ◽  
Tracking Accuracy ◽  
Short Baseline ◽  
Head Mounted Display ◽  
Lighting Conditions ◽  
Mobile Graphics ◽  
Custom Made ◽  
Manual Tasks

The increasing capability of computing power and mobile graphics has made possible the release of self-contained augmented reality (AR) headsets featuring efficient head-anchored tracking solutions. Ego motion estimation based on well-established infrared tracking of markers ensures sufficient accuracy and robustness. Unfortunately, wearable visible-light stereo cameras with short baseline and operating under uncontrolled lighting conditions suffer from tracking failures and ambiguities in pose estimation. To improve the accuracy of optical self-tracking and its resiliency to marker occlusions, degraded camera calibrations, and inconsistent lighting, in this work we propose a sensor fusion approach based on Kalman filtering that integrates optical tracking data with inertial tracking data when computing motion correlation. In order to measure improvements in AR overlay accuracy, experiments are performed with a custom-made AR headset designed for supporting complex manual tasks performed under direct vision. Experimental results show that the proposed solution improves the head-mounted display (HMD) tracking accuracy by one third and improves the robustness by also capturing the orientation of the target scene when some of the markers are occluded and when the optical tracking yields unstable and/or ambiguous results due to the limitations of using head-anchored stereo tracking cameras under uncontrollable lighting conditions.

scholarly journals Augmented Reality-Assisted Craniotomy for Parasagittal and Convexity En Plaque Meningiomas and Custom-Made Cranio-Plasty: A Preliminary Laboratory Report

2021 ◽  
Vol 18 (19) ◽  
pp. 9955
Author(s):  
Nicola Montemurro ◽  
Sara Condino ◽  
Nadia Cattari ◽  
Renzo D’Amato ◽  
Vincenzo Ferrari ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Surgical Simulation ◽  
Cranial Vault ◽  
Bone Flap ◽  
Patient Specific ◽  
Real Patient ◽  
Head Mounted Display ◽  
Laboratory Report ◽  
Custom Made ◽  
Wearable Augmented Reality

Background: This report discusses the utility of a wearable augmented reality platform in neurosurgery for parasagittal and convexity en plaque meningiomas with bone flap removal and custom-made cranioplasty. Methods: A real patient with en plaque cranial vault meningioma with diffuse and extensive dural involvement, extracranial extension into the calvarium, and homogeneous contrast enhancement on gadolinium-enhanced T1-weighted MRI, was selected for this case study. A patient-specific manikin was designed starting with the segmentation of the patient’s preoperative MRI images to simulate a craniotomy procedure. Surgical planning was performed according to the segmented anatomy, and customized bone flaps were designed accordingly. During the surgical simulation stage, the VOSTARS head-mounted display was used to accurately display the planned craniotomy trajectory over the manikin skull. The precision of the craniotomy was assessed based on the evaluation of previously prepared custom-made bone flaps. Results: A bone flap with a radius 0.5 mm smaller than the radius of an ideal craniotomy fitted perfectly over the performed craniotomy, demonstrating an error of less than ±1 mm in the task execution. The results of this laboratory-based experiment suggest that the proposed augmented reality platform helps in simulating convexity en plaque meningioma resection and custom-made cranioplasty, as carefully planned in the preoperative phase. Conclusions: Augmented reality head-mounted displays have the potential to be a useful adjunct in tumor surgical resection, cranial vault lesion craniotomy and also skull base surgery, but more study with large series is needed.

scholarly journals Gamification and Hazard Communication in Virtual Reality: A Qualitative Study

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4663
Author(s):  
Janaina Cavalcanti ◽  
Victor Valls ◽  
Manuel Contero ◽  
David Fonseca
Keyword(s):  
Virtual Reality ◽  
New Technologies ◽  
Training Environment ◽  
Hazard Communication ◽  
Head Mounted Display ◽  
Compliance Behavior ◽  
Before And After ◽  
Recorded Data ◽  
The Impact ◽  
Different Levels

An effective warning attracts attention, elicits knowledge, and enables compliance behavior. Game mechanics, which are directly linked to human desires, stand out as training, evaluation, and improvement tools. Immersive virtual reality (VR) facilitates training without risk to participants, evaluates the impact of an incorrect action/decision, and creates a smart training environment. The present study analyzes the user experience in a gamified virtual environment of risks using the HTC Vive head-mounted display. The game was developed in the Unreal game engine and consisted of a walk-through maze composed of evident dangers and different signaling variables while user action data were recorded. To demonstrate which aspects provide better interaction, experience, perception and memory, three different warning configurations (dynamic, static and smart) and two different levels of danger (low and high) were presented. To properly assess the impact of the experience, we conducted a survey about personality and knowledge before and after using the game. We proceeded with the qualitative approach by using questions in a bipolar laddering assessment that was compared with the recorded data during the game. The findings indicate that when users are engaged in VR, they tend to test the consequences of their actions rather than maintaining safety. The results also reveal that textual signal variables are not accessed when users are faced with the stress factor of time. Progress is needed in implementing new technologies for warnings and advance notifications to improve the evaluation of human behavior in virtual environments of high-risk surroundings.

Augmented Reality Procedure Assistance System for Operator Training and Simulation

2020 ◽  
Vol 64 (1) ◽  
pp. 1176-1180
Author(s):  
Eugene Hayden ◽  
Kang Wang ◽  
Chengjie Wu ◽  
Shi Cao
Keyword(s):  
Augmented Reality ◽  
Situation Awareness ◽  
Mental Workload ◽  
Field Of View ◽  
Task Completion ◽  
Operator Training ◽  
Head Mounted Display ◽  
Subject Design ◽  
Procedural Tasks ◽  
Speed Accuracy

This study explores the design, implementation, and evaluation of an Augmented Reality (AR) prototype that assists novice operators in performing procedural tasks in simulator environments. The prototype uses an optical see-through head-mounted display (OST HMD) in conjunction with a simulator display to supplement sequences of interactive visual and attention-guiding cues to the operator’s field of view. We used a 2x2 within-subject design to test two conditions: with/without AR-cues, each condition had a voice assistant and two procedural tasks (preflight and landing). An experiment examined twenty-six novice operators. The results demonstrated that augmented reality had benefits in terms of improved situation awareness and accuracy, however, it yielded longer task completion time by creating a speed-accuracy trade-off effect in favour of accuracy. No significant effect on mental workload is found. The results suggest that augmented reality systems have the potential to be used by a wider audience of operators.

scholarly journals Augmented Reality, Mixed Reality, and Hybrid Approach in Healthcare Simulation: A Systematic Review

2021 ◽  
Vol 11 (5) ◽  
pp. 2338
Author(s):  
Rosanna Maria Viglialoro ◽  
Sara Condino ◽  
Giuseppe Turini ◽  
Marina Carbone ◽  
Vincenzo Ferrari ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Mixed Reality ◽  
Ethical Issues ◽  
Medical Training ◽  
Hybrid Approach ◽  
Dental Surgery ◽  
Feasibility Assessment ◽  
Hybrid Approaches ◽  
Custom Made ◽  
Improved Performance

Simulation-based medical training is considered an effective tool to acquire/refine technical skills, mitigating the ethical issues of Halsted’s model. This review aims at evaluating the literature on medical simulation techniques based on augmented reality (AR), mixed reality (MR), and hybrid approaches. The research identified 23 articles that meet the inclusion criteria: 43% combine two approaches (MR and hybrid), 22% combine all three, 26% employ only the hybrid approach, and 9% apply only the MR approach. Among the studies reviewed, 22% use commercial simulators, whereas 78% describe custom-made simulators. Each simulator is classified according to its target clinical application: training of surgical tasks (e.g., specific tasks for training in neurosurgery, abdominal surgery, orthopedic surgery, dental surgery, otorhinolaryngological surgery, or also generic tasks such as palpation) and education in medicine (e.g., anatomy learning). Additionally, the review assesses the complexity, reusability, and realism of the physical replicas, as well as the portability of the simulators. Finally, we describe whether and how the simulators have been validated. The review highlights that most of the studies do not have a significant sample size and that they include only a feasibility assessment and preliminary validation; thus, further research is needed to validate existing simulators and to verify whether improvements in performance on a simulated scenario translate into improved performance on real patients.

scholarly journals Mediastinal anatomical landmarks, their variants and tips for video-assisted thoracoscopic navigation during oesophageal extirpation

2021 ◽  
Author(s):  
Sergey Dydykin ◽  
Friedrich Paulsen ◽  
Tatyana Khorobykh ◽  
Natalya Mishchenko ◽  
Marina Kapitonova ◽  
...  
Keyword(s):  
Prone Position ◽  
Anatomical Study ◽  
Structural Features ◽  
Posterior Mediastinum ◽  
Visual Navigation ◽  
Anatomical Landmarks ◽  
Video Assisted ◽  
Before And After ◽  
Human Cadavers ◽  
The Right

Abstract Purpose There is no systematic description of primary anatomical landmarks that allow a surgeon to reliably and safely navigate the superior and posterior mediastinum’s fat tissue spaces near large vessels and nerves during video-assisted endothoracoscopic interventions in the prone position of a patient. Our aim was to develop an algorithm of sequential visual navigation during thoracoscopic extirpation of the esophagus and determine the most permanent topographic and anatomical landmarks allowing safe thoracoscopic dissection of the esophagus in the prone position. Methods The anatomical study of the mediastinal structural features was carried out on 30 human cadavers before and after opening the right pleural cavity. Results For thoracoscopic extirpation of the esophagus in the prone position, anatomical landmarks are defined, their variants are assessed, and an algorithm for their selection is developed, allowing their direct visualization before and after opening the mediastinal pleura. Conclusion The proposed algorithm for topographic and anatomical navigation based on the key anatomical landmarks in the posterior mediastinum provides safe performance of the video-assisted thoracoscopic extirpation of the esophagus in the prone position.

scholarly journals Fooling the size–weight illusion—Using augmented reality to eliminate the effect of size on perceptions of heaviness and sensorimotor prediction

2021 ◽  
Author(s):  
Nina Rohrbach ◽  
Joachim Hermsdörfer ◽  
Lisa-Marie Huber ◽  
Annika Thierfelder ◽  
Gavin Buckingham
Keyword(s):  
Augmented Reality ◽  
Visual Cues ◽  
Equal Mass ◽  
Small Object ◽  
Verbal Reports ◽  
Fully Integrated ◽  
Head Mounted Display ◽  
Lift Off ◽  
Cognitive Knowledge ◽  
Weight Illusion

AbstractAugmented reality, whereby computer-generated images are overlaid onto the physical environment, is becoming significant part of the world of education and training. Little is known, however, about how these external images are treated by the sensorimotor system of the user – are they fully integrated into the external environmental cues, or largely ignored by low-level perceptual and motor processes? Here, we examined this question in the context of the size–weight illusion (SWI). Thirty-two participants repeatedly lifted and reported the heaviness of two cubes of unequal volume but equal mass in alternation. Half of the participants saw semi-transparent equally sized holographic cubes superimposed onto the physical cubes through a head-mounted display. Fingertip force rates were measured prior to lift-off to determine how the holograms influenced sensorimotor prediction, while verbal reports of heaviness after each lift indicated how the holographic size cues influenced the SWI. As expected, participants who lifted without augmented visual cues lifted the large object at a higher rate of force than the small object on early lifts and experienced a robust SWI across all trials. In contrast, participants who lifted the (apparently equal-sized) augmented cubes used similar force rates for each object. Furthermore, they experienced no SWI during the first lifts of the objects, with a SWI developing over repeated trials. These results indicate that holographic cues initially dominate physical cues and cognitive knowledge, but are dismissed when conflicting with cues from other senses.

Visual Coherence in Mixed Reality: A Systematic Enquiry

2017 ◽  
Vol 26 (1) ◽  
pp. 16-41 ◽  
Author(s):  
Jonny Collins ◽  
Holger Regenbrecht ◽  
Tobias Langlotz
Keyword(s):  
Augmented Reality ◽  
Mixed Reality ◽  
Focus Of Attention ◽  
User Experiences ◽  
Head Mounted Display ◽  
Virtual And Augmented Reality

Virtual and augmented reality, and other forms of mixed reality (MR), have become a focus of attention for companies and researchers. Before they can become successful in the market and in society, those MR systems must be able to deliver a convincing, novel experience for the users. By definition, the experience of mixed reality relies on the perceptually successful blending of reality and virtuality. Any MR system has to provide a sensory, in particular visually coherent, set of stimuli. Therefore, issues with visual coherence, that is, a discontinued experience of a MR environment, must be avoided. While it is very easy for a user to detect issues with visual coherence, it is very difficult to design and implement a system for coherence. This article presents a framework and exemplary implementation of a systematic enquiry into issues with visual coherence and possible solutions to address those issues. The focus is set on head-mounted display-based systems, notwithstanding its applicability to other types of MR systems. Our framework, together with a systematic discussion of tangible issues and solutions for visual coherence, aims at guiding developers of mixed reality systems for better and more effective user experiences.

Sign in / Sign up

Export Citation Format

Share Document