scholarly journals O49 The use of augmented reality for surgical guidance: an assessment of the human error

2021 ◽  
Vol 108 (Supplement_5) ◽  
Author(s):  
P R Sharma ◽  
H Brech ◽  
L Pérez-Pachón ◽  
J Gregory ◽  
T Lowe ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Human Error ◽  
Human Performance ◽  
Patient Specific ◽  
Surgical Guidance ◽  
Virtual Models ◽  
Significant Difference ◽  
Entry Points ◽  
Microsoft Hololens ◽  
Needle Entry

Abstract Introduction Microsoft HoloLens® is an augmented-reality headset which is increasingly used for surgical guidance. This headset allows the overlay of patient-specific virtual models obtained from medical images onto the patient’s body surface using automatic marker-based alignment. This can guide surgeons during certain surgical tasks, e.g. determining biopsy needle entry points. This study aims to measure the human error in the localisation of virtual models with the headset and discuss its surgical implications. Method 59 adults were recruited between the ages of 20–59 years. A 12 cm2 digital marker was displayed on a monitor in 9 different positions, one at a time. This was repeated 3 times, resulting in 27 markers shown to each participant. Once a marker was detected by the headset’s camera, a virtual hexagon was rendered on the headset’s transparent lenses. Participants were tasked to click on the hexagon’s vertices using a mouse. The clicks’ coordinates were recorded by the system and compared to the predicted coordinates. This allowed for the calculation of the vertex localisation error. Result The mean vertex localisation error was found to be 5.19 mm (±3.56) with a range from 0.08 to 29.77 mm. There was a significant difference between marker positions as determined by a one-way ANOVA (P < 0.001). Conclusion This study suggests that the error in the localisation of virtual models depends on the position of the markers relative to the user wearing the headset. Further research is required to explore whether training can reduce the human error with this headset. Take-home Message The range of the human error in localising virtual models via the Microsoft HoloLens® headset is large and may be dependent on the position of the marker relative to the user of the headset. Further research is needed to investigate whether training with the headset can improve human performance.

scholarly journals A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Nicole Wake ◽  
Andrew B. Rosenkrantz ◽  
William C. Huang ◽  
James S. Wysock ◽  
Samir S. Taneja ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Patient Care ◽  
Three Dimensional ◽  
3D Models ◽  
Patient Specific ◽  
Imaging Data ◽  
Urologic Oncology ◽  
Case Examples ◽  
Microsoft Hololens ◽  
Enhance Patient Care

AbstractAugmented reality (AR) and virtual reality (VR) are burgeoning technologies that have the potential to greatly enhance patient care. Visualizing patient-specific three-dimensional (3D) imaging data in these enhanced virtual environments may improve surgeons’ understanding of anatomy and surgical pathology, thereby allowing for improved surgical planning, superior intra-operative guidance, and ultimately improved patient care. It is important that radiologists are familiar with these technologies, especially since the number of institutions utilizing VR and AR is increasing. This article gives an overview of AR and VR and describes the workflow required to create anatomical 3D models for use in AR using the Microsoft HoloLens device. Case examples in urologic oncology (prostate cancer and renal cancer) are provided which depict how AR has been used to guide surgery at our institution.

scholarly journals Computer-assisted planning for the insertion of stereoelectroencephalography electrodes for the investigation of drug-resistant focal epilepsy: an external validation study

2019 ◽  
Vol 130 (2) ◽  
pp. 601-610 ◽  
Author(s):  
Vejay N. Vakharia ◽  
Rachel Sparks ◽  
Roman Rodionov ◽  
Sjoerd B. Vos ◽  
Christian Dorfer ◽  
...  
Keyword(s):  
Gray Matter ◽  
Focal Epilepsy ◽  
Electrode Placement ◽  
Patient Specific ◽  
Computer Assisted ◽  
Epileptogenic Zone ◽  
Vascular Segmentation ◽  
Electrode Length ◽  
Significant Difference ◽  
Entry Points

OBJECTIVEOne-third of cases of focal epilepsy are drug refractory, and surgery might provide a cure. Seizure-free outcome after surgery depends on the correct identification and resection of the epileptogenic zone. In patients with no visible abnormality on MRI, or in cases in which presurgical evaluation yields discordant data, invasive stereoelectroencephalography (SEEG) recordings might be necessary. SEEG is a procedure in which multiple electrodes are placed stereotactically in key targets within the brain to record interictal and ictal electrophysiological activity. Correlating this activity with seizure semiology enables identification of the seizure-onset zone and key structures within the ictal network. The main risk related to electrode placement is hemorrhage, which occurs in 1% of patients who undergo the procedure. Planning safe electrode placement for SEEG requires meticulous adherence to the following: 1) maximize the distance from cerebral vasculature, 2) avoid crossing sulcal pial boundaries (sulci), 3) maximize gray matter sampling, 4) minimize electrode length, 5) drill at an angle orthogonal to the skull, and 6) avoid critical neurological structures. The authors provide a validation of surgical strategizing and planning with EpiNav, a multimodal platform that enables automated computer-assisted planning (CAP) for electrode placement with user-defined regions of interest.METHODSThirteen consecutive patients who underwent implantation of a total 116 electrodes over a 15-month period were studied retrospectively. Models of the cortex, gray matter, and sulci were generated from patient-specific whole-brain parcellation, and vascular segmentation was performed on the basis of preoperative MR venography. Then, the multidisciplinary implantation strategy and precise trajectory planning were reconstructed using CAP and compared with the implemented manually determined plans. Paired results for safety metric comparisons were available for 104 electrodes. External validity of the suitability and safety of electrode entry points, trajectories, and target-point feasibility was sought from 5 independent, blinded experts from outside institutions.RESULTSCAP-generated electrode trajectories resulted in a statistically significant improvement in electrode length, drilling angle, gray matter–sampling ratio, minimum distance from segmented vasculature, and risk (p < 0.05). The blinded external raters had various opinions of trajectory feasibility that were not statistically significant, and they considered a mean of 69.4% of manually determined trajectories and 62.2% of CAP-generated trajectories feasible; 19.4% of the CAP-generated electrode-placement plans were deemed feasible when the manually determined plans were not, whereas 26.5% of the manually determined electrode-placement plans were rated feasible when CAP-determined plans were not (no significant difference).CONCLUSIONSCAP generates clinically feasible electrode-placement plans and results in statistically improved safety metrics. CAP is a useful tool for automating the placement of electrodes for SEEG; however, it requires the operating surgeon to review the results before implantation, because only 62% of electrode-placement plans were rated feasible, compared with 69% of the manually determined placement plans, mainly because of proximity of the electrodes to unsegmented vasculature. Improved vascular segmentation and sulcal modeling could lead to further improvements in the feasibility of CAP-generated trajectories.

scholarly journals Assessing Accuracy and Precision of 3D Augmented Reality Holographic models derived from DICOM data

2020 ◽  
Author(s):  
Jesus Uribe ◽  
Lan Vu ◽  
Benjamin Laguna ◽  
Jesse Courtier
Keyword(s):  
Augmented Reality ◽  
Gold Standard ◽  
Ar Model ◽  
Fiducial Markers ◽  
Accuracy And Precision ◽  
Significant Difference ◽  
Precision Assessment ◽  
Microsoft Hololens ◽  
High Degree ◽  
Ar Models

Abstract Objective: Assess accuracy and precision of measurements on 3D Augmented reality (AR) models derived from CT DICOM data, and compare AR model measurements with PACS measurements. Materials/Methods: 5 individual 3D hologram models were produced using a CT phantom with fiducial markers set at varying distances. DICOM files were translated into 3D AR models using open source software. AR models were adapted for display on the Microsoft HoloLens using a novel application. AR models were projected and distances between the projected fiducial markers were measured. Finally, 5 measurements each were obtained of the holographic projected distances between fiducials in the x1, y1, and z1 labeled planes respectively for precision assessment. Mann-Whitney U test was performed to compare measured distances on AGFA-PACS, AR models, and actual measured distances on phantom models. Results: No significant difference was found between gold standard measurements and either PACS measurements (p=0.9124) or AR measurements (p=0.8966). AR model measurements had a standard error of 0.24mm, 0.24mm, and 0.38mm in the x,y, and z planes respectively. Furthermore, measurements on AR models demonstrated a high degree of accuracy in comparison to gold standard measurements. Conclusion: Current AR technology is can produce reliable 3D AR models from CT DICOM data

scholarly journals Utilizing Steering Entropy as a Measure of Performance in Head-mounted Augmented Reality

2019 ◽  
Vol 63 (1) ◽  
pp. 63-67
Author(s):  
Kelly A. Sprehn ◽  
Tristan C. Endsley ◽  
Kimberly J. Ryan
Keyword(s):  
Augmented Reality ◽  
Human Performance ◽  
Measurement Theory ◽  
Display Type ◽  
Movement Data ◽  
Entropy Measurement ◽  
And Performance ◽  
Microsoft Hololens ◽  
Measure Of Performance ◽  
New Applications

The next phase of augmented reality (AR) technologies suggest that as both the hardware and software continue to improve, we can expect that AR will become more commonly used as a tool for a variety of applications in complex operational contexts (i.e. training, manufacturing, mission planning). As new applications are designed and developed within these contexts, there is a necessity to be able to measure the effectiveness of these systems and to understand their impact on human performance and workload, so that only the most appropriate designs are selected for use, growing the technology in usefulness, not novel hindrances. A unique opportunity presented by the Microsoft HoloLens platform, as an example of head-worn AR systems, is the ability to collect positional and movement data, which lends itself to the computation of behavioral (or steering) entropy data, which can be related to human workload and performance within the system environment. However, little reference exists to be able to verify the accuracy of tracking of the device with regards to the output data available for collection. Within this practitioner-oriented paper, we extend current entropy measurement theory typically used in control settings within a heads-up display type ‘controls' environment. Our findings indicate that in-situ measurements of entropy utilizing the onboard sensors within the AR platform are more accurate to those collected within a Motion Capture Facility. Extending this work, these measurements can be used as a correlate of performance.

scholarly journals Patient-Specific Instrumentation Accuracy Evaluated with 3D Virtual Models

2021 ◽  
Vol 10 (7) ◽  
pp. 1439
Author(s):  
Vicente J. León-Muñoz ◽  
Andrea Parrinello ◽  
Silvio Manca ◽  
Gianluca Galloni ◽  
Mirian López-López ◽  
...  
Keyword(s):  
Patient Reported Outcomes ◽  
Three Dimensional ◽  
Dimensional Accuracy ◽  
Tibial Slope ◽  
Patient Specific ◽  
Virtual Models ◽  
Relevant Effect ◽  
Patient Specific Instrumentation ◽  
Significant Difference ◽  
Patient Reported

There have been remarkable advances in knee replacement surgery over the last few decades. One of the concerns continues to be the accuracy in achieving the desired alignment. Patient-specific instrumentation (PSI) was developed to increase component placement accuracy, but the available evidence is not conclusive. Our study aimed to determine a PSI system’s three-dimensional accuracy on 3D virtual models obtained by post-operative computed tomography. We compared the angular placement values of 35 total knee arthroplasties (TKAs) operated within a year obtained with the planned ones, and we analyzed the possible relationships between alignment and patient-reported outcomes. The mean (SD) discrepancies measured by two experienced engineers to the planned values observed were 1.64° (1.3°) for the hip–knee–ankle angle, 1.45° (1.06°) for the supplementary angle of the femoral lateral distal angle, 1.44° (0.97°) for the proximal medial tibial angle, 2.28° (1.78°) for tibial slope, 0.64° (1.09°) for femoral sagittal flexion, and 1.42° (1.06°) for femoral rotation. Neither variables related to post-operative alignment nor the proportion of change between pre-and post-operative alignment influenced the patient-reported outcomes. The evaluated PSI system’s three-dimensional alignment analysis showed a statistically significant difference between the angular values planned and those obtained. However, we did not find a relevant effect size, and this slight discrepancy did not impact the clinical outcome.

Demonstration: Virtual Patient Immersive Trainer to Train Perceptual Skills Using Augmented Reality

2020 ◽  
Vol 64 (1) ◽  
pp. 470-472
Author(s):  
Christen E. Sushereba ◽  
Laura G. Militello
Keyword(s):  
Medical Students ◽  
Augmented Reality ◽  
Student Performance ◽  
Group Interaction ◽  
Virtual Patient ◽  
Perceptual Cues ◽  
Perceptual Skills ◽  
Individual Interaction ◽  
Severity Of Injury ◽  
Microsoft Hololens

In this session, we will demonstrate the Virtual Patient Immersive Trainer (VPIT). The VPIT system uses augmented reality (AR) to allow medics and medical students to experience a photorealistic, life-sized virtual patient. The VPIT supports learners in obtaining the perceptual skills required to recognize and interpret subtle perceptual cues critical to assessing a patient’s condition. We will conduct an interactive demonstration of the virtual patient using both a tablet (for group interaction) and an AR-enabled headset (Microsoft HoloLens) for individual interaction. In addition, we will demonstrate use of the instructor tablet to control what the learner sees (e.g., injury types, severity of injury) and to monitor student performance.

An evaluation of a shoulder rehabilitation class in a UK hospital following evidence-based modifications

2021 ◽  
pp. 1-8
Author(s):  
Emily Kell ◽  
John A. Hammond ◽  
Sophie Andrews ◽  
Christina Germeni ◽  
Helen Hingston ◽  
...  
Keyword(s):  
Patient Satisfaction ◽  
Shoulder Pain ◽  
Management Strategies ◽  
Cost Effective ◽  
Rehabilitation Programme ◽  
Patient Specific ◽  
Satisfaction Survey ◽  
Evidence Based ◽  
Patient Satisfaction Survey ◽  
Significant Difference

OBJECTIVES: Shoulder pain is a common musculoskeletal disorder, which carries a high cost to healthcare systems. Exercise is a common conservative management strategy for a range of shoulder conditions and can reduce shoulder pain and improve function. Exercise classes that integrate education and self-management strategies have been shown to be cost-effective, offer psycho-social benefits and promote self-efficacy. This study aimed to examine the effectiveness of an 8-week educational and exercise-based shoulder rehabilitation programme following the introduction of evidence-based modifications. METHODS: A retrospective evaluation of a shoulder rehabilitation programme at X Trust was conducted, comparing existing anonymised Shoulder Pain and Disability Index (SPADI) and Patient-Specific Functional Scale (PSFS) scores from two cohorts of class participants from 2017-18 and 2018-19 that were previously collected by the physiotherapy team. Data from the two cohorts were analysed separately, and in comparison, to assess class efficacy. Descriptive data were also analysed from a patient satisfaction survey from the 2018-19 cohort. RESULTS: A total of 47 patients completed the 8-week shoulder rehabilitation programme during the period of data collection (2018-2019). The 2018-19 cohort showed significant improvements in SPADI (p 0.001) and PSFS scores (p 0.001). No significant difference was found between the improvements seen in the 2017-18 cohort and the 2018-19 cohort. 96% of the 31 respondents who completed the patient satisfaction survey felt the class helped to achieve their goals. CONCLUSION: A group-based shoulder rehabilitation class, which included loaded exercises and patient education, led to improvements in pain, disability and function for patients with rotator cuff related shoulder pain (RCRSP) in this outpatient setting, but anticipated additional benefits based on evidence were not observed.

Evaluation and Comparison of Artificial Intelligence Vision Aids: Orcam MyEye 1 and Seeing AI

2021 ◽  
pp. 0145482X2110274
Author(s):  
Christina Granquist ◽  
Susan Y. Sun ◽  
Sandra R. Montezuma ◽  
Tu M. Tran ◽  
Rachel Gage ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Human Performance ◽  
Visual Impairments ◽  
Light Level ◽  
Viewing Distance ◽  
Light Perception ◽  
Time To Completion ◽  
Significant Difference ◽  
Text Reading ◽  
Size Contrast

Introduction: We compared the print-to-speech properties and human performance characteristics of two artificial intelligence vision aids, Orcam MyEye 1 (a portable device) and Seeing AI (an iPhone and iPad application). Methods: There were seven participants with visual impairments who had no experience with the two reading aids. Four participants had no light perception. Two individuals with measurable acuity and one with light perception were tested while blindfolded. We also tested performance with text of varying appearance in varying viewing conditions. To evaluate human performance, we asked the participants to use the devices to attempt 12 reading tasks similar to activities of daily living. We assessed the ranges of text attributes for which reading was possible, such as print size, contrast, and light level. We also assessed if individuals could complete tasks with the devices and measured accuracy and completion time. Participants also completed a survey concerning the two aids. Results: Both aids achieved greater than 95% accuracy in text recognition for flat, plain word documents and ranged from 13 to 57% accuracy for formatted text on curved surfaces. Both aids could read print sizes as small as 0.8M (20/40 Snellen equivalent, 40 cm viewing distance). Individuals successfully completed 71% and 55% ( p = .114) of tasks while using Orcam MyEye 1 and Seeing AI, respectively. There was no significant difference in time to completion of tasks ( p = .775). Individuals believed both aids would be helpful for daily activities. Discussion: Orcam MyEye 1 and Seeing AI had similar text-reading capability and usability. Both aids were useful to users with severe visual impairments in performing reading tasks. Implications for Practitioners: Selection of a reading device or aid should be based on individual preferences and prior familiarity with the platform, since we found no clear superiority of one solution over the other.

scholarly journals ARETT: Augmented Reality Eye Tracking Toolkit for Head Mounted Displays

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2234
Author(s):  
Sebastian Kapp ◽  
Michael Barz ◽  
Sergey Mukhametov ◽  
Daniel Sonntag ◽  
Jochen Kuhn
Keyword(s):  
Augmented Reality ◽  
Eye Tracking ◽  
User Interaction ◽  
R Package ◽  
Eye Tracker ◽  
Unity 3D ◽  
Accuracy And Precision ◽  
Head Mounted Displays ◽  
Virtual And Augmented Reality ◽  
Microsoft Hololens

Currently an increasing number of head mounted displays (HMD) for virtual and augmented reality (VR/AR) are equipped with integrated eye trackers. Use cases of these integrated eye trackers include rendering optimization and gaze-based user interaction. In addition, visual attention in VR and AR is interesting for applied research based on eye tracking in cognitive or educational sciences for example. While some research toolkits for VR already exist, only a few target AR scenarios. In this work, we present an open-source eye tracking toolkit for reliable gaze data acquisition in AR based on Unity 3D and the Microsoft HoloLens 2, as well as an R package for seamless data analysis. Furthermore, we evaluate the spatial accuracy and precision of the integrated eye tracker for fixation targets with different distances and angles to the user (n=21). On average, we found that gaze estimates are reported with an angular accuracy of 0.83 degrees and a precision of 0.27 degrees while the user is resting, which is on par with state-of-the-art mobile eye trackers.

scholarly journals An MRI-Based Patient-Specific Computational Framework for the Calculation of Range of Motion of Total Hip Replacements

2021 ◽  
Vol 11 (6) ◽  
pp. 2852
Author(s):  
Maeruan Kebbach ◽  
Christian Schulze ◽  
Christian Meyenburg ◽  
Daniel Kluess ◽  
Mevluet Sungu ◽  
...  
Keyword(s):  
Range Of Motion ◽  
Internal Rotation ◽  
Computer Aided Design ◽  
External Rotation ◽  
Patient Specific ◽  
Computational Framework ◽  
Total Hip ◽  
Total Hip Replacements ◽  
Significant Difference ◽  
Hip Replacements

The calculation of range of motion (ROM) is a key factor during preoperative planning of total hip replacements (THR), to reduce the risk of impingement and dislocation of the artificial hip joint. To support the preoperative assessment of THR, a magnetic resonance imaging (MRI)-based computational framework was generated; this enabled the estimation of patient-specific ROM and type of impingement (bone-to-bone, implant-to-bone, and implant-to-implant) postoperatively, using a three-dimensional computer-aided design (CAD) to visualize typical clinical joint movements. Hence, patient-specific CAD models from 19 patients were generated from MRI scans and a conventional total hip system (Bicontact® hip stem and Plasmacup® SC acetabular cup with a ceramic-on-ceramic bearing) was implanted virtually. As a verification of the framework, the ROM was compared between preoperatively planned and the postoperatively reconstructed situations; this was derived based on postoperative radiographs (n = 6 patients) during different clinically relevant movements. The data analysis revealed there was no significant difference between preoperatively planned and postoperatively reconstructed ROM (∆ROM) of maximum flexion (∆ROM = 0°, p = 0.854) and internal rotation (∆ROM = 1.8°, p = 0.917). Contrarily, minor differences were observed for the ROM during maximum external rotation (∆ROM = 9°, p = 0.046). Impingement, of all three types, was in good agreement with the preoperatively planned and postoperatively reconstructed scenarios during all movements. The calculated ROM reached physiological levels during flexion and internal rotation movement; however, it exceeded physiological levels during external rotation. Patients, where implant-to-implant impingement was detected, reached higher ROMs than patients with bone-to-bone impingement. The proposed framework provides the capability to predict postoperative ROM of THRs.

Sign in / Sign up

Export Citation Format

Share Document