Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance

Background: While performing surgeries in the OR, surgeons and assistants often need to access several information regarding surgical planning and/or procedures related to the surgery itself, or the accessory equipment to perform certain operations. The accessibility of this information often relies on the physical presence of technical and medical specialists in the OR, which is increasingly difficult due to the number of limitations imposed by the COVID emergency to avoid overcrowded environments or external personnel. Here, we analyze several scenarios where we equipped OR personnel with augmented reality (AR) glasses, allowing a remote specialist to guide OR operations through voice and ad-hoc visuals, superimposed to the field of view of the operator wearing them.Methods: This study is a preliminary case series of prospective collected data about the use of AR-assistance in spine surgery from January to July 2020. The technology has been used on a cohort of 12 patients affected by degenerative lumbar spine disease with lumbar sciatica co-morbidities. Surgeons and OR specialists were equipped with AR devices, customized with P2P videoconference commercial apps, or customized holographic apps. The devices were tested during surgeries for lumbar arthrodesis in a multicenter experience involving author's Institutions.Findings: A total number of 12 lumbar arthrodesis have been performed while using the described AR technology, with application spanning from telementoring (3), teaching (2), surgical planning superimposition and interaction with the hologram using a custom application for Microsoft hololens (1). Surgeons wearing the AR goggles reported a positive feedback as for the ergonomy, wearability and comfort during the procedure; being able to visualize a 3D reconstruction during surgery was perceived as a straightforward benefit, allowing to speed-up procedures, thus limiting post-operational complications. The possibility of remotely interacting with a specialist on the glasses was a potent added value during COVID emergency, due to limited access of non-resident personnel in the OR.Interpretation: By allowing surgeons to overlay digital medical content on actual surroundings, augmented reality surgery can be exploited easily in multiple scenarios by adapting commercially available or custom-made apps to several use cases. The possibility to observe directly the operatory theater through the eyes of the surgeon might be a game-changer, giving the chance to unexperienced surgeons to be virtually at the site of the operation, or allowing a remote experienced operator to guide wisely the unexperienced surgeon during a procedure.

Download Full-text

Augmented reality–assisted ventriculostomy

Neurosurgical FOCUS ◽

10.3171/2020.10.focus20779 ◽

2021 ◽

Vol 50 (1) ◽

pp. E16

Author(s):

Max Schneider ◽

Christian Kunz ◽

Andrej Pal’a ◽

Christian Rainer Wirtz ◽

Franziska Mathis-Ullrich ◽

...

Keyword(s):

Augmented Reality ◽

Head Model ◽

Neurosurgical Procedures ◽

Reference Trajectory ◽

Success Rates ◽

Clinical Implementation ◽

Head Fixation ◽

Custom Made ◽

3D Printed ◽

Microsoft Hololens

OBJECTIVEPlacement of a ventricular drain is one of the most common neurosurgical procedures. However, a higher rate of successful placements with this freehand procedure is desirable. The authors’ objective was to develop a compact navigational augmented reality (AR)–based tool that does not require rigid patient head fixation, to support the surgeon during the operation.METHODSSegmentation and tracking algorithms were developed. A commercially available Microsoft HoloLens AR headset in conjunction with Vuforia marker-based tracking was used to provide guidance for ventriculostomy in a custom-made 3D-printed head model. Eleven surgeons conducted a series of tests to place a total of 110 external ventricular drains under holographic guidance. The HoloLens was the sole active component; no rigid head fixation was necessary. CT was used to obtain puncture results and quantify success rates as well as precision of the suggested setup.RESULTSIn the proposed setup, the system worked reliably and performed well. The reported application showed an overall ventriculostomy success rate of 68.2%. The offset from the reference trajectory as displayed in the hologram was 5.2 ± 2.6 mm (mean ± standard deviation). A subgroup conducted a second series of punctures in which results and precision improved significantly. For most participants it was their first encounter with AR headset technology and the overall feedback was positive.CONCLUSIONSTo the authors’ knowledge, this is the first report on marker-based, AR-guided ventriculostomy. The results from this first application are encouraging. The authors would expect good acceptance of this compact navigation device in a supposed clinical implementation and assume a steep learning curve in the application of this technique. To achieve this translation, further development of the marker system and implementation of the new hardware generation are planned. Further testing to address visuospatial issues is needed prior to application in humans.

Download Full-text

TBE in Italy

Tick-borne encephalitis - The Book ◽

10.33442/26613980_12b15-3 ◽

2020 ◽

Keyword(s):

Surveillance System ◽

Ad Hoc ◽

National Level ◽

Case Series ◽

Incidence Rates ◽

Mountainous Regions ◽

North East ◽

The North ◽

Tick Borne Encephalitis ◽

Low Incidence

Italy is considered a low-incidence country for tick-borne encephalitis (TBE) in Europe.1 Areas at higher risk for TBE in Italy are geographically clustered in the forested and mountainous regions and provinces in the north east part of the country, as suggested by TBE case series published over the last decade.2-5 A national enhanced surveillance system for TBE has been established since 2017.6 Before this, information on the occurrence of TBE cases at the national level in Italy was lacking. Both incidence rates and the geographical distribution of the disease were mostly inferred from endemic areas where surveillance was already in place, ad hoc studies and international literature.1

Download Full-text

Augmented Reality in Spine Surgery

World Neurosurgery ◽

10.1016/j.wneu.2021.05.041 ◽

2021 ◽

Vol 151 ◽

pp. 290

Author(s):

Alexander J. Schupper ◽

Jeremy Steinberger ◽

Yakov Gologorsky

Keyword(s):

Augmented Reality ◽

Spine Surgery

Download Full-text

Demonstration: Virtual Patient Immersive Trainer to Train Perceptual Skills Using Augmented Reality

Proceedings of the Human Factors and Ergonomics Society Annual Meeting ◽

10.1177/1071181320641106 ◽

2020 ◽

Vol 64 (1) ◽

pp. 470-472

Author(s):

Christen E. Sushereba ◽

Laura G. Militello

Keyword(s):

Medical Students ◽

Augmented Reality ◽

Student Performance ◽

Group Interaction ◽

Virtual Patient ◽

Perceptual Cues ◽

Perceptual Skills ◽

Individual Interaction ◽

Severity Of Injury ◽

Microsoft Hololens

In this session, we will demonstrate the Virtual Patient Immersive Trainer (VPIT). The VPIT system uses augmented reality (AR) to allow medics and medical students to experience a photorealistic, life-sized virtual patient. The VPIT supports learners in obtaining the perceptual skills required to recognize and interpret subtle perceptual cues critical to assessing a patient’s condition. We will conduct an interactive demonstration of the virtual patient using both a tablet (for group interaction) and an AR-enabled headset (Microsoft HoloLens) for individual interaction. In addition, we will demonstrate use of the instructor tablet to control what the learner sees (e.g., injury types, severity of injury) and to monitor student performance.

Download Full-text

First experience with augmented reality neuronavigation in endoscopic assisted midline skull base pathologies in children

Child s Nervous System ◽

10.1007/s00381-021-05049-3 ◽

2021 ◽

Author(s):

Valentina Pennacchietti ◽

Katharina Stoelzel ◽

Anna Tietze ◽

Erwin Lankes ◽

Andreas Schaumann ◽

...

Keyword(s):

Augmented Reality ◽

Skull Base ◽

Skull Base Surgery ◽

Morphological Variability ◽

Case Series ◽

Relevant Information ◽

Navigation Systems ◽

Postoperative Mri ◽

The Mean ◽

Clinical Conditions

Abstract Introduction Endoscopic skull base approaches are broadly used in modern neurosurgery. The support of neuronavigation can help to effectively target the lesion avoiding complications. In children, endoscopic-assisted skull base surgery in combination with navigation systems becomes even more important because of the morphological variability and rare diseases affecting the sellar and parasellar regions. This paper aims to analyze our first experience on augmented reality navigation in endoscopic skull base surgery in a pediatric case series. Patients and methods A retrospective review identified seventeen endoscopic-assisted endonasal or transoral procedures performed in an interdisciplinary setting in a period between October 2011 and May 2020. In all the cases, the surgical target was a lesion in the sellar or parasellar region. Clinical conditions, MRI appearance, intraoperative conditions, postoperative MRI, possible complications, and outcomes were analyzed. Results The mean age of our patients was 14.5 ± 2.4 years. The diagnosis varied, but craniopharyngiomas (31.2%) were mostly represented. AR navigation was experienced to be very helpful for effectively targeting the lesion and defining the intraoperative extension of the pathology. In 65% of the oncologic cases, a radical removal was proven in postoperative MRI. The mean follow-up was 89 ± 79 months. There were no deaths in our series. No long-term complications were registered; two cerebrospinal fluid (CSF) fistulas and a secondary abscess required further surgery. Conclusion The implementation of augmented reality to endoscopic-assisted neuronavigated procedures within the skull base was feasible and did provide relevant information directly in the endoscopic field of view and was experienced to be useful in the pediatric cases, where anatomical variability and rarity of the pathologies make surgery more challenging.

Download Full-text

shARe-IT: Ad hoc Remote Troubleshooting through Augmented Reality

Computer Supported Cooperative Work (CSCW) ◽

10.1007/s10606-021-09393-5 ◽

2021 ◽

Author(s):

Thomas Ludwig ◽

Oliver Stickel ◽

Peter Tolmie ◽

Malte Sellmer

Keyword(s):

Augmented Reality ◽

Ad Hoc ◽

Cooperative Work ◽

Computer Supported Cooperative Work ◽

Limiting Factors ◽

Minor Role ◽

Articulation Work ◽

Remote Collaboration ◽

Expertise Sharing ◽

A Minor

Abstract10 years ago, Castellani et al. (Journal of Computer Supported Cooperative Work, vol. 18, no. 2–3, 2009, pp. 199–227, 2009) showed that using just an audio channel for remote troubleshooting can lead to a range of problems and already envisioned a future in which augmented reality (AR) could solve many of these issues. In the meantime, AR technologies have found their way into our everyday lives and using such technologies to support remote collaboration has been widely studied within the fields of Human-Computer Interaction and Computer-Supported Cooperative Work. In this paper, we contribute to this body of research by reporting on an extensive empirical study within a Fab Lab of troubleshooting and expertise sharing and the potential relevance of articulation work to their realization. Based on the findings of this study, we derived design challenges that led to an AR-based concept, implemented as a HoloLens application, called shARe-it. This application is designed to support remote troubleshooting and expertise sharing through different communication channels and AR-based interaction modalities. Early testing of the application revealed that novel interaction modalities such as AR-based markers and drawings play only a minor role in remote collaboration due to various limiting factors. Instead, the transmission of a shared view and especially arriving at a shared understanding of the situation as a prerequisite for articulation work continue to be the decisive factors in remote troubleshooting.

Download Full-text

Complications and Revisions After Spine Surgery in Patients With Skeletal Dysplasia: Have We Improved?

Global Spine Journal ◽

10.1177/2192568221994786 ◽

2021 ◽

pp. 219256822199478

Author(s):

Karim Shafi ◽

Francis Lovecchio ◽

Maria Sava ◽

Michael Steinhaus ◽

Andre Samuel ◽

...

Keyword(s):

Spine Surgery ◽

Skeletal Dysplasia ◽

Spinal Surgery ◽

Cardiovascular Health ◽

Surgical Complication ◽

Case Series ◽

Neurologic Complication ◽

Complication Rates ◽

Retrospective Case ◽

Average Patient

Study Design: Retrospective case series. Objective: To report contemporary rates of complications and subsequent surgery after spinal surgery in patients with skeletal dysplasia. Methods: A case series of 25 consecutive patients who underwent spinal surgery between 2007 and 2017 were identified from a single institution’s skeletal dysplasia registry. Patient demographics, medical history, surgical indication, complications, and subsequent surgeries (revisions, extension to adjacent levels, or for pathology at a non-contiguous level) were collected. Charlson comorbidity indices were calculated as a composite measure of overall health. Results: Achondroplasia was the most common skeletal dysplasia (76%) followed by spondyloepiphyseal dysplasia (20%); 1 patient had diastrophic dysplasia (4%). Average patient age was 53.2 ± 14.7 years and most patients were in excellent cardiovascular health (88% Charlson Comorbidity Index 0-4). Mean follow up after the index procedure was 57.4 ± 39.2 months (range). Indications for surgery were mostly for neurologic symptoms. The most commonly performed surgery was a multilevel thoracolumbar decompression without fusion (57%). Complications included durotomy (36%), neurologic complication (12%), and infection requiring irrigation and debridement (8%). Nine patients (36%) underwent a subsequent surgery. Three patients (12%) underwent a procedure at a non-contiguous anatomic zone, 3 (12%) underwent a revision of the previous surgery, and another 3 (12%) required extension of their previous decompression or fusion. Conclusions: Surgical complication rates remain high after spine surgery in patients with skeletal dysplasia, likely attributable to inherent characteristics of the disease. Patients should be counseled on their risk for complication and subsequent surgery.

Download Full-text

Same-Admission Microvascular Maxillofacial Ballistic Trauma Reconstruction Using Virtual Surgical Planning: A Case Series and Systematic Review

Craniomaxillofacial Trauma & Reconstruction ◽

10.1177/19433875211026432 ◽

2021 ◽

pp. 194338752110264

Author(s):

Sean A. Knudson ◽

Kristopher M. Day ◽

Patrick Kelley ◽

Pablo Padilla ◽

Ian X. Collier ◽

...

Keyword(s):

Systematic Review ◽

Surgical Planning ◽

Average Length ◽

Case Series ◽

Retrospective Case ◽

Oral Nutrition ◽

Microsurgical Reconstruction ◽

Virtual Surgical Planning ◽

And Function ◽

Ballistic Trauma

Study Design: Retrospective case series; systematic review. Objective: It is unknown whether the use of virtual surgical planning (VSP) to facilitate same-admission microsurgical reconstruction of the mandible following acute maxillofacial ballistic trauma (MBT) is sufficient to achieve definitive reconstruction and functional occlusion. Methods: A single-center retrospective analysis was conducted for patients who underwent microsurgical reconstruction of the mandible using VSP after acute MBT. The PubMed/MEDLINE, Embase, ScienceDirect, and Scopus databases were systematically reviewed using blinded screening. Studies were evaluated via thematic analysis. Results: Five patients were treated by same-admission and microsurgical reconstruction of the mandible using VSP. We observed an average of 16.4 ± 9.1 days between initial presentation and reconstruction, an average length of stay of 51.6 ± 17.9 days, 6.2 ± 2.8 operations, and 1.6 ± 0.9 free flaps per patient. Four types and 8 total flaps were employed, most commonly the anterior lateral thigh flap (37.5%). Care yielded complete flap survival. Each patient experienced at least 1 minor complication. All patients achieved centric occlusion, oral nutrition, and an approximation of their baseline facial aesthetic. Follow up was 191.0 ± 183.9 weeks. Systematic review produced 8 articles that adhered to inclusion criteria. Consensus themes in the literature were found for clinical goal and function of VSP when practicing MBT reconstruction, yet disagreement was found surrounding optimal treatment timeline. Conclusions: Same-admission microsurgical reconstruction after MBT is safe and effective to re-establish mandibular form and function. VSP did not delay reconstruction, given the need for preparation prior to definitive reconstruction.

Download Full-text

ARETT: Augmented Reality Eye Tracking Toolkit for Head Mounted Displays

Sensors ◽

10.3390/s21062234 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2234

Author(s):

Sebastian Kapp ◽

Michael Barz ◽

Sergey Mukhametov ◽

Daniel Sonntag ◽

Jochen Kuhn

Keyword(s):

Augmented Reality ◽

Eye Tracking ◽

User Interaction ◽

R Package ◽

Eye Tracker ◽

Unity 3D ◽

Accuracy And Precision ◽

Head Mounted Displays ◽

Virtual And Augmented Reality ◽

Microsoft Hololens

Currently an increasing number of head mounted displays (HMD) for virtual and augmented reality (VR/AR) are equipped with integrated eye trackers. Use cases of these integrated eye trackers include rendering optimization and gaze-based user interaction. In addition, visual attention in VR and AR is interesting for applied research based on eye tracking in cognitive or educational sciences for example. While some research toolkits for VR already exist, only a few target AR scenarios. In this work, we present an open-source eye tracking toolkit for reliable gaze data acquisition in AR based on Unity 3D and the Microsoft HoloLens 2, as well as an R package for seamless data analysis. Furthermore, we evaluate the spatial accuracy and precision of the integrated eye tracker for fixation targets with different distances and angles to the user (n=21). On average, we found that gaze estimates are reported with an angular accuracy of 0.83 degrees and a precision of 0.27 degrees while the user is resting, which is on par with state-of-the-art mobile eye trackers.

Download Full-text