Augmented reality in the operating room: a clinical feasibility study

Abstract Background Augmented Reality (AR) is a rapidly emerging technology finding growing acceptance and application in different fields of surgery. Various studies have been performed evaluating the precision and accuracy of AR guided navigation. This study investigates the feasibility of a commercially available AR head mounted device during orthopedic surgery. Methods Thirteen orthopedic surgeons from a Swiss university clinic performed 25 orthopedic surgical procedures wearing a holographic AR headset (HoloLens, Microsoft, Redmond, WA, USA) providing complementary three-dimensional, patient specific anatomic information. The surgeon’s experience of using the device during surgery was recorded using a standardized 58-item questionnaire grading different aspects on a 100-point scale with anchor statements. Results Surgeons were generally satisfied with image quality (85 ± 17 points) and accuracy of the virtual objects (84 ± 19 point). Wearing the AR device was rated as fairly comfortable (79 ± 13 points). Functionality of voice commands (68 ± 20 points) and gestures (66 ± 20 points) provided less favorable results. The greatest potential in the use of the AR device was found for surgical correction of deformities (87 ± 15 points). Overall, surgeons were satisfied with the application of this novel technology (78 ± 20 points) and future access to it was demanded (75 ± 22 points). Conclusion AR is a rapidly evolving technology with large potential in different surgical settings, offering the opportunity to provide a compact, low cost alternative requiring a minimum of infrastructure compared to conventional navigation systems. While surgeons where generally satisfied with image quality of the here tested head mounted AR device, some technical and ergonomic shortcomings were pointed out. This study serves as a proof of concept for the use of an AR head mounted device in a real-world sterile setting in orthopedic surgery.

Download Full-text

Three-dimensional–printed marker–based augmented reality neuronavigation: a new neuronavigation technique

Neurosurgical FOCUS ◽

10.3171/2021.5.focus21206 ◽

2021 ◽

Vol 51 (2) ◽

pp. E20

Author(s):

Gorkem Yavas ◽

Kadri Emre Caliskan ◽

Mehmet Sedat Cagli

Keyword(s):

Augmented Reality ◽

Mobile Device ◽

Low Cost ◽

Three Dimensional ◽

Skin Incision ◽

Optical Tracking ◽

Patient Specific ◽

Navigation Systems ◽

Intracranial Tumors ◽

3D Printed

OBJECTIVE The aim of this study was to assess the precision and feasibility of 3D-printed marker–based augmented reality (AR) neurosurgical navigation and its use intraoperatively compared with optical tracking neuronavigation systems (OTNSs). METHODS Three-dimensional–printed markers for CT and MRI and intraoperative use were applied with mobile devices using an AR light detection and ranging (LIDAR) camera. The 3D segmentations of intracranial tumors were created with CT and MR images, and preoperative registration of the marker and pathology was performed. A patient-specific, surgeon-facilitated mobile application was developed, and a mobile device camera was used for neuronavigation with high accuracy, ease, and cost-effectiveness. After accuracy values were preliminarily assessed, this technique was used intraoperatively in 8 patients. RESULTS The mobile device LIDAR camera was found to successfully overlay images of virtual tumor segmentations according to the position of a 3D-printed marker. The targeting error that was measured ranged from 0.5 to 3.5 mm (mean 1.70 ± 1.02 mm, median 1.58 mm). The mean preoperative preparation time was 35.7 ± 5.56 minutes, which is longer than that for routine OTNSs, but the amount of time required for preoperative registration and the placement of the intraoperative marker was very brief compared with other neurosurgical navigation systems (mean 1.02 ± 0.3 minutes). CONCLUSIONS The 3D-printed marker–based AR neuronavigation system was a clinically feasible, highly precise, low-cost, and easy-to-use navigation technique. Three-dimensional segmentation of intracranial tumors was targeted on the brain and was clearly visualized from the skin incision to the end of surgery.

Download Full-text

Augmented Reality as a Tool to Guide PSI Placement in Pelvic Tumor Resections

Sensors ◽

10.3390/s21237824 ◽

2021 ◽

Vol 21 (23) ◽

pp. 7824

Author(s):

Mónica García-Sevilla ◽

Rafael Moreta-Martinez ◽

David García-Mato ◽

Alicia Pose-Diez-de-la-Lastra ◽

Rubén Pérez-Mañanes ◽

...

Keyword(s):

Augmented Reality ◽

Surgical Navigation ◽

Tumor Resection ◽

Three Dimensional ◽

Target Surface ◽

Patient Specific ◽

Resection Margins ◽

Navigation Systems ◽

Pelvic Tumor ◽

Similar Accuracy

Patient-specific instruments (PSIs) have become a valuable tool for osteotomy guidance in complex surgical scenarios such as pelvic tumor resection. They provide similar accuracy to surgical navigation systems but are generally more convenient and faster. However, their correct placement can become challenging in some anatomical regions, and it cannot be verified objectively during the intervention. Incorrect installations can result in high deviations from the planned osteotomy, increasing the risk of positive resection margins. In this work, we propose to use augmented reality (AR) to guide and verify PSIs placement. We designed an experiment to assess the accuracy provided by the system using a smartphone and the HoloLens 2 and compared the results with the conventional freehand method. The results showed significant differences, where AR guidance prevented high osteotomy deviations, reducing maximal deviation of 54.03 mm for freehand placements to less than 5 mm with AR guidance. The experiment was performed in two versions of a plastic three-dimensional (3D) printed phantom, one including a silicone layer to simulate tissue, providing more realism. We also studied how differences in shape and location of PSIs affect their accuracy, concluding that those with smaller sizes and a homogeneous target surface are more prone to errors. Our study presents promising results that prove AR’s potential to overcome the present limitations of PSIs conveniently and effectively.

Download Full-text

Hybrid Spine Simulator Prototype for X-ray Free Pedicle Screws Fixation Training

Applied Sciences ◽

10.3390/app11031038 ◽

2021 ◽

Vol 11 (3) ◽

pp. 1038

Author(s):

Sara Condino ◽

Giuseppe Turini ◽

Virginia Mamone ◽

Paolo Domenico Parchi ◽

Vincenzo Ferrari

Keyword(s):

Augmented Reality ◽

Low Cost ◽

Lumbar Vertebrae ◽

Pedicle Screws ◽

Simulation Software ◽

Patient Specific ◽

Innovative Strategy ◽

X Ray ◽

3D Printed ◽

Harmful Radiation

Simulation for surgical training is increasingly being considered a valuable addition to traditional teaching methods. 3D-printed physical simulators can be used for preoperative planning and rehearsal in spine surgery to improve surgical workflows and postoperative patient outcomes. This paper proposes an innovative strategy to build a hybrid simulation platform for training of pedicle screws fixation: the proposed method combines 3D-printed patient-specific spine models with augmented reality functionalities and virtual X-ray visualization, thus avoiding any exposure to harmful radiation during the simulation. Software functionalities are implemented by using a low-cost tracking strategy based on fiducial marker detection. Quantitative tests demonstrate the accuracy of the method to track the vertebral model and surgical tools, and to coherently visualize them in either the augmented reality or virtual fluoroscopic modalities. The obtained results encourage further research and clinical validation towards the use of the simulator as an effective tool for training in pedicle screws insertion in lumbar vertebrae.

Download Full-text

Three-dimensional technologies used for patient specific applications in orthopedics

Acta Marisiensis - Seria Medica ◽

10.2478/amma-2021-0020 ◽

2021 ◽

Vol 67 (2) ◽

pp. 77-85

Author(s):

Flaviu Moldovan ◽

Tiberiu Bataga

Keyword(s):

Orthopedic Surgery ◽

Three Dimensional ◽

Medical Application ◽

Surgical Instruments ◽

Patient Specific ◽

Innovative Technology ◽

Clinical Environment ◽

Clinical Practices ◽

Content Description ◽

Almost All

Abstract Background: Three-dimensional (3D) technologies have numerous medical applications and have gained a lot of interest in medical world. After the advent of three-dimensional printing technology, and especially in last decade, orthopedic surgeons began to apply this innovative technology in almost all areas of orthopedic traumatic surgery. Objective: The aim of this paper is to give an overview of 3D technologies current usage in orthopedic surgery for patient specific applications. Methods: Two major databases PubMed and Web of Science were explored for content description and applications of 3D technologies in orthopedic surgery. It was considered papers presenting controlled studies and series of cases that include descriptions of 3D technologies compatible with applications to human medical purposes. Results: First it is presented the available three-dimensional technologies that can be used in orthopedic surgery as well as methods of integration in order to achieve the desired medical application for patient specific orthopedics. Technology starts with medical images acquisition, followed by design, numerical simulation, and printing. Then it is described the state of the art clinical applications of 3D technologies in orthopedics, by selecting the latest reported articles in medical literature. It is focused on preoperative visualization and planning, trauma, injuries, elective orthopedic surgery, guides and customized surgical instrumentation, implants, orthopedic fixators, orthoses and prostheses. Conclusion: The new 3D digital technologies are revolutionizing orthopedic clinical practices. The vast potential of 3D technologies is increasingly used in clinical practice. These technologies provide useful tools for clinical environment: accurate preoperative planning for cases of complex trauma and elective cases, personalized surgical instruments and personalized implants. There is a need to further explore the vast potential of 3D technologies in many other areas of orthopedics and to accommodate healthcare professionals with these technologies, as well as to study their effectiveness compared to conventional methods.

Download Full-text

Augmented Reality in the Integrative Internet of Things (AR-IoT): Application for Precision Farming

Sustainability ◽

10.3390/su11092658 ◽

2019 ◽

Vol 11 (9) ◽

pp. 2658 ◽

Cited By ~ 7

Author(s):

Pilaiwan Phupattanasilp ◽

Sheau-Ru Tong

Keyword(s):

Augmented Reality ◽

Internet Of Things ◽

Data Visualization ◽

Low Cost ◽

Three Dimensional ◽

Precision Farming ◽

The Internet ◽

Environmentally Sustainable ◽

Crop Monitoring ◽

Object Interaction

Benefitted by the Internet of Things (IoT), visualization capabilities facilitate the improvement of precision farming, especially in dynamic indoor planting. However, conventional IoT data visualization is usually carried out in offsite and textual environments, i.e., text and number, which do not promote a user’s sensorial perception and interaction. This paper introduces the use of augmented reality (AR) as a support to IoT data visualization, called AR-IoT. The AR-IoT system superimposes IoT data directly onto real-world objects and enhances object interaction. As a case study, this system is applied to crop monitoring. Multi-camera, a non-destructive and low-cost imaging platform of the IoT, is connected to the internet and integrated into the system to measure the three-dimensional (3D) coordinates of objects. The relationships among accuracy, object coordinates, augmented information (e.g., virtual objects), and object interaction are investigated. The proposed system shows a great potential to integrate IoT data with AR resolution, which will effectively contribute to updating precision agricultural techniques in an environmentally sustainable manner.

Download Full-text

A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology

3D Printing in Medicine ◽

10.1186/s41205-021-00125-5 ◽

2021 ◽

Vol 7 (1) ◽

Author(s):

Nicole Wake ◽

Andrew B. Rosenkrantz ◽

William C. Huang ◽

James S. Wysock ◽

Samir S. Taneja ◽

...

Keyword(s):

Augmented Reality ◽

Patient Care ◽

Three Dimensional ◽

3D Models ◽

Patient Specific ◽

Imaging Data ◽

Urologic Oncology ◽

Case Examples ◽

Microsoft Hololens ◽

Enhance Patient Care

AbstractAugmented reality (AR) and virtual reality (VR) are burgeoning technologies that have the potential to greatly enhance patient care. Visualizing patient-specific three-dimensional (3D) imaging data in these enhanced virtual environments may improve surgeons’ understanding of anatomy and surgical pathology, thereby allowing for improved surgical planning, superior intra-operative guidance, and ultimately improved patient care. It is important that radiologists are familiar with these technologies, especially since the number of institutions utilizing VR and AR is increasing. This article gives an overview of AR and VR and describes the workflow required to create anatomical 3D models for use in AR using the Microsoft HoloLens device. Case examples in urologic oncology (prostate cancer and renal cancer) are provided which depict how AR has been used to guide surgery at our institution.

Download Full-text

PHOTOGRAMMETRIC PROCESSES AND AUGMENTED REALITY APPLICATIONS USING MOBILE DEVICES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlvi-m-1-2021-101-2021 ◽

2021 ◽

Vol XLVI-M-1-2021 ◽

pp. 101-106

Author(s):

M. Campi ◽

A. di Luggo ◽

M. Falcone

Keyword(s):

Augmented Reality ◽

Continuous Monitoring ◽

Low Cost ◽

Three Dimensional ◽

Basic Element ◽

Significant Progress ◽

Rapid Mapping ◽

Ideal Solutions ◽

First Results ◽

Three Dimensional Models

Abstract. This contribution shows the first results of a research in fieri, which aims to introduce low-cost instruments for the continuous monitoring of architectures. The use of these devices in the architectural field is of great interest to the scientific community and therefore, with the aim of researching rapid mapping methodologies, the integrated camera on the new iPhone 12PRO is analyzed and then compared with the NikonD5000 reflex camera, whose use is more consolidated. In the era of digitalization and industry 4.0, smartphones have made significant progress and these devices are establishing as ideal solutions, thanks to their technical characteristics, costs and portability, compared to other acquisition techniques. The study is aimed at experimenting with image-based methodologies with the use of low-cost sensors where the three-dimensional models will constitute the basic element on which, through augmented reality applications, it will be possible to implement strategies aimed at documentation, conservation and monitoring. The experiment, reported therein, was conducted on the facade of the Quadriportico of the Cathedral of San Matteo in Salerno, Italy.

Download Full-text

Three-Dimensional Empirical AoA Localization Technique for Indoor Applications

Sensors ◽

10.3390/s19245544 ◽

2019 ◽

Vol 19 (24) ◽

pp. 5544 ◽

Cited By ~ 1

Author(s):

Abdallah Alma’aitah ◽

Baha’ Alsaify ◽

Raed Bani-Hani

Keyword(s):

Antenna Arrays ◽

Low Cost ◽

Three Dimensional ◽

Reference Points ◽

Average Error ◽

Navigation Systems ◽

Angle Of Arrival ◽

Received Signal Strength Indicator ◽

Localization Technique ◽

Localization Accuracy

Small and pervasive devices have been increasingly used to identify and track objects automatically. Consequently, several low-cost localization schemes have been proposed in the literature based on angle of arrival (AoA), time difference of arrival (TDoA), received signal strength indicator (RSSI) or their combinations. In this paper, we propose a three-dimensional empirical AoA localization (TDEAL) technique for battery-powered devices. The proposed technique processes the AoA measurements at fixed reader nodes to estimate the locations of the tags. The proposed technique provides localization accuracy that mitigates non-linear empirical errors in AoA measurements. We utilize two omni-directional antenna arrays at each fixed reader node to estimate the location vector. With multiple location estimations from different fixed reader nodes, each estimated location is assigned a weight that is inversely proportional to the AoA phase-difference error. Furthermore, the actual AoA parabolic formula of the location is approximated to a cone to simplify the location calculation process. The proposed localization technique has a low hardware cost, low computational requirements, and precise location estimates. Based on the performance evaluation, significant location accuracy is achieved by TDEAL; where, for instance, an average error margin of less than 13 cm is achieved using 10 readers in an area of 10 m × 10 m . TDEAL can be utilized to provide reference points when integrated with a relative (e.g., inertial navigation systems) localization systems.

Download Full-text

Fabrication of 3D Models for Microtia Reconstruction Using Smartphone-Based Technology

Annals of Otology Rhinology & Laryngology ◽

10.1177/00034894211024051 ◽

2021 ◽

pp. 000348942110240

Author(s):

Peng You ◽

Yi-Chun Carol Liu ◽

Rodrigo C. Silva

Keyword(s):

3D Model ◽

Low Cost ◽

Tertiary Care ◽

Three Dimensional ◽

Basic Research ◽

3D Models ◽

University Hospital ◽

Patient Specific ◽

Surface Scanning ◽

Barrier To Entry

Objective: Microtia reconstruction is technically challenging due to the intricate contours of the ear. It is common practice to use a two-dimensional tracing of the patient’s normal ear as a template for the reconstruction of the affected side. Recent advances in three-dimensional (3D) surface scanning and printing have expanded the ability to create surgical models preoperatively. This study aims to describe a simple and affordable process to fabricate patient-specific 3D ear models for use in the operating room. Study design: Applied basic research on a novel 3D optical scanning and fabrication pathway for microtia reconstruction. Setting: Tertiary care university hospital. Methods: Optical surface scanning of the patient’s normal ear was completed using a smartphone with facial recognition capability. The Heges application used the phone’s camera to capture the 3D image. The 3D model was digitally isolated and mirrored using the Meshmixer software and printed with a 3D printer (MonopriceTM Select Mini V2) using polylactic acid filaments. Results: The 3D model of the ear served as a helpful intraoperative reference and an adjunct to the traditional 2D template. Collectively, time for imaging acquisition, editing, and fabrication was approximately 3.5 hours. The upfront cost was around $210, and the recurring cost was approximately $0.35 per ear model. Conclusion: A novel, low-cost approach to fabricate customized 3D models of the ear is introduced. It is feasible to create individualized 3D models using currently available consumer technology. The low barrier to entry raises the possibility for clinicians to incorporate 3D printing into various clinical applications.

Download Full-text

Using 3D Printing for Pre-planning of Orthopedic Surgeries, a Review

Journal of Pharmaceutical Research International ◽

10.9734/jpri/2021/v33i231147 ◽

2021 ◽

pp. 53-62

Author(s):

Hamidreza Mosleh ◽

Shahab Aldin Nazeri ◽

Mehdi Mehdizadeh ◽

Fatemeh Moradi ◽

Hoda Mosleh ◽

...

Keyword(s):

3D Printing ◽

Three Dimensional ◽

Google Scholar ◽

Novel Technology ◽

Orthopedic Surgeons ◽

Recent Developments ◽

3D Printed ◽

Preoperative Plan ◽

Almost All ◽

Near Future

Background: Recent developments in 3D printing have gave orthopedic surgeons among a novel technology that has the ability to revolutionize preoperative planning. The appearance of 3D printing technology (3DPT) enables the digital preoperative plan & simulation to move from the virtual phase to the reality phase. Numerous fields of medicine are lately benefiting from the operate of 3D printing, including the arising part of 3D printing in orthopedic surgery. Methods: We searched on PubMed and Google Scholar databases in January 2020 to find papers and studies about using 3D printing in orthopedy for aim of preplanning. The key words for search were (“3D printing” OR “3D-printed Model” OR “three-dimensional Printer”) AND (“Orthopedy” OR “Orthopedics” OR “Orthopedics”) AND (“Surgery” OR “Operation”) AND (“Pre-planning” OR “Plan”) AND (“Fracture” OR “Trauma”) that we used compound. We exclude the papers which their titles or abstracts were not relevant. At last, we select the most related papers to use in this article. Results: The search on PubMed found 80 Papers and on Google Scholar found 104 papers. After excluding similar and unrelated papers, 44 papers were selected for this review article. Conclusion: Almost all studies have shown us that using a 3D model can have a very positive effect on the surgical process and its outcomes, as well as patient and surgeon satisfaction. Therefore, we anticipate that this technology will be used in many orthopedic surgeries in the near future.

Download Full-text