Surgical Navigation Systems Based On AR/VR Technologies

This study considers modern surgical navigation systems based on augmented reality technologies. Augmented reality glasses are used to construct holograms of the patient's organs from MRI and CT data, subsequently transmitted to the glasses. Thus, in addition to seeing the actual patient, the surgeon gains visualization inside the patient's body (bones, soft tissues, blood vessels, etc.). The solutions developed at Peter the Great St. Petersburg Polytechnic University allow reducing the invasiveness of the procedure and preserving healthy tissues. This also improves the navigation process, making it easier to estimate the location and size of the tumor to be removed.We describe the application of developed systems to different types of surgical operations (removal of a malignant brain tumor, removal of a cyst of the cervical spine). We consider the specifics of novel navigation systems designed for anesthesia, for endoscopic operations. Furthermore, we discuss the construction of novel visualization systems for ultrasound machines. Our findings indicate that the technologies proposed show potential for telemedicine.

Augmented Reality for Retrosigmoid Craniotomy Planning

While medical imaging data have traditionally been viewed on two-dimensional (2D) displays, augmented reality (AR) allows physicians to project the medical imaging data on patient's bodies to locate important anatomy. We present a surgical AR application to plan the retrosigmoid craniotomy, a standard approach to access the posterior fossa and the internal auditory canal. As a simple and accurate alternative to surface landmarks and conventional surgical navigation systems, our AR application augments the surgeon's vision to guide the optimal location of cortical bone removal. In this work, two surgeons performed a retrosigmoid approach 14 times on eight cadaver heads. In each case, the surgeon manually aligned a computed tomography (CT)-derived virtual rendering of the sigmoid sinus on the real cadaveric heads using a see-through AR display, allowing the surgeon to plan and perform the craniotomy accordingly. Postprocedure CT scans were acquired to assess the accuracy of the retrosigmoid craniotomies with respect to their intended location relative to the dural sinuses. The two surgeons had a mean margin of davg = 0.6 ± 4.7 mm and davg = 3.7 ± 2.3 mm between the osteotomy border and the dural sinuses over all their cases, respectively, and only positive margins for 12 of the 14 cases. The intended surgical approach to the internal auditory canal was successfully achieved in all cases using the proposed method, and the relatively small and consistent margins suggest that our system has the potential to be a valuable tool to facilitate planning a variety of similar skull-base procedures.

Model-based 3D Tracking for Augmented Orthopedic Surgery

Current surgical navigation systems offer sub-millimetric real-time localization, however they are expensive, require the use of invasive markers attached to the patient, and often add extra operation time. In this paper we propose an affordable markerless navigation approach, based on mid end depth sensors, as an alternative to answer medical applications needs in terms of accuracy and robustness. An algorithm called Fast Volumetric Reconstruction (FaVoR) implements a compute-efficient approach for real time 3D model registration based tracking, allowing computed 3D poses to be used for video scene augmentation. After early testing with a first proof-of-concept implementation, a preliminary accuracy evaluation was performed using a dynamic test bench, achieving an average 2mm registration error during tracking.

Towards Optical Imaging for Spine Tracking without Markers in Navigated Spine Surgery

Surgical navigation systems are increasingly used for complex spine procedures to avoid neurovascular injuries and minimize the risk for reoperations. Accurate patient tracking is one of the prerequisites for optimal motion compensation and navigation. Most current optical tracking systems use dynamic reference frames (DRFs) attached to the spine, for patient movement tracking. However, the spine itself is subject to intrinsic movements which can impact the accuracy of the navigation system. In this study, we aimed to detect the actual patient spine features in different image views captured by optical cameras, in an augmented reality surgical navigation (ARSN) system. Using optical images from open spinal surgery cases, acquired by two gray-scale cameras, spinal landmarks were identified and matched in different camera views. A computer vision framework was created for preprocessing of the spine images, detecting and matching local invariant image regions. We compared four feature detection algorithms, Speeded Up Robust Feature (SURF), Maximal Stable Extremal Region (MSER), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB) to elucidate the best approach. The framework was validated in 23 patients and the 3D triangulation error of the matched features was < 0.5 mm. Thus, the findings indicate that spine feature detection can be used for accurate tracking in navigated surgery.

HEALTHCARE 2019

The article provides a brief description of the exhibition "Healthcare 2019", which was attended by 630 companies from 30 countries and 71 cities in Russia. It was possible to get acquainted with the modern system of health care organization, the necessary simple and complex medical equipment, numerous accessories and consumables. The exhibition presented: medical x-ray equipment of wide application (computer tomographs, mammographs, mobile devices), magnetic resonance imaging, ultrasound medical equipment, microscopes, endoscopic equipment, surgical navigation systems, as well as magnetic therapy, thermography and radiothermometry. The companies showed the best examples of modern medical equipment.

A Skin-Conformal, Stretchable, and Breathable Fiducial Marker Patch for Surgical Navigation Systems

Augmented reality (AR) surgical navigation systems have attracted considerable attention as they assist medical professionals in visualizing the location of ailments within the human body that are not readily seen with the naked eye. Taking medical imaging with a parallel C-shaped arm (C-arm) as an example, surgical sites are typically targeted using an optical tracking device and a fiducial marker in real-time. These markers then guide operators who are using a multifunctional endoscope apparatus by signaling the direction or distance needed to reach the affected parts of the body. In this way, fiducial markers are used to accurately protect the vessels and nerves exposed during the surgical process. Although these systems have already shown potential for precision implantation, delamination of the fiducial marker, which is a critical component of the system, from human skin remains a challenge due to a mechanical mismatch between the marker and skin, causing registration problems that lead to poor position alignments and surgical degradation. To overcome this challenge, the mechanical modulus and stiffness of the marker patch should be lowered to approximately 150 kPa, which is comparable to that of the epidermis, while improving functionality. Herein, we present a skin-conformal, stretchable yet breathable fiducial marker for the application in AR-based surgical navigation systems. By adopting pore patterns, we were able to create a fiducial marker with a skin-like low modulus and breathability. When attached to the skin, the fiducial marker was easily identified using optical recognition equipment and showed skin-conformal adhesion when stretched and shrunk repeatedly. As such, we believe the marker would be a good fiducial marker candidate for patients under surgical navigation systems.

18F-Fluciclovine PET to distinguish treatment-related effects from disease progression in recurrent glioblastoma: PET fusion with MRI guides neurosurgical sampling

Differentiation of true tumor progression from treatment-related effects remains a major unmet need in caring for patients with glioblastoma. Here, we report how the intraoperative combination of MRI with18F-fluciclovine PET guided surgical sampling in 2 patients with recurrent glioblastoma.18F-Fluciclovine PET is FDA approved for use in prostate cancer and carries an orphan drug designation in glioma. To investigate its utility in recurrent glioblastoma, we fused PET and MRI images using 2 different surgical navigation systems and performed targeted stereotactic biopsies from the areas of high (“hot”) and low (“cold”) radiotracer uptake. Concordant histopathologic and imaging findings suggest that a combined18F-fluciclovine PET-MRI–guided approach can guide neurosurgical resection of viable recurrent glioblastoma in the background of treatment-related effects, which can otherwise look similar on MRI.

The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery

Conventional surgical navigation systems rely on preoperative imaging to provide guidance. In laparoscopic liver surgery, insufflation of the abdomen (pneumoperitoneum) can cause deformations on the liver, introducing inaccuracies in the correspondence between the preoperative images and the intraoperative reality. This study evaluates the improvements provided by intraoperative imaging for laparoscopic liver surgical navigation, when displayed as augmented reality (AR). Significant differences were found in terms of accuracy of the AR, in favor of intraoperative imaging. In addition, results showed an effect of user-induced error: image-to-patient registration based on annotations performed by clinicians caused 33% more inaccuracy as compared to image-to-patient registration algorithms that do not depend on user annotations. Hence, to achieve accurate surgical navigation for laparoscopic liver surgery, intraoperative imaging is recommendable to compensate for deformation. Moreover, user annotation errors may lead to inaccuracies in registration processes.