Using virtual 3D-models in surgical planning: workflow of an immersive virtual reality application in liver surgery

Abstract Purpose Three-dimensional (3D) surgical planning is widely accepted in liver surgery. Currently, the 3D reconstructions are usually presented as 3D PDF data on regular monitors. 3D-printed liver models are sometimes used for education and planning. Methods We developed an immersive virtual reality (VR) application that enables the presentation of preoperative 3D models. The 3D reconstructions are exported as STL files and easily imported into the application, which creates the virtual model automatically. The presentation is possible in “OpenVR”-ready VR headsets. To interact with the 3D liver model, VR controllers are used. Scaling is possible, as well as changing the opacity from invisible over transparent to fully opaque. In addition, the surgeon can draw potential resection lines on the surface of the liver. All these functions can be used in a single or multi-user mode. Results Five highly experienced HPB surgeons of our department evaluated the VR application after using it for the very first time and considered it helpful according to the “System Usability Scale” (SUS) with a score of 76.6%. Especially with the subitem “necessary learning effort,” it was shown that the application is easy to use. Conclusion We introduce an immersive, interactive presentation of medical volume data for preoperative 3D liver surgery planning. The application is easy to use and may have advantages over 3D PDF and 3D print in preoperative liver surgery planning. Prospective trials are needed to evaluate the optimal presentation mode of 3D liver models.

Download Full-text

SURG-03. IMMERSIVE VIRTUAL REALITY APPLICATIONS IN NEUROSURGICAL ONCOLOGY

Neuro-Oncology ◽

10.1093/neuonc/noaa222.800 ◽

2020 ◽

Vol 22 (Supplement_3) ◽

pp. iii461-iii461

Author(s):

Andrea Carai ◽

Angela Mastronuzzi ◽

Giovanna Stefania Colafati ◽

Paul Voicu ◽

Nicola Onorini ◽

...

Keyword(s):

Virtual Reality ◽

Real Time ◽

New Technologies ◽

Young Patients ◽

Spatial Relationship ◽

3D Models ◽

Immersive Virtual Reality ◽

Global Improvement ◽

3D Rendering ◽

Before And After

Abstract Tridimensional (3D) rendering of volumetric neuroimaging is increasingly been used to assist surgical management of brain tumors. New technologies allowing immersive virtual reality (VR) visualization of obtained models offer the opportunity to appreciate neuroanatomical details and spatial relationship between the tumor and normal neuroanatomical structures to a level never seen before. We present our preliminary experience with the Surgical Theatre, a commercially available 3D VR system, in 60 consecutive neurosurgical oncology cases. 3D models were developed from volumetric CT scans and MR standard and advanced sequences. The system allows the loading of 6 different layers at the same time, with the possibility to modulate opacity and threshold in real time. Use of the 3D VR was used during preoperative planning allowing a better definition of surgical strategy. A tailored craniotomy and brain dissection can be simulated in advanced and precisely performed in the OR, connecting the system to intraoperative neuronavigation. Smaller blood vessels are generally not included in the 3D rendering, however, real-time intraoperative threshold modulation of the 3D model assisted in their identification improving surgical confidence and safety during the procedure. VR was also used offline, both before and after surgery, in the setting of case discussion within the neurosurgical team and during MDT discussion. Finally, 3D VR was used during informed consent, improving communication with families and young patients. 3D VR allows to tailor surgical strategies to the single patient, contributing to procedural safety and efficacy and to the global improvement of neurosurgical oncology care.

Download Full-text

Streaming and Exploration of Dynamically Changing Dense 3D Reconstructions in Immersive Virtual Reality

2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) ◽

10.1109/ismar-adjunct.2016.0035 ◽

2016 ◽

Cited By ~ 7

Author(s):

Annette Mossel ◽

Manuel Kroeter

Keyword(s):

Virtual Reality ◽

Immersive Virtual Reality ◽

3D Reconstructions

Download Full-text

A Collaborative Virtual Reality Environment for Liver Surgery Planning

Computers & Graphics ◽

10.1016/j.cag.2021.07.009 ◽

2021 ◽

Author(s):

Vuthea Chheang ◽

Patrick Saalfeld ◽

Fabian Joeres ◽

Christian Boedecker ◽

Tobias Huber ◽

...

Keyword(s):

Virtual Reality ◽

Liver Surgery ◽

Virtual Reality Environment ◽

Surgery Planning ◽

Collaborative Virtual Reality

Download Full-text

INTERACTIVE IMMERSIVE VIRTUALMUSEUM: DIGITAL DOCUMENTATION FOR VIRTUAL INTERACTION

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-251-2018 ◽

2018 ◽

Vol XLII-2 ◽

pp. 251-257 ◽

Cited By ~ 3

Author(s):

P. Clini ◽

L. Ruggeri ◽

R. Angeloni ◽

M. Sasso

Keyword(s):

Virtual Reality ◽

Cultural Heritage ◽

3D Models ◽

Fragile Sites ◽

Virtual Space ◽

Public Access ◽

Immersive Virtual Reality ◽

Educational Approach ◽

Digital Documentation ◽

Roman Forum

Thanks to their playful and educational approach Virtual Museum systems are very effective for the communication of Cultural Heritage. Among the latest technologies Immersive Virtual Reality is probably the most appealing and potentially effective to serve this purpose; nevertheless, due to a poor user-system interaction, caused by an incomplete maturity of a specific technology for museum applications, it is still quite uncommon to find immersive installations in museums.<br> This paper explore the possibilities offered by this technology and presents a workflow that, starting from digital documentation, makes possible an interaction with archaeological finds or any other cultural heritage inside different kinds of immersive virtual reality spaces.<br> Two different cases studies are presented: the National Archaeological Museum of Marche in Ancona and the 3D reconstruction of the Roman Forum of Fanum Fortunae. Two different approaches not only conceptually but also in contents; while the Archaeological Museum is represented in the application simply using spherical panoramas to give the perception of the third dimension, the Roman Forum is a 3D model that allows visitors to move in the virtual space as in the real one.<br> In both cases, the acquisition phase of the artefacts is central; artefacts are digitized with the photogrammetric technique Structure for Motion then they are integrated inside the immersive virtual space using a PC with a HTC Vive system that allows the user to interact with the 3D models turning the manipulation of objects into a fun and exciting experience.<br> The challenge, taking advantage of the latest opportunities made available by photogrammetry and ICT, is to enrich visitors’ experience in Real Museum making possible the interaction with perishable, damaged or lost objects and the public access to inaccessible or no longer existing places promoting in this way the preservation of fragile sites.

Download Full-text

Immersive virtual reality surgical planning of minimally invasive coronary artery bypass for Kawasaki disease

European Heart Journal ◽

10.1093/eurheartj/ehaa518 ◽

2020 ◽

Vol 41 (34) ◽

pp. 3279-3279 ◽

Cited By ~ 1

Author(s):

Amir H Sadeghi ◽

Yannick J H J Taverne ◽

Ad J J C Bogers ◽

Edris A F Mahtab

Keyword(s):

Virtual Reality ◽

Coronary Artery ◽

Kawasaki Disease ◽

Minimally Invasive ◽

Coronary Artery Bypass ◽

Surgical Planning ◽

Artery Bypass ◽

Immersive Virtual Reality

Download Full-text

Remixing real and imaginary in art education with fully immersive virtual reality

International Journal of Education through Art ◽

10.1386/eta_00077_1 ◽

2021 ◽

Vol 17 (3) ◽

pp. 415-431 ◽

Cited By ~ 1

Author(s):

Martina Paatela-Nieminen

Keyword(s):

Virtual Reality ◽

Student Teachers ◽

Art Education ◽

Visual Art ◽

Real Life ◽

3D Models ◽

Immersive Virtual Reality ◽

3D Objects ◽

Teachers Experiences ◽

Digital Material

This article explores digital material/ism by examining student teachers’ experiences, processes and products with fully immersive virtual reality (VR) as part of visual art education. The students created and painted a virtual world, given the name Gretan puutarha (‘Greta’s Garden’), using the Google application Tilt Brush. They also applied photogrammetry techniques to scan 3D objects from the real world in order to create 3D models for their VR world. Additionally, they imported 2D photographs and drawings along with applied animated effects to construct their VR world digitally, thereby remixing elements from real life and fantasy. The students were asked open-ended questions to find out how they created art virtually and the results were analysed using Burdea’s VR concepts of immersion, interaction and imagination. Digital material was created intersubjectively and intermedially while it was also remixed with real and imaginary. Various webs of meanings were created, both intertextual and rhizomatic in nature.

Download Full-text

IMHOTEP: cross-professional evaluation of a three-dimensional virtual reality system for interactive surgical operation planning, tumor board discussion and immersive training for complex liver surgery in a head-mounted display

Surgical Endoscopy ◽

10.1007/s00464-020-08246-4 ◽

2021 ◽

Author(s):

Hannes Götz Kenngott ◽

Micha Pfeiffer ◽

Anas Amin Preukschas ◽

Lisa Bettscheider ◽

Philipp Anthony Wise ◽

...

Keyword(s):

Virtual Reality ◽

Patient Care ◽

Liver Surgery ◽

Preoperative Planning ◽

Medical Training ◽

3D Models ◽

Patient Data ◽

Tumor Board ◽

Operation Planning ◽

And Training

Abstract Background Virtual reality (VR) with head-mounted displays (HMD) may improve medical training and patient care by improving display and integration of different types of information. The aim of this study was to evaluate among different healthcare professions the potential of an interactive and immersive VR environment for liver surgery that integrates all relevant patient data from different sources needed for planning and training of procedures. Methods 3D-models of the liver, other abdominal organs, vessels, and tumors of a sample patient with multiple hepatic masses were created. 3D-models, clinical patient data, and other imaging data were visualized in a dedicated VR environment with an HMD (IMHOTEP). Users could interact with the data using head movements and a computer mouse. Structures of interest could be selected and viewed individually or grouped. IMHOTEP was evaluated in the context of preoperative planning and training of liver surgery and for the potential of broader surgical application. A standardized questionnaire was voluntarily answered by four groups (students, nurses, resident and attending surgeons). Results In the evaluation by 158 participants (57 medical students, 35 resident surgeons, 13 attending surgeons and 53 nurses), 89.9% found the VR system agreeable to work with. Participants generally agreed that complex cases in particular could be assessed better (94.3%) and faster (84.8%) with VR than with traditional 2D display methods. The highest potential was seen in student training (87.3%), resident training (84.6%), and clinical routine use (80.3%). Least potential was seen in nursing training (54.8%). Conclusions The present study demonstrates that using VR with HMD to integrate all available patient data for the preoperative planning of hepatic resections is a viable concept. VR with HMD promises great potential to improve medical training and operation planning and thereby to achieve improvement in patient care.

Download Full-text

O74: ASSESSING THE USEFULNESS OF THREE-DIMENSIONAL IMAGING AND VIRTUAL REALITY IN SURGICAL PLANNING: A MEDICAL STUDENT LED PILOT STUDY

British Journal of Surgery ◽

10.1093/bjs/znab117.074 ◽

2021 ◽

Vol 108 (Supplement_1) ◽

Author(s):

C Ryan ◽

E O'Malley ◽

D Sheppard

Keyword(s):

Virtual Reality ◽

Medical Student ◽

Pilot Study ◽

Surgical Planning ◽

Three Dimensional ◽

Surgical Techniques ◽

3D Models ◽

Bleeding Complications ◽

Three Dimensional Imaging ◽

Patient Counselling

Abstract Introduction Nephron-sparing surgery is becoming more common as surgical techniques advance. VR and 3D visualisation appear to provide better anatomical understanding in presurgical planning than two-dimensional alone. 3D models may enable greater tissue salvation and fewer complications. 3D model preparation and advancing research is expensive and time consuming. We aim to pilot study led by medical student, create reliable anatomical kidney models and assess usefulness in surgical planning. Method Routine CT urograms were performed on 128 slice scanner using split bolus technique. Medical student segmented and displayed models in VR using 3DSlicer. Radiology registrar and consultant validated models. Two urology surgeons completed qualitative questionnaires. Result We included two patients. Only minor segmentation tweaks by radiologist ensured accurately demonstrated tumors. Tissue contrast quality varied between CT scans complicating segmentation. Both surgeons deemed models helpful in visualising hilar anatomy, predicting bleeding complications, determining laparoscopic/open and partial/full nephrectomy approach. Surgeons prioritised vasculature visualisation over collecting system. Surgeons suggested gauging tumor depth would be useful. Considering 3D printing cost, surgeons agreed VR alone may suffice. Conclusion Surgeons found 3D and VR enabled accurate surgical planning and patient counselling regarding nephrectomy risk. Minor CT protocol recommendations enable easier and more accurate segmentation, without increasing patient's radiation exposure. Annual leave during 8-week summer project reflects case numbers. Since, we've identified more cases to assess surgical parameters against matched cohort. We've begun work for adrenal surgery. It's feasible for medical students with minimal surgical/radiological knowledge to advance this research, gaining valuable experience. Abbrev Virtual Reality(VR), Three-dimensional(3D), Computerised Tomography(CT) Take-home message Three-dimensional imaging and virtual reality may improve surgical planning and patient counselling. Regarding nephrectomy, this could give surgeons the confidence to convert from full nephrectomy to partial nephrectomy approach, improving patient outcome.

Download Full-text

Can Immersive Virtual Reality Videogames Help Parkinson’s Disease Patients? A Case Study

Sensors ◽

10.3390/s21144825 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4825

Author(s):

Pablo Campo-Prieto ◽

Gustavo Rodríguez-Fuentes ◽

José Mª Cancela-Carral

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Virtual Reality ◽

Video Games ◽

Adverse Effects ◽

Ad Hoc ◽

The Elderly ◽

Video Gaming ◽

Immersive Virtual Reality ◽

System Usability Scale

Video games have proven useful in physical rehabilitation therapy. Accessibility, however, is limited for some groups such as the elderly or patients with Parkinson’s disease (PD). We explore the potential of fully immersive video games as a rehabilitation tool in PD patients. Four patients with mild-moderate PD (3 males:1 female, 53–71 years) participated in the study. Training consisted in two immersive virtual reality video gaming sessions. Outcomes were evaluated using System Usability Scale (SUS), Simulator Sickness Questionnaire (SSQ), Game Experience Questionnaire-post game (GEQ), an ad hoc satisfaction questionnaire and perceived effort. All participants completed the sessions without adverse effects (100%), without SSQ symptoms reported. Post-gaming SUS was >75% in both sessions (range 75–80%). Post-gaming GEQ scores were 3.3–4.0/4 in both sessions. Immersive virtual reality video gaming is feasible in patients with mild-moderate PD, with positive usability and patient satisfaction, and no adverse effects.

Download Full-text