A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology

AbstractAugmented reality (AR) and virtual reality (VR) are burgeoning technologies that have the potential to greatly enhance patient care. Visualizing patient-specific three-dimensional (3D) imaging data in these enhanced virtual environments may improve surgeons’ understanding of anatomy and surgical pathology, thereby allowing for improved surgical planning, superior intra-operative guidance, and ultimately improved patient care. It is important that radiologists are familiar with these technologies, especially since the number of institutions utilizing VR and AR is increasing. This article gives an overview of AR and VR and describes the workflow required to create anatomical 3D models for use in AR using the Microsoft HoloLens device. Case examples in urologic oncology (prostate cancer and renal cancer) are provided which depict how AR has been used to guide surgery at our institution.

Download Full-text

Custom Patient-Specific Three-Dimensional Printed Mitral Valve Models for Pre-Operative Patient Education Enhance Patient Satisfaction and Understanding

Journal of Medical Devices ◽

10.1115/1.4043737 ◽

2019 ◽

Vol 13 (3) ◽

Author(s):

Kay S. Hung ◽

Michael J. Paulsen ◽

Hanjay Wang ◽

Camille Hironaka ◽

Y. Joseph Woo

Keyword(s):

Patient Education ◽

Mitral Valve ◽

Three Dimensional ◽

3D Models ◽

Patient Specific ◽

Imaging Data ◽

Anatomical Models ◽

Mitral Valves ◽

Flexible Polymer ◽

3D Printed

In recent years, advances in medical imaging and three-dimensional (3D) additive manufacturing techniques have increased the use of 3D-printed anatomical models for surgical planning, device design and testing, customization of prostheses, and medical education. Using 3D-printing technology, we generated patient-specific models of mitral valves from their pre-operative cardiac imaging data and utilized these custom models to educate patients about their anatomy, disease, and treatment. Clinical 3D transthoracic and transesophageal echocardiography images were acquired from patients referred for mitral valve repair surgery and segmented using 3D modeling software. Patient-specific mitral valves were 3D-printed using a flexible polymer material to mimic the precise geometry and tissue texture of the relevant anatomy. 3D models were presented to patients at their pre-operative clinic visit and patient education was performed using either the 3D model or the standard anatomic illustrations. Afterward, patients completed questionnaires assessing knowledge and satisfaction. Responses were calculated based on a 1–5 Likert scale and analyzed using a nonparametric Mann–Whitney test. Twelve patients were presented with a patient-specific 3D-printed mitral valve model in addition to standard education materials and twelve patients were presented with only standard educational materials. The mean survey scores were 64.2 (±1.7) and 60.1 (±5.9), respectively (p = 0.008). The use of patient-specific anatomical models positively impacts patient education and satisfaction, and is a feasible method to open new opportunities in precision medicine.

Download Full-text

3D modelling for realistic training and learning

Turkish Journal of Biochemistry ◽

10.1515/tjb-2019-0182 ◽

2020 ◽

Vol 0 (0) ◽

Author(s):

Ayse Hilal Bati

Keyword(s):

3D Printing ◽

3D Reconstruction ◽

Three Dimensional ◽

3D Models ◽

3D Modelling ◽

Patient Specific ◽

Physical Interaction ◽

Imaging Data ◽

Data Set ◽

Modelling Techniques

AbstractObjectivesThree-dimensional (3D) reconstruction and modelling techniques based on computer vision have shown significant progress in recent years. Patient-specific models, which are derived from the imaging data set and are anatomically consistent with each other, are important for the development of knowledge and skills. The purpose of this article is to share information about three-dimensional (3D) reconstruction and modelling techniques and its importance in medical education.MethodsAs 3D printing technology develops and costs are lower, adaptation to the original model will increase, thus making models suitable for the anatomical structure and texture. 3D printing has emerged as an innovative way to help surgeons implement more complex procedures.ResultsRecent studies have shown that 3D modelling is a powerful tool for pre-operative planning, proofing, and decision-making. 3D models have excellent potential for alternative interventions and surgical training on both normal and pathological anatomy. 3D printing is an attractive, powerful and versatile technology.ConclusionsPatient-specific models can improve performance and improve learning faster, while improving the knowledge, management and confidence of trainees, whatever their area of expertise. Physical interaction with models has proven to be the key to gaining the necessary motor skills for surgical intervention.

Download Full-text

Combining patient-specific, digital 3D models with tele-education for adolescents with CHD

Cardiology in the Young ◽

10.1017/s1047951121003243 ◽

2021 ◽

pp. 1-6

Author(s):

David Liddle ◽

Sheri Balsara ◽

Karin Hamann ◽

Adam Christopher ◽

Laura Olivieri ◽

...

Keyword(s):

Medical Condition ◽

Three Dimensional ◽

Medical Knowledge ◽

3D Models ◽

Patient Specific ◽

Coarctation Of Aorta ◽

Education Session ◽

Post Intervention ◽

Cardiac Defects ◽

Transposition Of Great Arteries

Abstract Introduction: Adolescents with CHD require transition to specialised adult-centred care. Previous studies have shown that adolescents’ knowledge of their medical condition is correlated with transition readiness. Three-dimensional printed models of CHD have been used to educate medical trainees and patients, although no studies have focused on adolescents with CHD. This study investigates the feasibility of combining patient-specific, digital 3D heart models with tele-education interventions to improve the medical knowledge of adolescents with CHD. Methods: Adolescent patients with CHD, aged between 13 and 18 years old, were enrolled and scheduled for a tele-education session. Patient-specific digital 3D heart models were created using images from clinically indicated cardiac magnetic resonance studies. The tele-education session was performed using commercially available, web-conferencing software (Zoom, Zoom Video Communications Inc.) and a customised software (Cardiac Review 3D, Indicated Inc.) incorporating an interactive display of the digital 3D heart model. Medical knowledge was assessed using pre- and post-session questionnaires that were scored by independent reviewers. Results: Twenty-two adolescents completed the study. The average age of patients was 16 years old (standard deviation 1.5 years) and 56% of patients identified as female. Patients had a variety of cardiac defects, including tetralogy of Fallot, transposition of great arteries, and coarctation of aorta. Post-intervention, adolescents’ medical knowledge of their cardiac defects and cardiac surgeries improved compared to pre-intervention (p < 0.01). Conclusions: Combining patient-specific, digital 3D heart models with tele-education sessions can improve adolescents’ medical knowledge and may assist with transition to adult-centred care.

Download Full-text

Evaluation of Spherical Parameterization Methods for Three-Dimensional Reconstruction of Medical Images

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.365-366.1342 ◽

2013 ◽

Vol 365-366 ◽

pp. 1342-1349

Author(s):

Xing Hui Wu ◽

Zhi Xiu Hao

Keyword(s):

Medical Images ◽

Three Dimensional ◽

3D Models ◽

Mapping Method ◽

Geometric Error ◽

Patient Specific ◽

Shape Modelling ◽

Dimensional Reconstruction ◽

Statistical Shape ◽

Spherical Parameterization

The spherical parameterization is important for the correspondence problem that is a major part of statistical shape modelling for the reconstruction of patient-specific 3D models from medical images. In this paper, we present comparative studies of five common spherical mapping methods applied to the femur and tibia models: the Issenburg et al. method, the Alexa method, the Saba et al. method, the Praun et al. method and the Shen et al. method. These methods are evaluated using three sets of measures: distortion property, geometric error and distance to standard landmarks. Results show that the Praun et al. method performs better than other methods while the Shen et al. method can be regarded as the most reliable one for providing an acceptable correspondence result. We suggest that the area preserving property can be used as a sufficient condition while the angle preserving property is not important when choosing a spherical mapping method for correspondence application.

Download Full-text

Three-Dimensional Modelling for Cultural Heritage

Methods and Applications of Geospatial Technology in Sustainable Urbanism - Advances in Geospatial Technologies ◽

10.4018/978-1-7998-2249-3.ch014 ◽

2021 ◽

pp. 418-443

Author(s):

Luis Marques ◽

Josep Roca

Keyword(s):

Augmented Reality ◽

3D Modeling ◽

Three Dimensional ◽

Human Perception ◽

3D Models ◽

3D Modelling ◽

Digital Archives ◽

Spatial Environment ◽

The City ◽

Model Visualization

The creation of 3D models of urban elements is extremely relevant for urbanists constituting digital archives and being especially useful for enriching maps and databases or reconstructing and analyzing objects/areas through time, building/recreating scenarios and implementing intuitive methods of interaction. The widespread data available online offer new opportunities to generate realistic 3D models without the need to go physically to the place. This chapter aims to demonstrate the potential 3D modeling and visualization/interaction of urban elements in the city for multiple purposes, and it is organized in four main topics: The first deals with the theoretical framework regarding the bases of the human perception of the spatial environment and the importance of 3D modelling. The second and third deal with technical procedures on terrestrial/aerial data acquisition and demonstrate alternatively data gathered online to generate 3D models for the visualization of urban elements of the city, and the fourth introduces 3D model visualization within an augmented reality environment.

Download Full-text

Main Clinical Use of Additive Manufacturing (Three-Dimensional Printing) in Finland Restricted to the Head and Neck Area in 2016–2017

Scandinavian Journal of Surgery ◽

10.1177/1457496919840958 ◽

2019 ◽

Vol 109 (2) ◽

pp. 166-173 ◽

Cited By ~ 3

Author(s):

A.B.V. Pettersson ◽

M. Salmi ◽

P. Vallittu ◽

W. Serlo ◽

J. Tuomi ◽

...

Keyword(s):

Additive Manufacturing ◽

Computer Aided Design ◽

Three Dimensional ◽

Patient Specific ◽

Future Research ◽

Imaging Data ◽

University Hospitals ◽

Three Dimensional Printing ◽

Dimensional Printing ◽

Head Area

Background and Aims: Additive manufacturing or three-dimensional printing is a novel production methodology for producing patient-specific models, medical aids, tools, and implants. However, the clinical impact of this technology is unknown. In this study, we sought to characterize the clinical adoption of medical additive manufacturing in Finland in 2016–2017. We focused on non-dental usage at university hospitals. Materials and Methods: A questionnaire containing five questions was sent by email to all operative, radiologic, and oncologic departments of all university hospitals in Finland. Respondents who reported extensive use of medical additive manufacturing were contacted with additional, personalized questions. Results: Of the 115 questionnaires sent, 58 received answers. Of the responders, 41% identified as non-users, including all general/gastrointestinal (GI) and vascular surgeons, urologists, and gynecologists; 23% identified as experimenters or previous users; and 36% identified as heavy users. Usage was concentrated around the head area by various specialties (neurosurgical, craniomaxillofacial, ear, nose and throat diseases (ENT), plastic surgery). Applications included repair of cranial vault defects and malformations, surgical oncology, trauma, and cleft palate reconstruction. Some routine usage was also reported in orthopedics. In addition to these patient-specific uses, we identified several off-the-shelf medical components that were produced by additive manufacturing, while some important patient-specific components were produced by traditional methodologies such as milling. Conclusion: During 2016–2017, medical additive manufacturing in Finland was routinely used at university hospitals for several applications in the head area. Outside of this area, usage was much less common. Future research should include all patient-specific products created by a computer-aided design/manufacture workflow from imaging data, instead of concentrating on the production methodology.

Download Full-text

Placing Three-Dimensional Models in an Uncalibrated Single Image of an Architectural Scene

Presence Teleoperators & Virtual Environments ◽

10.1162/1054746043280583 ◽

2004 ◽

Vol 13 (6) ◽

pp. 692-707 ◽

Cited By ~ 3

Author(s):

Sara Keren ◽

Ilan Shimshoni ◽

Ayellet Tal

Keyword(s):

Augmented Reality ◽

Three Dimensional ◽

3D Models ◽

Automatic Extraction ◽

Single Image ◽

Vanishing Points ◽

Correct Position ◽

3D Objects ◽

Position And Orientation ◽

Three Dimensional Models

This paper discusses the problem of inserting 3D models into a single image. The main focus of the paper is on the accurate recovery of the camera's parameters, so that 3D models can be inserted in the “correct” position and orientation. The paper addresses two issues. The first is an automatic extraction of the principal vanishing points from an image. The second is a theoretical and an experimental analysis of the errors. To test the concept, a system that “plants” virtual 3D objects in the image was implemented. It was tested on many indoor augmented-reality scenes. Our analysis and experiments have shown that errors in the placement of the objects are unnoticeable.

Download Full-text

Digital Design and 3D Printing of Aortic Arch Reconstruction in HLHS for Surgical Simulation and Training

World Journal for Pediatric and Congenital Heart Surgery ◽

10.1177/2150135118771323 ◽

2018 ◽

Vol 9 (4) ◽

pp. 454-458 ◽

Cited By ~ 8

Author(s):

Sarah A. Chen ◽

Chin Siang Ong ◽

Nagina Malguria ◽

Luca A. Vricella ◽

Juan R. Garcia ◽

...

Keyword(s):

3D Printing ◽

Aortic Arch ◽

Surgical Simulation ◽

Three Dimensional ◽

3D Models ◽

Digital Design ◽

Patient Specific ◽

Aortic Arch Reconstruction ◽

Arch Reconstruction ◽

And Training

Purpose: Patients with hypoplastic left heart syndrome (HLHS) present a diverse spectrum of aortic arch morphology. Suboptimal geometry of the reconstructed aortic arch may result from inappropriate size and shape of an implanted patch and may be associated with poor outcomes. Meanwhile, advances in diagnostic imaging, computer-aided design, and three-dimensional (3D) printing technology have enabled the creation of 3D models. The purpose of this study is to create a surgical simulation and training model for aortic arch reconstruction. Description: Specialized segmentation software was used to isolate aortic arch anatomy from HLHS computed tomography scan images to create digital 3D models. Three-dimensional modeling software was used to modify the exported segmented models and digitally design printable customized patches that were optimally sized for arch reconstruction. Evaluation: Life-sized models of HLHS aortic arch anatomy and a digitally derived customized patch were 3D printed to allow simulation of surgical suturing and reconstruction. The patient-specific customized patch was successfully used for surgical simulation. Conclusions: Feasibility of digital design and 3D printing of patient-specific patches for aortic arch reconstruction has been demonstrated. The technology facilitates surgical simulation. Surgical training that leads to an understanding of optimal aortic patch geometry is one element that may potentially influence outcomes for patients with HLHS.

Download Full-text

Personalized Three-Dimensional Printed Models in Congenital Heart Disease

Journal of Clinical Medicine ◽

10.3390/jcm8040522 ◽

2019 ◽

Vol 8 (4) ◽

pp. 522 ◽

Cited By ~ 13

Author(s):

Sun ◽

Lau ◽

Wong ◽

Yeong

Keyword(s):

Congenital Heart Disease ◽

Heart Disease ◽

Congenital Heart ◽

Three Dimensional ◽

Patient Specific ◽

Future Research ◽

Imaging Data ◽

Doctor Patient Communication ◽

Medical Education And Training ◽

3D Printed

Patient-specific three-dimensional (3D) printed models have been increasingly used in cardiology and cardiac surgery, in particular, showing great value in the domain of congenital heart disease (CHD). CHD is characterized by complex cardiac anomalies with disease variations between individuals; thus, it is difficult to obtain comprehensive spatial conceptualization of the cardiac structures based on the current imaging visualizations. 3D printed models derived from patient’s cardiac imaging data overcome this limitation by creating personalized 3D heart models, which not only improve spatial visualization, but also assist preoperative planning and simulation of cardiac procedures, serve as a useful tool in medical education and training, and improve doctor–patient communication. This review article provides an overall view of the clinical applications and usefulness of 3D printed models in CHD. Current limitations and future research directions of 3D printed heart models are highlighted.

Download Full-text

Does the DSA reconstruction kernel affect hemodynamic predictions in intracranial aneurysms? An analysis of geometry and blood flow variations

Journal of NeuroInterventional Surgery ◽

10.1136/neurintsurg-2017-012996 ◽

2017 ◽

Vol 10 (3) ◽

pp. 290-296 ◽

Cited By ~ 19

Author(s):

P Berg ◽

S Saalfeld ◽

S Voß ◽

T Redel ◽

B Preim ◽

...

Keyword(s):

Blood Flow ◽

Intracranial Aneurysms ◽

Three Dimensional ◽

Patient Specific ◽

Parent Artery ◽

Imaging Data ◽

Specific Flow ◽

Aneurysm Neck ◽

Velocity Magnitude ◽

Reconstruction Kernel

BackgroundComputational fluid dynamics (CFD) blood flow predictions in intracranial aneurysms promise great potential to reveal patient-specific flow structures. Since the workflow from image acquisition to the final result includes various processing steps, quantifications of the individual introduced potential error sources are required.MethodsThree-dimensional (3D) reconstruction of the acquired imaging data as input to 3D model generation was evaluated. Six different reconstruction modes for 3D digital subtraction angiography (DSA) acquisitions were applied to eight patient-specific aneurysms. Segmentations were extracted to compare the 3D luminal surfaces. Time-dependent CFD simulations were carried out in all 48 configurations to assess the velocity and wall shear stress (WSS) variability due to the choice of reconstruction kernel.ResultsAll kernels yielded good segmentation agreement in the parent artery; deviations of the luminal surface were present at the aneurysm neck (up to 34.18%) and in distal or perforating arteries. Observations included pseudostenoses as well as noisy surfaces, depending on the selected reconstruction kernel. Consequently, the hemodynamic predictions show a mean SD of 11.09% for the aneurysm neck inflow rate, 5.07% for the centerline-based velocity magnitude, and 17.83%/9.53% for the mean/max aneurysmal WSS, respectively. In particular, vessel sections distal to the aneurysms yielded stronger variations of the CFD values.ConclusionsThe choice of reconstruction kernel for DSA data influences the segmentation result, especially for small arteries. Therefore, if precise morphology measurements or blood flow descriptions are desired, a specific reconstruction setting is required. Furthermore, research groups should be encouraged to denominate the kernel types used in future hemodynamic studies.

Download Full-text