scholarly journals Semi-Supervised Deep Learning-Based Image Registration Method with Volume Penalty for Real-Time Breast Tumor Bed Localization

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4085
Author(s):  
Marek Wodzinski ◽  
Izabela Ciepiela ◽  
Tomasz Kuszewski ◽  
Piotr Kedzierawski ◽  
Andrzej Skalski
Keyword(s):  
Deep Learning ◽  
Image Registration ◽  
Real Time ◽  
Breast Tumor ◽  
Tumor Resection ◽  
Registration Method ◽  
Registration Error ◽  
Target Registration Error ◽  
Tumor Bed ◽  
Registration Process

Breast-conserving surgery requires supportive radiotherapy to prevent cancer recurrence. However, the task of localizing the tumor bed to be irradiated is not trivial. The automatic image registration could significantly aid the tumor bed localization and lower the radiation dose delivered to the surrounding healthy tissues. This study proposes a novel image registration method dedicated to breast tumor bed localization addressing the problem of missing data due to tumor resection that may be applied to real-time radiotherapy planning. We propose a deep learning-based nonrigid image registration method based on a modified U-Net architecture. The algorithm works simultaneously on several image resolutions to handle large deformations. Moreover, we propose a dedicated volume penalty that introduces the medical knowledge about tumor resection into the registration process. The proposed method may be useful for improving real-time radiation therapy planning after the tumor resection and, thus, lower the surrounding healthy tissues’ irradiation. The data used in this study consist of 30 computed tomography scans acquired in patients with diagnosed breast cancer, before and after tumor surgery. The method is evaluated using the target registration error between manually annotated landmarks, the ratio of tumor volume, and the subjective visual assessment. We compare the proposed method to several other approaches and show that both the multilevel approach and the volume regularization improve the registration results. The mean target registration error is below 6.5 mm, and the relative volume ratio is close to zero. The registration time below 1 s enables the real-time processing. These results show improvements compared to the classical, iterative methods or other learning-based approaches that do not introduce the knowledge about tumor resection into the registration process. In future research, we plan to propose a method dedicated to automatic localization of missing regions that may be used to automatically segment tumors in the source image and scars in the target image.

scholarly journals GroupRegNet: a groupwise one-shot deep learning-based 4D image registration method

2021 ◽  
Author(s):  
Yunlu Zhang ◽  
Xue Wu ◽  
H Michael Gach ◽  
H Harold Li ◽  
Deshan Yang
Keyword(s):  
Deep Learning ◽  
Image Registration ◽  
Registration Method

scholarly journals PathFlow-MixMatch for Whole Slide Image Registration: An Investigation of a Segment-Based Scalable Image Registration Method

2020 ◽  
Author(s):  
Joshua J. Levy ◽  
Christopher R. Jackson ◽  
Christian C. Haudenschild ◽  
Brock C. Christensen ◽  
Louis J. Vaickus
Keyword(s):  
Image Registration ◽  
Large Scale ◽  
Digital Pathology ◽  
Superior Performance ◽  
Registration Method ◽  
Target Registration Error ◽  
Alignment Algorithms ◽  
Slide Image ◽  
Segment Matching ◽  
The Impact

AbstractImage registration involves finding the best alignment between different images of the same object. In these tasks, the object in question is viewed differently in each of the images (e.g. different rotation or light conditions, etc.). In digital pathology, image registration aligns correspondent regions of tissue from different stereotactic viewpoints (e.g. subsequent deeper sections of the same tissue). These comparisons are important for histological analysis and can facilitate previously unavailable manipulations, such as 3D tissue reconstruction and cell-level alignment of immunohistochemical (IHC) and special stains. Several benchmarks have been established for evaluating image registration techniques for histological tissue; however, little work has evaluated the impact of scaling registration techniques to Giga-Pixel Whole Slide Images (WSI), which are large enough for significant memory limitations, and contain recurrent patterns and deformations that hinder traditional alignment algorithms. Furthermore, as tissue sections often contain multiple, discrete, smaller tissue fragments, it is unnecessary to align an entire image when the bulk of the image is background whitespace and tissue fragments’ orientations are often agnostic of each other. We present a methodology for circumventing large-scale image registration issues in histopathology and accompanying software. By removing background pixels, parsing the slide into discrete tissue segments, and matching, orienting and registering smaller segment pairs, we recovered registrations with lower Target Registration Error (TRE) when compared to utilizing the unmanipulated WSI. We tested our technique by having a pathologist annotate landmarks from 13 pairs of differently stained liver biopsy slides, performing WSI and segment-based registration techniques, and comparing overall TRE. Preliminary results demonstrate superior performance of registering segment pairs versus registering WSI (difference of median TRE of 44 pixels, p<0.001). Segment matching within WSI is an effective solution for histology image registration but requires further testing and validation to ensure its viability for stain translation and 3D histology analysis.

scholarly journals Grey-Wolf-Based Wang’s Demons for Retinal Image Registration

Entropy ◽  
2020 ◽  
Vol 22 (6) ◽  
pp. 659 ◽  
Author(s):  
Sayan Chakraborty ◽  
Ratika Pradhan ◽  
Amira S. Ashour ◽  
Luminita Moraru ◽  
Nilanjan Dey
Keyword(s):  
Image Registration ◽  
Retinal Image ◽  
Firefly Algorithm ◽  
Cuckoo Search ◽  
Grey Wolf Optimizer ◽  
Grey Wolf ◽  
Swarm Optimization ◽  
Registration Error ◽  
Registration Process ◽  
Fast Processing

Image registration has an imperative role in medical imaging. In this work, a grey-wolf optimizer (GWO)-based non-rigid demons registration is proposed to support the retinal image registration process. A comparative study of the proposed GWO-based demons registration framework with cuckoo search, firefly algorithm, and particle swarm optimization-based demons registration is conducted. In addition, a comparative analysis of different demons registration methods, such as Wang’s demons, Tang’s demons, and Thirion’s demons which are optimized using the proposed GWO is carried out. The results established the superiority of the GWO-based framework which achieved 0.9977 correlation, and fast processing compared to the use of the other optimization algorithms. Moreover, GWO-based Wang’s demons performed better accuracy compared to the Tang’s demons and Thirion’s demons framework. It also achieved the best less registration error of 8.36 × 10−5.

scholarly journals Grey Wolf Based Wang’s Demons for Retinal Image Registration

2020 ◽  
Author(s):  
Sayan Chakraborty ◽  
Ratika Pradhan ◽  
Amira S. Ashour ◽  
Luminita Moraru ◽  
Nilanjan Dey
Keyword(s):  
Image Registration ◽  
Retinal Image ◽  
Firefly Algorithm ◽  
Cuckoo Search ◽  
Grey Wolf Optimizer ◽  
Grey Wolf ◽  
Swarm Optimization ◽  
Registration Error ◽  
Registration Process ◽  
Fast Processing

Image registration has an imperative role in medical imaging. In this work, a grey-wolf optimizer (GWO) based non-rigid demons registration is proposed to support the retinal image registration process. A comparative study of the proposed GWO-based demons registration framework with cuckoo search, firefly algorithm, and particle swarm optimization- based demons registration is conducted. In addition, a comparative analysis of different demons registration methods, such as Wang&rsquo;s demons, Tang&rsquo;s demons, and Thirion&rsquo;s demons which are optimized using the proposed GWO is carried out. The results established the superiority of the GWO-based framework which achieved 0.9977 correlation, and fast processing compared to the use of the other optimization algorithms. Moreover, GWO-based Wang&rsquo;s demons performed better accuracy compared to the Tang&rsquo;s demons and Thirion&rsquo;s demons framework. It also achieved the best less registration error of 8.36&times;10-5.

scholarly journals Photoacoustic-MR Images Registration Based on Co-Sparse Analysis Model to Compensate for Brain Shift

2021 ◽  
Author(s):  
Parastoo Farnia ◽  
Bahador Makkiabadi ◽  
Meysam Alimohammadi ◽  
Ebrahim Najafzadeh ◽  
Maryam Basij ◽  
...  
Keyword(s):  
Image Registration ◽  
Real Time ◽  
Mouse Brain ◽  
Brain Shift ◽  
Challenging Problem ◽  
Analysis Model ◽  
Mr Images ◽  
Registration Method ◽  
Phantom Data ◽  
Brain Deformation

Brain shift is an important obstacle for the application of image guidance during neurosurgical interventions. There has been a growing interest in intra-operative imaging systems to update the image-guided surgery systems with real-time data. However, due to the innate limitations of the current imaging modalities, accurate and real-time brain shift compensation remains as a challenging problem. In this study, application of the intra-operative photoacoustic (PA) imaging and registration of the intra-operative PA images with pre-operative brain MR images is proposed to compensate brain deformation during surgery. Finding a satisfactory multimodal image registration method is a challenging problem due to complicated and unpredictable nature of brain deformation. In this study, the co-sparse analysis model is proposed for PA-MR image registration which can capture the interdependency of two modalities. The proposed algorithm works based on the minimization of mapping transform by using a pair of analysis operators. These operators are learned by the alternating direction method of multipliers. The method was evaluated using experimental phantom and ex-vivo data obtained from mouse brain. The results of phantom data show about 60% and 63% improvement in root mean square error (RMSE) and target registration error (TRE) in comparison with commonly used normalized mutual information registration method. In addition, the results of mouse brain and phantom data shown more accurate performance for PA versus ultrasound imaging for brain shift calculation. Finally, by using the proposed registration method, the intra-operative PA images could become a promising tool when the brain shift invalidated pre-operative MRI.

scholarly journals Frameless neuronavigation with computer vision and real-time tracking for bedside external ventricular drain placement: a cadaveric study

2021 ◽  
pp. 1-10
Author(s):  
Faith C. Robertson ◽  
Raahil M. Sha ◽  
Jose M. Amich ◽  
Walid Ibn Essayed ◽  
Avinash Lal ◽  
...  
Keyword(s):  
Computer Vision ◽  
Image Registration ◽  
Real Time ◽  
Tracking System ◽  
External Ventricular Drain ◽  
Surface Registration ◽  
Ct Scans ◽  
Registration System ◽  
Registration Error ◽  
Real Time Tracking

OBJECTIVE A major obstacle to improving bedside neurosurgical procedure safety and accuracy with image guidance technologies is the lack of a rapidly deployable, real-time registration and tracking system for a moving patient. This deficiency explains the persistence of freehand placement of external ventricular drains, which has an inherent risk of inaccurate positioning, multiple passes, tract hemorrhage, and injury to adjacent brain parenchyma. Here, the authors introduce and validate a novel image registration and real-time tracking system for frameless stereotactic neuronavigation and catheter placement in the nonimmobilized patient. METHODS Computer vision technology was used to develop an algorithm that performed near-continuous, automatic, and marker-less image registration. The program fuses a subject’s preprocedure CT scans to live 3D camera images (Snap-Surface), and patient movement is incorporated by artificial intelligence–driven recalibration (Real-Track). The surface registration error (SRE) and target registration error (TRE) were calculated for 5 cadaveric heads that underwent serial movements (fast and slow velocity roll, pitch, and yaw motions) and several test conditions, such as surgical draping with limited anatomical exposure and differential subject lighting. Six catheters were placed in each cadaveric head (30 total placements) with a simulated sterile technique. Postprocedure CT scans allowed comparison of planned and actual catheter positions for user error calculation. RESULTS Registration was successful for all 5 cadaveric specimens, with an overall mean (± standard deviation) SRE of 0.429 ± 0.108 mm for the catheter placements. Accuracy of TRE was maintained under 1.2 mm throughout specimen movements of low and high velocities of roll, pitch, and yaw, with the slowest recalibration time of 0.23 seconds. There were no statistically significant differences in SRE when the specimens were draped or fully undraped (p = 0.336). Performing registration in a bright versus a dimly lit environment had no statistically significant effect on SRE (p = 0.742 and 0.859, respectively). For the catheter placements, mean TRE was 0.862 ± 0.322 mm and mean user error (difference between target and actual catheter tip) was 1.674 ± 1.195 mm. CONCLUSIONS This computer vision–based registration system provided real-time tracking of cadaveric heads with a recalibration time of less than one-quarter of a second with submillimetric accuracy and enabled catheter placements with millimetric accuracy. Using this approach to guide bedside ventriculostomy could reduce complications, improve safety, and be extrapolated to other frameless stereotactic applications in awake, nonimmobilized patients.

An Underwater Image Real-Time Registration Approach Based on SURF

2013 ◽  
Vol 437 ◽  
pp. 888-893 ◽  
Author(s):  
Chao Li ◽  
Yong Jie Pang ◽  
Ming Wei Sheng ◽  
Hai Huang
Keyword(s):  
Image Registration ◽  
Real Time ◽  
Median Filter ◽  
Filter Method ◽  
Interest Points ◽  
Registration Method ◽  
Speeded Up Robust Features ◽  
Time Performance ◽  
Underwater Image ◽  
Image Contrast Enhancement

In order to meet the demands of real-time performance and robustness for underwater image registration, a novel image registration method based on the SURF (Speeded-Up Robust Features) algorithm is proposed. During the image acquisition process, noise was generated inevitably because of many influencing factors such as atmospheric turbulence, camera defocus during image capturing or relative motion between the camera and the object. Firstly, median filter method was involved during the image preprocessing for underwater image contrast enhancement. Secondly, the SURF algorithm was used to obtain the interest points of the reference and registering images, and the nearest neighbor method was applied to search for coarse matching points. To obtain the precise matching points, the dominant orientations of the coarse matching points were used to eliminate the mismatching points. Finally, the precise matching points were adapted to calculate the mapping relationship between the registering and reference images, the bilinear interpolation method was applied to resample the registering image, and then the registered image was obtained. Experimental results indicated that the proposed preprocessing methods obviously enhanced the image quality, and the introduced image registration approach effectively improved the real-time performance and guaranteed the robustness at the same time.

Sign in / Sign up

Export Citation Format

Share Document