Unsupervised Deep Learning Network with Self-Attention Mechanism for Non-Rigid Registration of 3D Brain MR Images

2021 ◽  
Vol 11 (3) ◽  
pp. 736-751
Author(s):  
Donggeon Oh ◽  
Bohyoung Kim ◽  
Jeongjin Lee ◽  
Yeong-Gil Shin
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Unsupervised Learning ◽  
High Accuracy ◽  
Magnetic Resonance Images ◽  
Attention Mechanism ◽  
Learning Networks ◽  
Rigid Registration ◽  
Learning Network ◽  
The Brain

In non-rigid registration for medical imaging analysis, computation is complicated, and the high accuracy and robustness needed for registration are difficult to obtain. Recently, many studies have been conducted for nonrigid registration via unsupervised learning networks. This study proposes a method to improve the performance of this unsupervised learning network approach, through the use of a self-attention mechanism. In this paper, the self-attention mechanism is combined with deep learning networks to identify information of higher importance, among large amounts of data, and thereby solve specific tasks. Furthermore, the proposed method extracts both local and non-local information so that the network can create feature vectors with more information. As a result, the limitation of the existing network is addressed: alignment based solely on the entire silhouette of the brain is mitigated in favor of a network which also learns to perform registration of the parts of the brain that have internal structural characteristics. To the best of our knowledge, this is the first such utilization of the attention mechanism in this unsupervised learning network for non-rigid registration. The proposed attention network performs registration that takes into account the overall characteristics of the data, thus yielding more accurate matching results than those of the existing methods. In particular, matching is achieved with especially high accuracy in the gray matter and cortical ventricle areas, since these areas contain many of the structural features of the brain. The experiment was performed on 3D magnetic resonance images of the brains of 50 people. The measured average dice similarity coefficient after registration was 70.40%, which is an improvement of 17.48% compared to that before registration. This improvement indicates that application of the attention block can further improve the performance by an additional 8.5%, as relative to that without attention block. Ultimately, through implementation of non-rigid registration via the attention block method, the internal structure and overall shape of the brain can be addressed, without additional data input. Additionally, attention blocks have the advantage of being able to easily connect to existing networks without a significant computational overhead. Furthermore, by producing an attention map, the area of the brain around which registration was more performed can be visualized. This approach can be used for non-rigid registration with various types of medical imaging data.

scholarly journals UAVs in rail damage image diagnostics supported by deep-learning networks

2021 ◽  
Vol 11 (1) ◽  
pp. 339-348
Author(s):  
Piotr Bojarczak ◽  
Piotr Lesiak
Keyword(s):  
Deep Learning ◽  
Head Width ◽  
Learning Networks ◽  
Image Brightness ◽  
Learning Network ◽  
Rail Head ◽  
Image Recording ◽  
Python Language ◽  
Deep Learning Network ◽  
Efficiency Rate

Abstract The article uses images from Unmanned Aerial Vehicles (UAVs) for rail diagnostics. The main advantage of such a solution compared to traditional surveys performed with measuring vehicles is the elimination of decreased train traffic. The authors, in the study, limited themselves to the diagnosis of hazardous split defects in rails. An algorithm has been proposed to detect them with an efficiency rate of about 81% for defects not less than 6.9% of the rail head width. It uses the FCN-8 deep-learning network, implemented in the Tensorflow environment, to extract the rail head by image segmentation. Using this type of network for segmentation increases the resistance of the algorithm to changes in the recorded rail image brightness. This is of fundamental importance in the case of variable conditions for image recording by UAVs. The detection of these defects in the rail head is performed using an algorithm in the Python language and the OpenCV library. To locate the defect, it uses the contour of a separate rail head together with a rectangle circumscribed around it. The use of UAVs together with artificial intelligence to detect split defects is an important element of novelty presented in this work.

scholarly journals Application and Construction of Deep Learning Networks in Medical Imaging

2020 ◽  
pp. 1-1 ◽  
Author(s):  
Maribel Torres-Velazquez ◽  
Wei-Jie Chen ◽  
Xue Li ◽  
Alan B. McMillan
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Learning Networks

scholarly journals Attentional Reinforcement Learning in the Brain

2020 ◽  
Vol 38 (1) ◽  
pp. 49-64 ◽  
Author(s):  
Hiroshi Yamakawa
Keyword(s):  
Deep Learning ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Language Processing ◽  
Information Source ◽  
Attention Mechanism ◽  
Transmission Route ◽  
Thalamic Relay ◽  
Signal Changes ◽  
The Brain

AbstractRecently, attention mechanisms have significantly boosted the performance of natural language processing using deep learning. An attention mechanism can select the information to be used, such as by conducting a dictionary lookup; this information is then used, for example, to select the next utterance word in a sentence. In neuroscience, the basis of the function of sequentially selecting words is considered to be the cortico-basal ganglia-thalamocortical loop. Here, we first show that the attention mechanism used in deep learning corresponds to the mechanism in which the cerebral basal ganglia suppress thalamic relay cells in the brain. Next, we demonstrate that, in neuroscience, the output of the basal ganglia is associated with the action output in the actor of reinforcement learning. Based on these, we show that the aforementioned loop can be generalized as reinforcement learning that controls the transmission of the prediction signal so as to maximize the prediction reward. We call this attentional reinforcement learning (ARL). In ARL, the actor selects the information transmission route according to the attention, and the prediction signal changes according to the context detected by the information source of the route. Hence, ARL enables flexible action selection that depends on the situation, unlike traditional reinforcement learning, wherein the actor must directly select an action.

scholarly journals Deep Learning for Accurate Segmentation of Venous Thrombus from Black-Blood Magnetic Resonance Images: A Multicenter Study

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Chuanqi Sun ◽  
Xiangyu Xiong ◽  
Tianjing Zhang ◽  
Xiuhong Guan ◽  
Huan Mao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Magnetic Resonance ◽  
Ground Truth ◽  
Magnetic Resonance Images ◽  
Black Blood ◽  
Mr Images ◽  
Vessel Segment ◽  
Learning Network ◽  
Venous Thrombus ◽  
Deep Learning Network

Objective. Deep vein thrombosis (DVT) is the third-largest cardiovascular disease, and accurate segmentation of venous thrombus from the black-blood magnetic resonance (MR) images can provide additional information for personalized DVT treatment planning. Therefore, a deep learning network is proposed to automatically segment venous thrombus with high accuracy and reliability. Methods. In order to train, test, and external test the developed network, total images of 110 subjects are obtained from three different centers with two different black-blood MR techniques (i.e., DANTE-SPACE and DANTE-FLASH). Two experienced radiologists manually contoured each venous thrombus, followed by reediting, to create the ground truth. 5-fold cross-validation strategy is applied for training and testing. The segmentation performance is measured on pixel and vessel segment levels. For the pixel level, the dice similarity coefficient (DSC), average Hausdorff distance (AHD), and absolute volume difference (AVD) of segmented thrombus are calculated. For the vessel segment level, the sensitivity (SE), specificity (SP), accuracy (ACC), and positive and negative predictive values (PPV and NPV) are used. Results. The proposed network generates segmentation results in good agreement with the ground truth. Based on the pixel level, the proposed network achieves excellent results on testing and the other two external testing sets, DSC are 0.76, 0.76, and 0.73, AHD (mm) are 4.11, 6.45, and 6.49, and AVD are 0.16, 0.18, and 0.22. On the vessel segment level, SE are 0.95, 0.93, and 0.81, SP are 0.97, 0.92, and 0.97, ACC are 0.96, 0.94, and 0.95, PPV are 0.97, 0.82, and 0.96, and NPV are 0.97, 0.96, and 0.94. Conclusions. The proposed deep learning network is effective and stable for fully automatic segmentation of venous thrombus on black blood MR images.

scholarly journals Recognization and Systematization of MR Images using K Means Clustering and DNN

2020 ◽  
Vol 9 (6) ◽  
pp. 924-927
Keyword(s):  
Deep Learning ◽  
Brain Tumor ◽  
Brain Tumors ◽  
Digital Health ◽  
High Accuracy ◽  
Tumor Segmentation ◽  
Mr Images ◽  
Brain Tumor Segmentation ◽  
Brain Tumor Diagnosis ◽  
The Brain

Brain tumors are the result of unusual growth and unrestrained cell disunity in the brain. Most of the medical image application lack in segmentation and labeling. Brain tumors can lead to loss of lives if they are not detected early and correctly. Recently, deep learning has been an important role in the field of digital health. One of its action is the reduction of manual decision in the diagnosis of diseases specifically brain tumor diagnosis needs high accuracy, where minute errors in judgment may lead to loss therefore, brain tumor segmentation is an necessary challenge in medical side. In recent time numerous ,methods exist for tumor segmentation with lack of accuracy. Deep learning is used to achieve the goal of brain tumor segmentation. In this work, three network of brain MR images segmentation is employed .A single network is compared to achieve segmentation of MR images using separate network .In this paper segmentation has improved and result is obtained with high accuracy and efficiency.

scholarly journals A New Deep Learning Network for Automatic Bridge Detection from SAR Images Based on Balanced and Attention Mechanism

2020 ◽  
Vol 12 (3) ◽  
pp. 441
Author(s):  
Lifu Chen ◽  
Ting Weng ◽  
Jin Xing ◽  
Zhouhao Pan ◽  
Zhihui Yuan ◽  
...  
Keyword(s):  
Deep Learning ◽  
Region Of Interest ◽  
Recall Rate ◽  
Attention Mechanism ◽  
Sar Images ◽  
Learning Network ◽  
Feature Pyramid ◽  
Pooling Strategy ◽  
Classification And Regression ◽  
Deep Learning Network

Bridge detection from Synthetic Aperture Radar (SAR) images has very important strategic significance and practical value, but there are still many challenges in end-to-end bridge detection. In this paper, a new deep learning-based network is proposed to identify bridges from SAR images, namely, multi-resolution attention and balance network (MABN). It mainly includes three parts, the attention and balanced feature pyramid (ABFP) network, the region proposal network (RPN), and the classification and regression. First, the ABFP network extracts various features from SAR images, which integrates the ResNeXt backbone network, balanced feature pyramid, and the attention mechanism. Second, extracted features are used by RPN to generate candidate boxes of different resolutions and fused. Furthermore, the candidate boxes are combined with the features extracted by the ABFP network through the region of interest (ROI) pooling strategy. Finally, the detection results of the bridges are produced by the classification and regression module. In addition, intersection over union (IOU) balanced sampling and balanced L1 loss functions are introduced for optimal training of the classification and regression network. In the experiment, TerraSAR data with 3-m resolution and Gaofen-3 data with 1-m resolution are used, and the results are compared with faster R-CNN and SSD. The proposed network has achieved the highest detection precision (P) and average precision (AP) among the three networks, as 0.877 and 0.896, respectively, with the recall rate (RR) as 0.917. Compared with the other two networks, the false alarm targets and missed targets of the proposed network in this paper are greatly reduced, so the precision is greatly improved.

scholarly journals Impact of Deep Learning in Medical Imaging: a Systematic New Proposed Model

2019 ◽  
Vol 8 (3S2) ◽  
pp. 112-118
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Learning Algorithms ◽  
Brain Lesion ◽  
Research Area ◽  
Magnetic Resonance Images ◽  
Machine Learning Algorithms ◽  
Proposed Model ◽  
Active Research ◽  
Active Research Area

For years’ radiologist and clinician continues to employs various approaches, machine learning algorithms included to detect, diagnose, and prevent diseases using medical imaging. Recent advances in deep learning made medical imaging analysis and processing an active research area, various algorithms for segmentation, detection, and classification have been proposed. In this survey, we describe the trends of deep learning algorithms use in medical imaging, their architecture, hardware, and software used are all discussed. We concluded with the proposed model for brain lesion segmentation and classification using Magnetic Resonance Images (MRI).

scholarly journals MRI image segmentation using machine learning networks and level set approaches

2022 ◽  
Vol 12 (1) ◽  
pp. 793
Author(s):  
Layth Kamil Adday Almajmaie ◽  
Ahmed Raad Raheem ◽  
Wisam Ali Mahmood ◽  
Saad Albawi
Keyword(s):  
Deep Learning ◽  
Brain Tumour ◽  
Level Set ◽  
Magnetic Resonance Images ◽  
Tumor Segmentation ◽  
Learning Networks ◽  
Brain Tumor Segmentation ◽  
Brain Images ◽  
Accuracy And Precision ◽  
Deep Learning Network

<span>The segmented brain tissues from magnetic resonance images (MRI) always pose substantive challenges to the clinical researcher community, especially while making precise estimation of such tissues. In the recent years, advancements in deep learning techniques, more specifically in fully convolution neural networks (FCN) have yielded path breaking results in segmenting brain tumour tissues with pin-point accuracy and precision, much to the relief of clinical physicians and researchers alike. A new hybrid deep learning architecture combining SegNet and U-Net techniques to segment brain tissue is proposed here. Here, a skip connection of the concerned U-Net network was suitably explored. The results indicated optimal multi-scale information generated from the SegNet, which was further exploited to obtain precise tissue boundaries from the brain images. Further, in order to ensure that the segmentation method performed better in conjunction with precisely delineated contours, the output is incorporated as the level set layer in the deep learning network. The proposed method primarily focused on analysing brain tumor segmentation (BraTS) 2017 and BraTS 2018, dedicated datasets dealing with MRI brain tumour. The results clearly indicate better performance in segmenting brain tumours than existing ones.</span>

scholarly journals Cross-Modal Guidance Assisted Hierarchical Learning Based Siamese Network for MR Image Denoising

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2855
Author(s):  
Rabia Naseem ◽  
Faouzi Alaya Cheikh ◽  
Azeddine Beghdadi ◽  
Khan Muhammad ◽  
Muhammad Sajjad
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Medical Image Analysis ◽  
Similarity Index ◽  
Structural Similarity ◽  
Magnetic Resonance Images ◽  
Perceptual Quality ◽  
Hierarchical Learning ◽  
Siamese Network ◽  
The Impact

Cross-modal medical imaging techniques are predominantly being used in the clinical suite. The ensemble learning methods using cross-modal medical imaging adds reliability to several medical image analysis tasks. Motivated by the performance of deep learning in several medical imaging tasks, a deep learning-based denoising method Cross-Modality Guided Denoising Network CMGDNet for removing Rician noise in T1-weighted (T1-w) Magnetic Resonance Images (MRI) is proposed in this paper. CMGDNet uses a guidance image, which is a cross-modal (T2-w) image of better perceptual quality to guide the model in denoising its noisy T1-w counterpart. This cross-modal combination allows the network to exploit complementary information existing in both images and therefore improve the learning capability of the model. The proposed framework consists of two components: Paired Hierarchical Learning (PHL) module and Cross-Modal Assisted Reconstruction (CMAR) module. PHL module uses Siamese network to extract hierarchical features from dual images, which are then combined in a densely connected manner in the CMAR module to finally reconstruct the image. The impact of using registered guidance data is investigated in removing noise as well as retaining structural similarity with the original image. Several experiments were conducted on two publicly available brain imaging datasets available on the IXI database. The quantitative assessment using Peak Signal to noise ratio (PSNR), Structural Similarity Index (SSIM), and Feature Similarity Index (FSIM) demonstrates that the proposed method exhibits 4.7% and 2.3% gain (average), respectively, in SSIM and FSIM values compared to other state-of-the-art denoising methods that do not integrate cross-modal image information in removing various levels of noise.

Sign in / Sign up

Export Citation Format

Share Document