scholarly journals TransMed: Transformers Advance Multi-Modal Medical Image Classification

Diagnostics ◽  
2021 ◽  
Vol 11 (8) ◽  
pp. 1384
Author(s):  
Yin Dai ◽  
Yifan Gao ◽  
Fayu Liu
Keyword(s):  
Image Analysis ◽  
Image Classification ◽  
Long Range ◽  
Medical Image ◽  
Large Scale ◽  
Medical Images ◽  
Medical Image Analysis ◽  
Lesion Detection ◽  
Tumor Segmentation ◽  
Medical Image Classification

Over the past decade, convolutional neural networks (CNN) have shown very competitive performance in medical image analysis tasks, such as disease classification, tumor segmentation, and lesion detection. CNN has great advantages in extracting local features of images. However, due to the locality of convolution operation, it cannot deal with long-range relationships well. Recently, transformers have been applied to computer vision and achieved remarkable success in large-scale datasets. Compared with natural images, multi-modal medical images have explicit and important long-range dependencies, and effective multi-modal fusion strategies can greatly improve the performance of deep models. This prompts us to study transformer-based structures and apply them to multi-modal medical images. Existing transformer-based network architectures require large-scale datasets to achieve better performance. However, medical imaging datasets are relatively small, which makes it difficult to apply pure transformers to medical image analysis. Therefore, we propose TransMed for multi-modal medical image classification. TransMed combines the advantages of CNN and transformer to efficiently extract low-level features of images and establish long-range dependencies between modalities. We evaluated our model on two datasets, parotid gland tumors classification and knee injury classification. Combining our contributions, we achieve an improvement of 10.1% and 1.9% in average accuracy, respectively, outperforming other state-of-the-art CNN-based models. The results of the proposed method are promising and have tremendous potential to be applied to a large number of medical image analysis tasks. To our best knowledge, this is the first work to apply transformers to multi-modal medical image classification.

scholarly journals Deep Learning for Brain Tumor Segmentation: A Survey of State-of-the-Art

2021 ◽  
Vol 7 (2) ◽  
pp. 19
Author(s):  
Tirivangani Magadza ◽  
Serestina Viriri
Keyword(s):  
Image Analysis ◽  
Deep Learning ◽  
Brain Tumor ◽  
Quantitative Analysis ◽  
Medical Image ◽  
State Of The Art ◽  
Medical Image Analysis ◽  
Building Blocks ◽  
Tumor Segmentation ◽  
Brain Tumor Segmentation

Quantitative analysis of the brain tumors provides valuable information for understanding the tumor characteristics and treatment planning better. The accurate segmentation of lesions requires more than one image modalities with varying contrasts. As a result, manual segmentation, which is arguably the most accurate segmentation method, would be impractical for more extensive studies. Deep learning has recently emerged as a solution for quantitative analysis due to its record-shattering performance. However, medical image analysis has its unique challenges. This paper presents a review of state-of-the-art deep learning methods for brain tumor segmentation, clearly highlighting their building blocks and various strategies. We end with a critical discussion of open challenges in medical image analysis.

scholarly journals Resolving Clinicians’ Queries Across a Grid’s Infrastructure

2005 ◽  
Vol 44 (02) ◽  
pp. 149-153 ◽  
Author(s):  
F. Estrella ◽  
C. del Frate ◽  
T. Hauer ◽  
M. Odeh ◽  
D. Rogulin ◽  
...  
Keyword(s):  
Image Analysis ◽  
Data Storage ◽  
Medical Image ◽  
Large Scale ◽  
Medical Image Analysis ◽  
Large Data ◽  
Computing Power ◽  
European Database ◽  
Order Of Magnitude ◽  
Network Speed

Summary Objectives: The past decade has witnessed order of magnitude increases in computing power, data storage capacity and network speed, giving birth to applications which may handle large data volumes of increased complexity, distributed over the internet. Methods: Medical image analysis is one of the areas for which this unique opportunity likely brings revolutionary advances both for the scientist’s research study and the clinician’s everyday work. Grids [1] computing promises to resolve many of the difficulties in facilitating medical image analysis to allow radiologists to collaborate without having to co-locate. Results: The EU-funded MammoGrid project [2] aims to investigate the feasibility of developing a Grid-enabled European database of mammograms and provide an information infrastructure which federates multiple mammogram databases. This will enable clinicians to develop new common, collaborative and co-operative approaches to the analysis of mammographic data. Conclusion: This paper focuses on one of the key requirements for large-scale distributed mammogram analysis: resolving queries across a grid-connected federation of images.

scholarly journals A Survey of Crowdsourcing in Medical Image Analysis

2020 ◽  
Vol 7 ◽  
pp. 1-26 ◽  
Author(s):  
Silas Nyboe Ørting ◽  
Andrew Doyle ◽  
Arno Van Hilten ◽  
Matthias Hirth ◽  
Oana Inel ◽  
...  
Keyword(s):  
Machine Learning ◽  
Image Processing ◽  
Image Analysis ◽  
Medical Image ◽  
Large Scale ◽  
Medical Image Analysis ◽  
Machine Learning Algorithms ◽  
Imaging Analysis ◽  
High Quality ◽  
Comprehensive Literature Review

Rapid advances in image processing capabilities have been seen across many domains, fostered by the  application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches, challenges and considerations, providing guidance of utility to researchers adopting this approach. Finally, we discuss future opportunities for development within this emerging domain.

scholarly journals Medical Image Classification Algorithm Based on Visual Attention Mechanism-MCNN

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Fengping An ◽  
Xiaowei Li ◽  
Xingmin Ma
Keyword(s):  
Deep Learning ◽  
Visual Attention ◽  
Image Classification ◽  
Medical Image ◽  
Medical Images ◽  
Attention Mechanism ◽  
Training Strategy ◽  
Medical Image Classification ◽  
Deep Learning Network ◽  
Visual Attention Mechanism

Due to the complexity of medical images, traditional medical image classification methods have been unable to meet the actual application needs. In recent years, the rapid development of deep learning theory has provided a technical approach for solving medical image classification. However, deep learning has the following problems in the application of medical image classification. First, it is impossible to construct a deep learning model with excellent performance according to the characteristics of medical images. Second, the current deep learning network structure and training strategies are less adaptable to medical images. Therefore, this paper first introduces the visual attention mechanism into the deep learning model so that the information can be extracted more effectively according to the problem of medical images, and the reasoning is realized at a finer granularity. It can increase the interpretability of the model. Additionally, to solve the problem of matching the deep learning network structure and training strategy to medical images, this paper will construct a novel multiscale convolutional neural network model that can automatically extract high-level discriminative appearance features from the original image, and the loss function uses the Mahalanobis distance optimization model to obtain a better training strategy, which can improve the robust performance of the network model. The medical image classification task is completed by the above method. Based on the above ideas, this paper proposes a medical classification algorithm based on a visual attention mechanism-multiscale convolutional neural network. The lung nodules and breast cancer images were classified by the method in this paper. The experimental results show that the accuracy of medical image classification in this paper is not only higher than that of traditional machine learning methods but also improved compared with other deep learning methods, and the method has good stability and robustness.

Artificial Intelligence and Big Data

2020 ◽  
Vol 237 (12) ◽  
pp. 1438-1441
Author(s):  
Soenke Langner ◽  
Ebba Beller ◽  
Felix Streckenbach
Keyword(s):  
Artificial Intelligence ◽  
Image Analysis ◽  
Big Data ◽  
Deep Learning ◽  
Medical Image ◽  
Medical Images ◽  
Medical Image Analysis ◽  
Clinical Applications ◽  
Radiological Imaging ◽  
Learning Techniques

AbstractMedical images play an important role in ophthalmology and radiology. Medical image analysis has greatly benefited from the application of “deep learning” techniques in clinical and experimental radiology. Clinical applications and their relevance for radiological imaging in ophthalmology are presented.

scholarly journals Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

2021 ◽  
Vol 11 (9) ◽  
pp. 4247
Author(s):  
Minh-Trieu Tran ◽  
Soo-Hyung Kim ◽  
Hyung-Jeong Yang ◽  
Guee-Sang Lee
Keyword(s):  
Image Analysis ◽  
Medical Diagnosis ◽  
Medical Image ◽  
Medical Images ◽  
Medical Image Analysis ◽  
Image Inpainting ◽  
Ct Images ◽  
Task Learning ◽  
Specular Reflections ◽  
Simultaneous Training

Distorted medical images can significantly hamper medical diagnosis, notably in the analysis of Computer Tomography (CT) images and organ segmentation specifics. Therefore, improving diagnostic imagery accuracy and reconstructing damaged portions are important for medical diagnosis. Recently, these issues have been studied extensively in the field of medical image inpainting. Inpainting techniques are emerging in medical image analysis since local deformations in medical modalities are common because of various factors such as metallic implants, foreign objects or specular reflections during the image captures. The completion of such missing or distorted regions is important for the enhancement of post-processing tasks such as segmentation or classification. In this paper, a novel framework for medical image inpainting is presented by using a multi-task learning model for CT images targeting the learning of the shape and structure of the organs of interest. This novelty has been accomplished through simultaneous training for the prediction of edges and organ boundaries with the image inpainting, while state-of-the-art methods still focus only on the inpainting area without considering the global structure of the target organ. Therefore, our model reproduces medical images with sharp contours and exact organ locations. Consequently, our technique generates more realistic and believable images compared to other approaches. Additionally, in quantitative evaluation, the proposed method achieved the best results in the literature so far, which include a PSNR value of 43.44 dB and SSIM of 0.9818 for the square-shaped regions; a PSNR value of 38.06 dB and SSIM of 0.9746 for the arbitrary-shaped regions. The proposed model generates the sharp and clear images for inpainting by learning the detailed structure of organs. Our method was able to show how promising the method is when applying it in medical image analysis, where the completion of missing or distorted regions is still a challenging task.

Sign in / Sign up

Export Citation Format

Share Document