Transformer Based Multi-model Fusion for Medical Image Segmentation

We present our solutions to the MedAI for all three tasks: polyp segmentation task, instrument segmentation task, and transparency task. We use the same framework to process the two segmentation tasks of polyps and instruments. The key improvement over last year is new state-of-the-art vision architectures, especially transformers which significantly outperform ConvNets for the medical image segmentation tasks. Our solution consists of multiple segmentation models, and each model uses a transformer as the backbone network. we get the best IoU score of 0.915 on the instrument segmentation task and 0.836 on polyp segmentation task after submitting. Meanwhile, we provide complete solutions in https://github.com/dongbo811/MedAI-2021.

Download Full-text

Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.3301865 ◽

2019 ◽

Vol 33 ◽

pp. 865-872 ◽

Cited By ~ 27

Author(s):

Cheng Chen ◽

Qi Dou ◽

Hao Chen ◽

Jing Qin ◽

Pheng-Ann Heng

Keyword(s):

Image Segmentation ◽

Medical Image ◽

Domain Adaptation ◽

State Of The Art ◽

Medical Image Segmentation ◽

Target Domain ◽

Learning Procedure ◽

The Neural Networks ◽

Feature Adaptation ◽

Segmentation Task

This paper presents a novel unsupervised domain adaptation framework, called Synergistic Image and Feature Adaptation (SIFA), to effectively tackle the problem of domain shift. Domain adaptation has become an important and hot topic in recent studies on deep learning, aiming to recover performance degradation when applying the neural networks to new testing domains. Our proposed SIFA is an elegant learning diagram which presents synergistic fusion of adaptations from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features towards the segmentation task. The feature encoder layers are shared by both perspectives to grasp their mutual benefits during the end-to-end learning procedure. Without using any annotation from the target domain, the learning of our unified model is guided by adversarial losses, with multiple discriminators employed from various aspects. We have extensively validated our method with a challenging application of crossmodality medical image segmentation of cardiac structures. Experimental results demonstrate that our SIFA model recovers the degraded performance from 17.2% to 73.0%, and outperforms the state-of-the-art methods by a significant margin.

Download Full-text

An Annotation Sparsification Strategy for 3D Medical Image Segmentation via Representative Selection and Self-Training

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6175 ◽

2020 ◽

Vol 34 (04) ◽

pp. 6925-6932 ◽

Cited By ~ 1

Author(s):

Hao Zheng ◽

Yizhe Zhang ◽

Lin Yang ◽

Chaoli Wang ◽

Danny Z. Chen

Keyword(s):

Image Segmentation ◽

Medical Image ◽

Piriform Cortex ◽

Medical Image Segmentation ◽

Improve Performance ◽

3D Images ◽

Data Annotation ◽

Representative Selection ◽

Public Datasets ◽

Segmentation Models

Image segmentation is critical to lots of medical applications. While deep learning (DL) methods continue to improve performance for many medical image segmentation tasks, data annotation is a big bottleneck to DL-based segmentation because (1) DL models tend to need a large amount of labeled data to train, and (2) it is highly time-consuming and label-intensive to voxel-wise label 3D medical images. Significantly reducing annotation effort while attaining good performance of DL segmentation models remains a major challenge. In our preliminary experiments, we observe that, using partially labeled datasets, there is indeed a large performance gap with respect to using fully annotated training datasets. In this paper, we propose a new DL framework for reducing annotation effort and bridging the gap between full annotation and sparse annotation in 3D medical image segmentation. We achieve this by (i) selecting representative slices in 3D images that minimize data redundancy and save annotation effort, and (ii) self-training with pseudo-labels automatically generated from the base-models trained using the selected annotated slices. Extensive experiments using two public datasets (the HVSMR 2016 Challenge dataset and mouse piriform cortex dataset) show that our framework yields competitive segmentation results comparing with state-of-the-art DL methods using less than ∼20% of annotated data.

Download Full-text

Render U-Net: A Unique Perspective on Render to Explore Accurate Medical Image Segmentation

Applied Sciences ◽

10.3390/app10186439 ◽

2020 ◽

Vol 10 (18) ◽

pp. 6439

Author(s):

Chen Li ◽

Wei Chen ◽

Yusong Tan

Keyword(s):

Image Segmentation ◽

Mortality Rate ◽

High Mortality ◽

Medical Image ◽

Sampling Method ◽

Medical Image Segmentation ◽

Natural Image ◽

High Quality ◽

Public Datasets ◽

Segmentation Models

Organ lesions have a high mortality rate, and pose a serious threat to people’s lives. Segmenting organs accurately is helpful for doctors to diagnose. There is a demand for the advanced segmentation model for medical images. However, most segmentation models directly migrated from natural image segmentation models. These models usually ignore the importance of the boundary. To solve this difficulty, in this paper, we provided a unique perspective on rendering to explore accurate medical image segmentation. We adapt a subdivision-based point-sampling method to get high-quality boundaries. In addition, we integrated the attention mechanism and nested U-Net architecture into the proposed network Render U-Net.Render U-Net was evaluated on three public datasets, including LiTS, CHAOS, and DSB. This model obtained the best performance on five medical image segmentation tasks.

Download Full-text

Accelerating 3D Medical Image Segmentation by Adaptive Small-Scale Target Localization

Journal of Imaging ◽

10.3390/jimaging7020035 ◽

2021 ◽

Vol 7 (2) ◽

pp. 35

Author(s):

Boris Shirokikh ◽

Alexey Shevtsov ◽

Alexandra Dalechina ◽

Egor Krivov ◽

Valery Kostjuchenko ◽

...

Keyword(s):

Image Segmentation ◽

Graphics Processing Units ◽

Medical Image ◽

State Of The Art ◽

Three Dimensional ◽

Medical Image Segmentation ◽

Small Scale ◽

Processing Unit ◽

Central Processing ◽

Lung Cancer Diagnosis

The prevailing approach for three-dimensional (3D) medical image segmentation is to use convolutional networks. Recently, deep learning methods have achieved human-level performance in several important applied problems, such as volumetry for lung-cancer diagnosis or delineation for radiation therapy planning. However, state-of-the-art architectures, such as U-Net and DeepMedic, are computationally heavy and require workstations accelerated with graphics processing units for fast inference. However, scarce research has been conducted concerning enabling fast central processing unit computations for such networks. Our paper fills this gap. We propose a new segmentation method with a human-like technique to segment a 3D study. First, we analyze the image at a small scale to identify areas of interest and then process only relevant feature-map patches. Our method not only reduces the inference time from 10 min to 15 s but also preserves state-of-the-art segmentation quality, as we illustrate in the set of experiments with two large datasets.

Download Full-text

EM-Net: An Efficient M-Net for segmentation of surgical instruments in colonoscopy frames

Nordic Machine Intelligence ◽

10.5617/nmi.9122 ◽

2021 ◽

Vol 1 (1) ◽

pp. 14-16

Author(s):

Debapriya Banik ◽

Kaushiki Roy ◽

Debotosh Bhattacharjee

Keyword(s):

Image Segmentation ◽

Deep Learning ◽

Medical Image ◽

Medical Image Segmentation ◽

Surgical Instruments ◽

Test Dataset ◽

Segmentation Task

This paper addresses the Instrument Segmentation Task, a subtask for the “MedAI: Transparency in Medical Image Segmentation” challenge. To accomplish the subtask, our team “Med_Seg_JU” has proposed a deep learning-based framework, namely “EM-Net: An Efficient M-Net for segmentation of surgical instruments in colonoscopy frames”. The proposed framework is inspired by the M-Net architecture. In this architecture, we have incorporated the EfficientNet B3 module with U-Net as the backbone. Our proposed method obtained a JC of 0.8205, DSC of 0.8632, PRE of 0.8464, REC of 0.9005, F1 of 0.8632, and ACC of 0.9799 as evaluated by the challenge organizers on a separate test dataset. These results justify the efficacy of our proposed method in the segmentation of the surgical instruments.

Download Full-text

More Birds in the Hand -Medical Image Segmentation using a Multi-Model Ensemble Framework

Nordic Machine Intelligence ◽

10.5617/nmi.9128 ◽

2021 ◽

Vol 1 (1) ◽

pp. 23-25

Author(s):

Yung-Han Chen ◽

Pei-Hsuan Kuo ◽

Yi-Zeng Fang ◽

Wei-Lin Wang

Keyword(s):

Image Segmentation ◽

Medical Image ◽

State Of The Art ◽

Medical Image Segmentation ◽

Model Ensemble ◽

Specific Training ◽

The Given

In this paper, we introduce a multi-model ensemble framework for medical image segmentation. We first collect a set of state-of-the-art models in this field and further improve them through a series of architecture refinement moves and a set of specific training skills. We then integrate these fine-tuned models into a more powerful ensemble framework. Preliminary experiment results show that the proposed multi-model ensemble framework performs well under the given polyp and instrument datasets.

Download Full-text

Optic Disc Segmentation Using Attention-Based U-Net and the Improved Cross-Entropy Convolutional Neural Network

Entropy ◽

10.3390/e22080844 ◽

2020 ◽

Vol 22 (8) ◽

pp. 844

Author(s):

Baixin Jin ◽

Pingping Liu ◽

Peng Wang ◽

Lida Shi ◽

Jing Zhao

Keyword(s):

Image Processing ◽

Image Segmentation ◽

Deep Learning ◽

Optic Disc ◽

Medical Image ◽

Medical Image Analysis ◽

Medical Image Segmentation ◽

Attention Mechanism ◽

Cross Entropy ◽

Segmentation Task

Medical image segmentation is an important part of medical image analysis. With the rapid development of convolutional neural networks in image processing, deep learning methods have achieved great success in the field of medical image processing. Deep learning is also used in the field of auxiliary diagnosis of glaucoma, and the effective segmentation of the optic disc area plays an important assistant role in the diagnosis of doctors in the clinical diagnosis of glaucoma. Previously, many U-Net-based optic disc segmentation methods have been proposed. However, the channel dependence of different levels of features is ignored. The performance of fundus image segmentation in small areas is not satisfactory. In this paper, we propose a new aggregation channel attention network to make full use of the influence of context information on semantic segmentation. Different from the existing attention mechanism, we exploit channel dependencies and integrate information of different scales into the attention mechanism. At the same time, we improved the basic classification framework based on cross entropy, combined the dice coefficient and cross entropy, and balanced the contribution of dice coefficients and cross entropy loss to the segmentation task, which enhanced the performance of the network in small area segmentation. The network retains more image features, restores the significant features more accurately, and further improves the segmentation performance of medical images. We apply it to the fundus optic disc segmentation task. We demonstrate the segmentation performance of the model on the Messidor dataset and the RIM-ONE dataset, and evaluate the proposed architecture. Experimental results show that our network architecture improves the prediction performance of the base architectures under different datasets while maintaining the computational efficiency. The results render that the proposed technologies improve the segmentation with 0.0469 overlapping error on Messidor.

Download Full-text

A deep fully residual convolutional neural network for segmentation in EM images

International Journal of Wavelets Multiresolution and Information Processing ◽

10.1142/s0219691320500071 ◽

2020 ◽

Vol 18 (03) ◽

pp. 2050007

Author(s):

Juanjuan He ◽

Song Xiang ◽

Ziqi Zhu

Keyword(s):

Neural Network ◽

Image Segmentation ◽

Convolutional Neural Network ◽

Medical Image ◽

Spatial Information ◽

State Of The Art ◽

Ct Images ◽

Medical Image Segmentation ◽

Post Process ◽

Proposed Model

In standard U-net, researchers only use long skip connections to skip features from the encoding path to the decoding path in order to recover spatial information loss during downsampling. However, it would result in gradient vanishing and limit the depth of the network. To address this issue, we propose a novel deep fully residual convolutional neural network that combines the U-net with the ResNet for medical image segmentation. By applying short skip connections, this new extension of U-net decreases the amount of parameters compared to the standard U-net, although the depth of the layer is increased. We evaluate the performance of the proposed model and other state-of-the-art models on the Electron Microscopy (EM) images dataset and the Computed Tomography (CT) images dataset. The result shows that our model achieves competitive accuracy on the EM benchmark without any further post-process. Moreover, the performance of image segmentation on CT images of the lungs is improved in contrast to the standard U-net.

Download Full-text