MF-Net: Multi-Scale Information Fusion Network for CNV Segmentation in Retinal OCT Images

Choroid neovascularization (CNV) is one of the blinding ophthalmologic diseases. It is mainly caused by new blood vessels growing in choroid and penetrating Bruch's membrane. Accurate segmentation of CNV is essential for ophthalmologists to analyze the condition of the patient and specify treatment plan. Although many deep learning-based methods have achieved promising results in many medical image segmentation tasks, CNV segmentation in retinal optical coherence tomography (OCT) images is still very challenging as the blur boundary of CNV, large morphological differences, speckle noise, and other similar diseases interference. In addition, the lack of pixel-level annotation data is also one of the factors that affect the further improvement of CNV segmentation accuracy. To improve the accuracy of CNV segmentation, a novel multi-scale information fusion network (MF-Net) based on U-Shape architecture is proposed for CNV segmentation in retinal OCT images. A novel multi-scale adaptive-aware deformation module (MAD) is designed and inserted into the top of the encoder path, aiming at guiding the model to focus on multi-scale deformation of the targets, and aggregates the contextual information. Meanwhile, to improve the ability of the network to learn to supplement low-level local high-resolution semantic information to high-level feature maps, a novel semantics-details aggregation module (SDA) between encoder and decoder is proposed. In addition, to leverage unlabeled data to further improve the CNV segmentation, a semi-supervised version of MF-Net is designed based on pseudo-label data augmentation strategy, which can leverage unlabeled data to further improve CNV segmentation accuracy. Finally, comprehensive experiments are conducted to validate the performance of the proposed MF-Net and SemiMF-Net. The experiment results show that both proposed MF-Net and SemiMF-Net outperforms other state-of-the-art algorithms.

Download Full-text

U2-ONet: A Two-Level Nested Octave U-Structure Network with a Multi-Scale Attention Mechanism for Moving Object Segmentation

Remote Sensing ◽

10.3390/rs13010060 ◽

2020 ◽

Vol 13 (1) ◽

pp. 60

Author(s):

Chenjie Wang ◽

Chengyuan Li ◽

Jun Liu ◽

Bin Luo ◽

Xin Su ◽

...

Keyword(s):

Moving Objects ◽

Object Segmentation ◽

Contextual Information ◽

Attention Mechanism ◽

Moving Object ◽

Feature Maps ◽

Moving Object Segmentation ◽

Practical Applications ◽

Multi Scale ◽

Spatial Redundancy

Most scenes in practical applications are dynamic scenes containing moving objects, so accurately segmenting moving objects is crucial for many computer vision applications. In order to efficiently segment all the moving objects in the scene, regardless of whether the object has a predefined semantic label, we propose a two-level nested octave U-structure network with a multi-scale attention mechanism, called U2-ONet. U2-ONet takes two RGB frames, the optical flow between these frames, and the instance segmentation of the frames as inputs. Each stage of U2-ONet is filled with the newly designed octave residual U-block (ORSU block) to enhance the ability to obtain more contextual information at different scales while reducing the spatial redundancy of the feature maps. In order to efficiently train the multi-scale deep network, we introduce a hierarchical training supervision strategy that calculates the loss at each level while adding knowledge-matching loss to keep the optimization consistent. The experimental results show that the proposed U2-ONet method can achieve a state-of-the-art performance in several general moving object segmentation datasets.

Download Full-text

Multiscale Object Detection in Infrared Streetscape Images Based on Deep Learning and Instance Level Data Augmentation

Applied Sciences ◽

10.3390/app9030565 ◽

2019 ◽

Vol 9 (3) ◽

pp. 565 ◽

Cited By ~ 6

Author(s):

Hao Qu ◽

Lilian Zhang ◽

Xuesong Wu ◽

Xiaofeng He ◽

Xiaoping Hu ◽

...

Keyword(s):

Object Detection ◽

Data Augmentation ◽

Region Of Interest ◽

Complex Environments ◽

Feature Maps ◽

Multi Scale ◽

Level Data ◽

Training Stage ◽

Street Scene ◽

Layer Region

The development of object detection in infrared images has attracted more attention in recent years. However, there are few studies on multi-scale object detection in infrared street scene images. Additionally, the lack of high-quality infrared datasets hinders research into such algorithms. In order to solve these issues, we firstly make a series of modifications based on Faster Region-Convolutional Neural Network (R-CNN). In this paper, a double-layer region proposal network (RPN) is proposed to predict proposals of different scales on both fine and coarse feature maps. Secondly, a multi-scale pooling module is introduced into the backbone of the network to explore the response of objects on different scales. Furthermore, the inception4 module and the position sensitive region of interest (ROI) align (PSalign) pooling layer are utilized to explore richer features of the objects. Thirdly, this paper proposes instance level data augmentation, which takes into account the imbalance between categories while enlarging dataset. In the training stage, the online hard example mining method is utilized to further improve the robustness of the algorithm in complex environments. The experimental results show that, compared with baseline, our detection method has state-of-the-art performance.

Download Full-text

DMCNN: A Deep Multiscale Convolutional Neural Network Model for Medical Image Segmentation

Journal of Healthcare Engineering ◽

10.1155/2019/8597606 ◽

2019 ◽

Vol 2019 ◽

pp. 1-10 ◽

Cited By ~ 2

Author(s):

Lin Teng ◽

Hang Li ◽

Shahid Karim

Keyword(s):

Neural Network ◽

Feature Extraction ◽

Image Segmentation ◽

Convolutional Neural Network ◽

Medical Image ◽

Data Augmentation ◽

Medical Images ◽

Medical Image Segmentation ◽

Related Area ◽

Segmentation Accuracy

Medical image segmentation is one of the hot issues in the related area of image processing. Precise segmentation for medical images is a vital guarantee for follow-up treatment. At present, however, low gray contrast and blurred tissue boundaries are common in medical images, and the segmentation accuracy of medical images cannot be effectively improved. Especially, deep learning methods need more training samples, which lead to time-consuming process. Therefore, we propose a novelty model for medical image segmentation based on deep multiscale convolutional neural network (CNN) in this article. First, we extract the region of interest from the raw medical images. Then, data augmentation is operated to acquire more training datasets. Our proposed method contains three models: encoder, U-net, and decoder. Encoder is mainly responsible for feature extraction of 2D image slice. The U-net cascades the features of each block of the encoder with those obtained by deconvolution in the decoder under different scales. The decoding is mainly responsible for the upsampling of the feature graph after feature extraction of each group. Simulation results show that the new method can boost the segmentation accuracy. And, it has strong robustness compared with other segmentation methods.

Download Full-text

Small Object Detection in Traffic Scenes Based on Attention Feature Fusion

Sensors ◽

10.3390/s21093031 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3031

Author(s):

Jing Lian ◽

Yuhang Yin ◽

Linhui Li ◽

Zhenghao Wang ◽

Yafu Zhou

Keyword(s):

Object Detection ◽

Feature Fusion ◽

Contextual Information ◽

Detection Accuracy ◽

Small Object ◽

Limited Information ◽

Feature Maps ◽

Multi Scale ◽

Validation Set ◽

Small Object Detection

There are many small objects in traffic scenes, but due to their low resolution and limited information, their detection is still a challenge. Small object detection is very important for the understanding of traffic scene environments. To improve the detection accuracy of small objects in traffic scenes, we propose a small object detection method in traffic scenes based on attention feature fusion. First, a multi-scale channel attention block (MS-CAB) is designed, which uses local and global scales to aggregate the effective information of the feature maps. Based on this block, an attention feature fusion block (AFFB) is proposed, which can better integrate contextual information from different layers. Finally, the AFFB is used to replace the linear fusion module in the object detection network and obtain the final network structure. The experimental results show that, compared to the benchmark model YOLOv5s, this method has achieved a higher mean Average Precison (mAP) under the premise of ensuring real-time performance. It increases the mAP of all objects by 0.9 percentage points on the validation set of the traffic scene dataset BDD100K, and at the same time, increases the mAP of small objects by 3.5%.

Download Full-text

Semantic Segmentation of High-Resolution Airborne Images With Dual-Stream DeepLabV3+

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi11010023 ◽

2021 ◽

Vol 11 (1) ◽

pp. 23

Author(s):

Ozgun Akcay ◽

Ahmet Cumhur Kinaci ◽

Emin Ozgur Avsar ◽

Umut Aydar

Keyword(s):

Data Augmentation ◽

Contextual Information ◽

Imbalanced Data ◽

Semantic Segmentation ◽

Multi Scale ◽

Segmentation Algorithms ◽

Geographic Datasets ◽

Stream Architecture ◽

The Given

In geospatial applications such as urban planning and land use management, automatic detection and classification of earth objects are essential and primary subjects. When the significant semantic segmentation algorithms are considered, DeepLabV3+ stands out as a state-of-the-art CNN. Although the DeepLabV3+ model is capable of extracting multi-scale contextual information, there is still a need for multi-stream architectural approaches and different training approaches of the model that can leverage multi-modal geographic datasets. In this study, a new end-to-end dual-stream architecture that considers geospatial imagery was developed based on the DeepLabV3+ architecture. As a result, the spectral datasets other than RGB provided increments in semantic segmentation accuracies when they were used as additional channels to height information. Furthermore, both the given data augmentation and Tversky loss function which is sensitive to imbalanced data accomplished better overall accuracies. Also, it has been shown that the new dual-stream architecture using Potsdam and Vaihingen datasets produced 88.87% and 87.39% overall semantic segmentation accuracies, respectively. Eventually, it was seen that enhancement of the traditional significant semantic segmentation networks has a great potential to provide higher model performances, whereas the contribution of geospatial data as the second stream to RGB to segmentation was explicitly shown.

Download Full-text

Medical Image Segmentation Algorithm Based on Multi-scale Color Wavelet Texture

International Journal of Circuits, Systems and Signal Processing ◽

10.46300/9106.2021.15.99 ◽

2021 ◽

Vol 15 ◽

pp. 928-935

Author(s):

Qi Zhang ◽

Yan Li

Keyword(s):

Brain Tumor ◽

Magnetic Resonance ◽

Medical Image ◽

Feature Detection ◽

Medical Image Segmentation ◽

Block Matching ◽

Bright Spot ◽

Multi Scale ◽

Segmentation Accuracy ◽

Tumor Mri

In order to improve the segmentation accuracy of brain tumor magnetic resonance medical image, a segmentation method of brain tumor magnetic resonance medical image based on multi-scale color wavelet texture features is proposed. The segmentation model of brain tumor magnetic resonance medical image is established, and the motion damage information of brain tumor magnetic resonance medical image is adaptively fused in the ultrasound imaging environment. The medical image information is enhanced by using the motion skeletal muscle block matching technology. According to the suspicious point feature matching method of brain tumor, the fusion detection and processing of brain tumor magnetic resonance medical image are carried out. The multi-scale color wavelet texture feature detection method is used to extract the image features of brain tumor MRI points, and the CT bright spot features are used to analyze the features of brain tumor MRI medical images. Combined with the adaptive neural network training method, the automatic detection of brain tumor magnetic resonance medical image is completed, and the suspected brain tumor points are extracted, so as to realize the segmentation of brain tumor magnetic resonance medical image. Simulation results show that the proposed method can effectively improve the segmentation accuracy of brain tumor MRI medical image, and has high resolution and accuracy for suspicious brain tumor detection.

Download Full-text

A Multi-Scale Feature Fusion Method Based on U-Net for Retinal Vessel Segmentation

Entropy ◽

10.3390/e22080811 ◽

2020 ◽

Vol 22 (8) ◽

pp. 811

Author(s):

Dan Yang ◽

Guoru Liu ◽

Mengcheng Ren ◽

Bin Xu ◽

Jiao Wang

Keyword(s):

Data Augmentation ◽

Feature Fusion ◽

Automatic Segmentation ◽

Retinal Vessel ◽

Vessel Segmentation ◽

Feature Maps ◽

Scale Feature ◽

Multi Scale ◽

Retinal Blood Vessels ◽

Retinal Vessel Segmentation

Computer-aided automatic segmentation of retinal blood vessels plays an important role in the diagnosis of diseases such as diabetes, glaucoma, and macular degeneration. In this paper, we propose a multi-scale feature fusion retinal vessel segmentation model based on U-Net, named MSFFU-Net. The model introduces the inception structure into the multi-scale feature extraction encoder part, and the max-pooling index is applied during the upsampling process in the feature fusion decoder of an improved network. The skip layer connection is used to transfer each set of feature maps generated on the encoder path to the corresponding feature maps on the decoder path. Moreover, a cost-sensitive loss function based on the Dice coefficient and cross-entropy is designed. Four transformations—rotating, mirroring, shifting and cropping—are used as data augmentation strategies, and the CLAHE algorithm is applied to image preprocessing. The proposed framework is tested and trained on DRIVE and STARE, and sensitivity (Sen), specificity (Spe), accuracy (Acc), and area under curve (AUC) are adopted as the evaluation metrics. Detailed comparisons with U-Net model, at last, it verifies the effectiveness and robustness of the proposed model. The Sen of 0.7762 and 0.7721, Spe of 0.9835 and 0.9885, Acc of 0.9694 and 0.9537 and AUC value of 0.9790 and 0.9680 were achieved on DRIVE and STARE databases, respectively. Results are also compared to other state-of-the-art methods, demonstrating that the performance of the proposed method is superior to that of other methods and showing its competitive results.

Download Full-text

Adaptive Multi-Scale Feature Fusion based U-net for fracture segmentation in coal rock images

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-211968 ◽

2021 ◽

pp. 1-14

Author(s):

Fengli Lu ◽

Chengcai Fu ◽

Guoying Zhang ◽

Jie Shi

Keyword(s):

Coalbed Methane ◽

Data Augmentation ◽

Feature Fusion ◽

Ct Images ◽

Training Data ◽

Feature Maps ◽

Scale Feature ◽

Multi Scale ◽

Surrounding Matrix ◽

Coal Rock

Accurate segmentation of fractures in coal rock CT images is important for the development of coalbed methane. However, due to the large variation of fracture scale and the similarity of gray values between weak fractures and the surrounding matrix, it remains a challenging task. And there is no published dataset of coal rock, which make the task even harder. In this paper, a novel adaptive multi-scale feature fusion method based on U-net (AMSFF-U-net) is proposed for fracture segmentation in coal rock CT images. Specifically, encoder and decoder path consist of residual blocks (ReBlock), respectively. The attention skip concatenation (ASC) module is proposed to capture more representative and distinguishing features by combining the high-level and low-level features of adjacent layers. The adaptive multi-scale feature fusion (AMSFF) module is presented to adaptively fuse different scale feature maps of encoder path; it can effectively capture rich multi-scale features. In response to the lack of coal rock fractures training data, we applied a set of comprehensive data augmentation operations to increase the diversity of training samples. These extensive experiments are conducted via seven state-of-the-art methods (i.e., FCEM, U-net, Res-Unet, Unet++, MSN-Net, WRAU-Net and ours). The experiment results demonstrate that the proposed AMSFF-U-net can achieve better segmentation performance in our works, particularly for weak fractures and tiny scale fractures.

Download Full-text

RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans

Frontiers in Bioengineering and Biotechnology ◽

10.3389/fbioe.2020.605132 ◽

2020 ◽

Vol 8 ◽

Author(s):

Qiangguo Jin ◽

Zhaopeng Meng ◽

Changming Sun ◽

Hui Cui ◽

Ran Su

Keyword(s):

Liver Tumor ◽

Contextual Information ◽

Medical Image Segmentation ◽

Tumor Segmentation ◽

Feature Maps ◽

Deep Convolutional Neural Networks ◽

Volumetric Images ◽

Liver Tumor Segmentation ◽

Liver Region ◽

High Level

Automatic extraction of liver and tumor from CT volumes is a challenging task due to their heterogeneous and diffusive shapes. Recently, 2D deep convolutional neural networks have become popular in medical image segmentation tasks because of the utilization of large labeled datasets to learn hierarchical features. However, few studies investigate 3D networks for liver tumor segmentation. In this paper, we propose a 3D hybrid residual attention-aware segmentation method, i.e., RA-UNet, to precisely extract the liver region and segment tumors from the liver. The proposed network has a basic architecture as U-Net which extracts contextual information combining low-level feature maps with high-level ones. Attention residual modules are integrated so that the attention-aware features change adaptively. This is the first work that an attention residual mechanism is used to segment tumors from 3D medical volumetric images. We evaluated our framework on the public MICCAI 2017 Liver Tumor Segmentation dataset and tested the generalization on the 3DIRCADb dataset. The experiments show that our architecture obtains competitive results.

Download Full-text

Segmenting Medical MRI via Recurrent Decoding Cell

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6932 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12452-12459 ◽

Cited By ~ 2

Author(s):

Ying Wen ◽

Kai Xie ◽

Lianghua He

Keyword(s):

Feature Fusion ◽

Image Features ◽

Medical Image Segmentation ◽

Network Robustness ◽

Feature Maps ◽

Segmentation Accuracy ◽

Model Size ◽

Score Map ◽

A Chain

The encoder-decoder networks are commonly used in medical image segmentation due to their remarkable performance in hierarchical feature fusion. However, the expanding path for feature decoding and spatial recovery does not consider the long-term dependency when fusing feature maps from different layers, and the universal encoder-decoder network does not make full use of the multi-modality information to improve the network robustness especially for segmenting medical MRI. In this paper, we propose a novel feature fusion unit called Recurrent Decoding Cell (RDC) which leverages convolutional RNNs to memorize the long-term context information from the previous layers in the decoding phase. An encoder-decoder network, named Convolutional Recurrent Decoding Network (CRDN), is also proposed based on RDC for segmenting multi-modality medical MRI. CRDN adopts CNN backbone to encode image features and decode them hierarchically through a chain of RDCs to obtain the final high-resolution score map. The evaluation experiments on BrainWeb, MRBrainS and HVSMR datasets demonstrate that the introduction of RDC effectively improves the segmentation accuracy as well as reduces the model size, and the proposed CRDN owns its robustness to image noise and intensity non-uniformity in medical MRI.

Download Full-text