Adaptive Multi-Scale Feature Fusion based U-net for fracture segmentation in coal rock images

2021 ◽  
pp. 1-14
Author(s):  
Fengli Lu ◽  
Chengcai Fu ◽  
Guoying Zhang ◽  
Jie Shi
Keyword(s):  
Coalbed Methane ◽  
Data Augmentation ◽  
Feature Fusion ◽  
Ct Images ◽  
Training Data ◽  
Feature Maps ◽  
Scale Feature ◽  
Multi Scale ◽  
Surrounding Matrix ◽  
Coal Rock

Accurate segmentation of fractures in coal rock CT images is important for the development of coalbed methane. However, due to the large variation of fracture scale and the similarity of gray values between weak fractures and the surrounding matrix, it remains a challenging task. And there is no published dataset of coal rock, which make the task even harder. In this paper, a novel adaptive multi-scale feature fusion method based on U-net (AMSFF-U-net) is proposed for fracture segmentation in coal rock CT images. Specifically, encoder and decoder path consist of residual blocks (ReBlock), respectively. The attention skip concatenation (ASC) module is proposed to capture more representative and distinguishing features by combining the high-level and low-level features of adjacent layers. The adaptive multi-scale feature fusion (AMSFF) module is presented to adaptively fuse different scale feature maps of encoder path; it can effectively capture rich multi-scale features. In response to the lack of coal rock fractures training data, we applied a set of comprehensive data augmentation operations to increase the diversity of training samples. These extensive experiments are conducted via seven state-of-the-art methods (i.e., FCEM, U-net, Res-Unet, Unet++, MSN-Net, WRAU-Net and ours). The experiment results demonstrate that the proposed AMSFF-U-net can achieve better segmentation performance in our works, particularly for weak fractures and tiny scale fractures.

scholarly journals Adaptive Multi-Scale Feature Fusion Based Residual U-net for Fracture Segmentation in Coal Rock Images

2020 ◽  
Author(s):  
Fengli Lu ◽  
Chengcai Fu ◽  
Guoying Zhang ◽  
Jie Shi
Keyword(s):  
Data Augmentation ◽  
Spatial Information ◽  
Feature Fusion ◽  
Ct Images ◽  
Rock Fractures ◽  
Feature Maps ◽  
High Background ◽  
Scale Feature ◽  
Multi Scale ◽  
Coal Rock

Abstract Accurate segmentation of fractures in coal rock CT images is important for safe production and the development of coalbed methane.However,to make segment coal rock fractures accurate,the challenges as the following:1)The coal rock CT images have the characteristics which are high background noise, sparse target, weak boundary information, uneven gray level, low contrast etc.; 2)There is no a public dataset of coal rock CT images;3)Limited coal rock CT images samples.In the paper,we proposed adaptive multi-scale feature fusion based residual U-uet(AMSFFRU-uet) for fracture segmentation in coal rock CT images to address the issues.In order to reduce the loss of tiny and weak fractures, dilated residual blocks (DResBlock) are embedded into the U-uet structure, which expand the receptive field and extract fracture information atdifferent scales.Furthermore, for reducing the loss of spatial information during the down-sampling process, feature maps of different sizes in the encoding branch are concatenated by adaptive multi-scale featurefusion module,which is as the input of the first up-sampling in the decoding branch.And we applieda set of comprehensive data augmentation operations to increase the diversity of training samples. Our network,U-net and ResU-net are tested on our dataset of coal rock CT images with 5 different textures.The experimental results show that compared with U-net and ResU-net, our proposed approach improve the average Dice coefficient by 5.1% and 2.9% and the average accuracy by 4.5% and 2%,respectively.Therefore,AMSFFRU-net can achieve better segmentation of coal rock fractures,and has stronger generalization ability and robustness.

scholarly journals Adaptive Multi-Scale Feature Fusion Based Residual U-net for Fracture Segmentation in Coal Rock Images

2020 ◽  
Author(s):  
Fengli Lu ◽  
Chengcai Fu ◽  
Guoying Zhang ◽  
Jie Shi
Keyword(s):  
Spatial Information ◽  
Feature Fusion ◽  
Ct Images ◽  
Published Data ◽  
Rock Fractures ◽  
Feature Maps ◽  
Data Set ◽  
Scale Feature ◽  
Multi Scale ◽  
Coal Rock

Abstract Accurate segmentation of fractures in coal rock CT images is important for safe production and the development of coalbed methane. However, the coal rock fractures formed through natural geological evolution, which are complex, low contrast and different scales. Furthermore, there is no published data set of coal rock. In this paper, we proposed adaptive multi-scale feature fusion based residual U-uet (AMSFFR-U-uet) for fracture segmentation in coal rock CT images. The dilated residual blocks (DResBlock) with dilated ratio (1,2,3) are embedded into encoding branch of the U-uet structure, which can improve the ability of extract feature of network and capture different scales fractures. Furthermore, feature maps of different sizes in the encoding branch are concatenated by adaptive multi-scale feature fusion (AMSFF) module. And AMSFF can not only capture different scales fractures but also improve the restoration of spatial information. To alleviate the lack of coal rock fractures training data, we applied a set of comprehensive data augmentation operations to increase the diversity of training samples. Our network, U-net and Res-U-net are tested on our test set of coal rock CT images with five different region coal rock samples. The experimental results show that our proposed approach improve the average Dice coefficient by 2.9%, the average precision by 7.2% and the average Recall by 9.1% , respectively. Therefore, AMSFFR-U-net can achieve better segmentation results of coal rock fractures, and has stronger generalization ability and robustness.

scholarly journals A Multi-Scale Feature Fusion Method Based on U-Net for Retinal Vessel Segmentation

Entropy ◽  
2020 ◽  
Vol 22 (8) ◽  
pp. 811
Author(s):  
Dan Yang ◽  
Guoru Liu ◽  
Mengcheng Ren ◽  
Bin Xu ◽  
Jiao Wang
Keyword(s):  
Data Augmentation ◽  
Feature Fusion ◽  
Automatic Segmentation ◽  
Retinal Vessel ◽  
Vessel Segmentation ◽  
Feature Maps ◽  
Scale Feature ◽  
Multi Scale ◽  
Retinal Blood Vessels ◽  
Retinal Vessel Segmentation

Computer-aided automatic segmentation of retinal blood vessels plays an important role in the diagnosis of diseases such as diabetes, glaucoma, and macular degeneration. In this paper, we propose a multi-scale feature fusion retinal vessel segmentation model based on U-Net, named MSFFU-Net. The model introduces the inception structure into the multi-scale feature extraction encoder part, and the max-pooling index is applied during the upsampling process in the feature fusion decoder of an improved network. The skip layer connection is used to transfer each set of feature maps generated on the encoder path to the corresponding feature maps on the decoder path. Moreover, a cost-sensitive loss function based on the Dice coefficient and cross-entropy is designed. Four transformations—rotating, mirroring, shifting and cropping—are used as data augmentation strategies, and the CLAHE algorithm is applied to image preprocessing. The proposed framework is tested and trained on DRIVE and STARE, and sensitivity (Sen), specificity (Spe), accuracy (Acc), and area under curve (AUC) are adopted as the evaluation metrics. Detailed comparisons with U-Net model, at last, it verifies the effectiveness and robustness of the proposed model. The Sen of 0.7762 and 0.7721, Spe of 0.9835 and 0.9885, Acc of 0.9694 and 0.9537 and AUC value of 0.9790 and 0.9680 were achieved on DRIVE and STARE databases, respectively. Results are also compared to other state-of-the-art methods, demonstrating that the performance of the proposed method is superior to that of other methods and showing its competitive results.

scholarly journals Multi-Scale Feature Fusion for Coal-Rock Recognition Based on Completed Local Binary Pattern and Convolution Neural Network

Entropy ◽  
2019 ◽  
Vol 21 (6) ◽  
pp. 622 ◽  
Author(s):  
Xiaoyang Liu ◽  
Wei Jing ◽  
Mingxuan Zhou ◽  
Yuxing Li
Keyword(s):  
Neural Network ◽  
Local Binary Pattern ◽  
Feature Vector ◽  
Feature Fusion ◽  
Scale Feature ◽  
Multi Scale ◽  
Texture Information ◽  
Deep Feature ◽  
Coal Rock ◽  
Rock Image

Automatic coal-rock recognition is one of the critical technologies for intelligent coal mining and processing. Most existing coal-rock recognition methods have some defects, such as unsatisfactory performance and low robustness. To solve these problems, and taking distinctive visual features of coal and rock into consideration, the multi-scale feature fusion coal-rock recognition (MFFCRR) model based on a multi-scale Completed Local Binary Pattern (CLBP) and a Convolution Neural Network (CNN) is proposed in this paper. Firstly, the multi-scale CLBP features are extracted from coal-rock image samples in the Texture Feature Extraction (TFE) sub-model, which represents texture information of the coal-rock image. Secondly, the high-level deep features are extracted from coal-rock image samples in the Deep Feature Extraction (DFE) sub-model, which represents macroscopic information of the coal-rock image. The texture information and macroscopic information are acquired based on information theory. Thirdly, the multi-scale feature vector is generated by fusing the multi-scale CLBP feature vector and deep feature vector. Finally, multi-scale feature vectors are input to the nearest neighbor classifier with the chi-square distance to realize coal-rock recognition. Experimental results show the coal-rock image recognition accuracy of the proposed MFFCRR model reaches 97.9167%, which increased by 2%–3% compared with state-of-the-art coal-rock recognition methods.

A Novel Multi-Scale Feature Fusion Method for Region Proposal Network in Fast Object Detection

2020 ◽  
Vol 16 (3) ◽  
pp. 132-145
Author(s):  
Gang Liu ◽  
Chuyi Wang
Keyword(s):  
Object Detection ◽  
Multiple Scales ◽  
Feature Fusion ◽  
Uniform Space ◽  
Fusion Method ◽  
Well Performance ◽  
Feature Maps ◽  
Neural Network Models ◽  
Scale Feature ◽  
Multi Scale

Neural network models have been widely used in the field of object detecting. The region proposal methods are widely used in the current object detection networks and have achieved well performance. The common region proposal methods hunt the objects by generating thousands of the candidate boxes. Compared to other region proposal methods, the region proposal network (RPN) method improves the accuracy and detection speed with several hundred candidate boxes. However, since the feature maps contains insufficient information, the ability of RPN to detect and locate small-sized objects is poor. A novel multi-scale feature fusion method for region proposal network to solve the above problems is proposed in this article. The proposed method is called multi-scale region proposal network (MS-RPN) which can generate suitable feature maps for the region proposal network. In MS-RPN, the selected feature maps at multiple scales are fine turned respectively and compressed into a uniform space. The generated fusion feature maps are called refined fusion features (RFFs). RFFs incorporate abundant detail information and context information. And RFFs are sent to RPN to generate better region proposals. The proposed approach is evaluated on PASCAL VOC 2007 and MS COCO benchmark tasks. MS-RPN obtains significant improvements over the comparable state-of-the-art detection models.

scholarly journals Adaptive Weighted Multi-Level Fusion of Multi-Scale Features: A New Approach to Pedestrian Detection

2021 ◽  
Vol 13 (2) ◽  
pp. 38
Author(s):  
Yao Xu ◽  
Qin Yu
Keyword(s):  
Deep Learning ◽  
Feature Fusion ◽  
Pedestrian Detection ◽  
Feature Maps ◽  
Scale Feature ◽  
Multi Scale ◽  
One Stage ◽  
Current State ◽  
Multi Level ◽  
Feature Utilization

Great achievements have been made in pedestrian detection through deep learning. For detectors based on deep learning, making better use of features has become the key to their detection effect. While current pedestrian detectors have made efforts in feature utilization to improve their detection performance, the feature utilization is still inadequate. To solve the problem of inadequate feature utilization, we proposed the Multi-Level Feature Fusion Module (MFFM) and its Multi-Scale Feature Fusion Unit (MFFU) sub-module, which connect feature maps of the same scale and different scales by using horizontal and vertical connections and shortcut structures. All of these connections are accompanied by weights that can be learned; thus, they can be used as adaptive multi-level and multi-scale feature fusion modules to fuse the best features. Then, we built a complete pedestrian detector, the Adaptive Feature Fusion Detector (AFFDet), which is an anchor-free one-stage pedestrian detector that can make full use of features for detection. As a result, compared with other methods, our method has better performance on the challenging Caltech Pedestrian Detection Benchmark (Caltech) and has quite competitive speed. It is the current state-of-the-art one-stage pedestrian detection method.

scholarly journals A Single Shot Framework with Multi-Scale Feature Fusion for Geospatial Object Detection

2019 ◽  
Vol 11 (5) ◽  
pp. 594 ◽  
Author(s):  
Shuo Zhuang ◽  
Ping Wang ◽  
Boran Jiang ◽  
Gang Wang ◽  
Cong Wang
Keyword(s):  
Remote Sensing ◽  
Object Detection ◽  
Large Scale ◽  
Feature Fusion ◽  
Aerial Images ◽  
Detection Methods ◽  
Single Shot ◽  
Feature Maps ◽  
Scale Feature ◽  
Multi Scale

With the rapid advances in remote-sensing technologies and the larger number of satellite images, fast and effective object detection plays an important role in understanding and analyzing image information, which could be further applied to civilian and military fields. Recently object detection methods with region-based convolutional neural network have shown excellent performance. However, these two-stage methods contain region proposal generation and object detection procedures, resulting in low computation speed. Because of the expensive manual costs, the quantity of well-annotated aerial images is scarce, which also limits the progress of geospatial object detection in remote sensing. In this paper, on the one hand, we construct and release a large-scale remote-sensing dataset for geospatial object detection (RSD-GOD) that consists of 5 different categories with 18,187 annotated images and 40,990 instances. On the other hand, we design a single shot detection framework with multi-scale feature fusion. The feature maps from different layers are fused together through the up-sampling and concatenation blocks to predict the detection results. High-level features with semantic information and low-level features with fine details are fully explored for detection tasks, especially for small objects. Meanwhile, a soft non-maximum suppression strategy is put into practice to select the final detection results. Extensive experiments have been conducted on two datasets to evaluate the designed network. Results show that the proposed approach achieves a good detection performance and obtains the mean average precision value of 89.0% on a newly constructed RSD-GOD dataset and 83.8% on the Northwestern Polytechnical University very high spatial resolution-10 (NWPU VHR-10) dataset at 18 frames per second (FPS) on a NVIDIA GTX-1080Ti GPU.

Deep multi-scale feature fusion for pancreas segmentation from CT images

2020 ◽  
Vol 15 (3) ◽  
pp. 415-423
Author(s):  
Zhanlan Chen ◽  
Xiuying Wang ◽  
Ke Yan ◽  
Jiangbin Zheng
Keyword(s):  
Feature Fusion ◽  
Ct Images ◽  
Scale Feature ◽  
Multi Scale ◽  
Pancreas Segmentation

Multi-Scale Feature Fusion Convolutional Neural Network for Concurrent Segmentation of Left Ventricle and Myocardium in Cardiac MR Images

2020 ◽  
Vol 10 (5) ◽  
pp. 1023-1032
Author(s):  
Lin Qi ◽  
Haoran Zhang ◽  
Xuehao Cao ◽  
Xuyang Lyu ◽  
Lisheng Xu ◽  
...  
Keyword(s):  
Neural Network ◽  
Left Ventricle ◽  
Convolutional Neural Network ◽  
Feature Fusion ◽  
Left Ventricular ◽  
Mr Images ◽  
Feature Maps ◽  
Scale Feature ◽  
Multi Scale ◽  
Cine Mr

Accurate segmentation of the blood pool of left ventricle (LV) and myocardium (or left ventricular epicardium, MYO) from cardiac magnetic resonance (MR) can help doctors to quantify LV ejection fraction and myocardial deformation. To reduce doctor’s burden of manual segmentation, in this study, we propose an automated and concurrent segmentation method of the LV and MYO. First, we employ a convolutional neural network (CNN) architecture to extract the region of interest (ROI) from short-axis cardiac cine MR images as a preprocessing step. Next, we present a multi-scale feature fusion (MSFF) CNN with a new weighted Dice index (WDI) loss function to get the concurrent segmentation of the LV and MYO. We use MSFF modules with three scales to extract different features, and then concatenate feature maps by the short and long skip connections in the encoder and decoder path to capture more complete context information and geometry structure for better segmentation. Finally, we compare the proposed method with Fully Convolutional Networks (FCN) and U-Net on the combined cardiac datasets from MICCAI 2009 and ACDC 2017. Experimental results demonstrate that the proposed method could perform effectively on LV and MYOs segmentation in the combined datasets, indicating its potential for clinical application.

scholarly journals Automated Ventricular System Segmentation in Paediatric Patients Treated for Hydrocephalus Using Deep Learning Methods

2019 ◽  
Vol 2019 ◽  
pp. 1-9 ◽  
Author(s):  
Michał Klimont ◽  
Mateusz Flieger ◽  
Jacek Rzeszutek ◽  
Joanna Stachera ◽  
Aleksandra Zakrzewska ◽  
...  
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Ct Images ◽  
Policy Transfer ◽  
Training Data ◽  
Intraobserver Variability ◽  
Practical Applications ◽  
Brain Scans ◽  
Rate Policy ◽  
Ct Brain

Hydrocephalus is a common neurological condition that can have traumatic ramifications and can be lethal without treatment. Nowadays, during therapy radiologists have to spend a vast amount of time assessing the volume of cerebrospinal fluid (CSF) by manual segmentation on Computed Tomography (CT) images. Further, some of the segmentations are prone to radiologist bias and high intraobserver variability. To improve this, researchers are exploring methods to automate the process, which would enable faster and more unbiased results. In this study, we propose the application of U-Net convolutional neural network in order to automatically segment CT brain scans for location of CSF. U-Net is a neural network that has proven to be successful for various interdisciplinary segmentation tasks. We optimised training using state of the art methods, including “1cycle” learning rate policy, transfer learning, generalized dice loss function, mixed float precision, self-attention, and data augmentation. Even though the study was performed using a limited amount of data (80 CT images), our experiment has shown near human-level performance. We managed to achieve a 0.917 mean dice score with 0.0352 standard deviation on cross validation across the training data and a 0.9506 mean dice score on a separate test set. To our knowledge, these results are better than any known method for CSF segmentation in hydrocephalic patients, and thus, it is promising for potential practical applications.

Sign in / Sign up

Export Citation Format

Share Document