scholarly journals HR-MPF: high-resolution representation network with multi-scale progressive fusion for pulmonary nodule segmentation and classification

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Ling Zhu ◽  
Hongqing Zhu ◽  
Suyi Yang ◽  
Pengyu Wang ◽  
Yang Yu
Keyword(s):  
High Resolution ◽  
Loss Function ◽  
Pulmonary Nodule ◽  
Feature Fusion ◽  
Back Propagation ◽  
Pulmonary Nodules ◽  
Improve Patient Survival ◽  
Multi Scale ◽  
Training Scheme ◽  
Adversarial Training

AbstractAccurate segmentation and classification of pulmonary nodules are of great significance to early detection and diagnosis of lung diseases, which can reduce the risk of developing lung cancer and improve patient survival rate. In this paper, we propose an effective network for pulmonary nodule segmentation and classification at one time based on adversarial training scheme. The segmentation network consists of a High-Resolution network with Multi-scale Progressive Fusion (HR-MPF) and a proposed Progressive Decoding Module (PDM) recovering final pixel-wise prediction results. Specifically, the proposed HR-MPF firstly incorporates boosted module to High-Resolution Network (HRNet) in a progressive feature fusion manner. In this case, feature communication is augmented among all levels in this high-resolution network. Then, downstream classification module would identify benign and malignant pulmonary nodules based on feature map from PDM. In the adversarial training scheme, a discriminator is set to optimize HR-MPF and PDM through back propagation. Meanwhile, a reasonably designed multi-task loss function optimizes performance of segmentation and classification overall. To improve the accuracy of boundary prediction crucial to nodule segmentation, a boundary consistency constraint is designed and incorporated in the segmentation loss function. Experiments on publicly available LUNA16 dataset show that the framework outperforms relevant advanced methods in quantitative evaluation and visual perception.

scholarly journals High-Resolution SAR Image Classification Using Multi-Scale Deep Feature Fusion and Covariance Pooling Manifold Network

2021 ◽  
Vol 13 (2) ◽  
pp. 328
Author(s):  
Wenkai Liang ◽  
Yan Wu ◽  
Ming Li ◽  
Yice Cao ◽  
Xin Hu
Keyword(s):  
High Resolution ◽  
Image Classification ◽  
Feature Fusion ◽  
Representation Learning ◽  
Sar Image ◽  
Gabor Filtering ◽  
Feature Maps ◽  
Sar Images ◽  
Multi Scale ◽  
Deep Feature

The classification of high-resolution (HR) synthetic aperture radar (SAR) images is of great importance for SAR scene interpretation and application. However, the presence of intricate spatial structural patterns and complex statistical nature makes SAR image classification a challenging task, especially in the case of limited labeled SAR data. This paper proposes a novel HR SAR image classification method, using a multi-scale deep feature fusion network and covariance pooling manifold network (MFFN-CPMN). MFFN-CPMN combines the advantages of local spatial features and global statistical properties and considers the multi-feature information fusion of SAR images in representation learning. First, we propose a Gabor-filtering-based multi-scale feature fusion network (MFFN) to capture the spatial pattern and get the discriminative features of SAR images. The MFFN belongs to a deep convolutional neural network (CNN). To make full use of a large amount of unlabeled data, the weights of each layer of MFFN are optimized by unsupervised denoising dual-sparse encoder. Moreover, the feature fusion strategy in MFFN can effectively exploit the complementary information between different levels and different scales. Second, we utilize a covariance pooling manifold network to extract further the global second-order statistics of SAR images over the fusional feature maps. Finally, the obtained covariance descriptor is more distinct for various land covers. Experimental results on four HR SAR images demonstrate the effectiveness of the proposed method and achieve promising results over other related algorithms.

scholarly journals Learning conditional photometric stereo with high-resolution features

2021 ◽  
Vol 8 (1) ◽  
pp. 105-118
Author(s):  
Yakun Ju ◽  
Yuxin Peng ◽  
Muwei Jian ◽  
Feng Gao ◽  
Junyu Dong
Keyword(s):  
Neural Networks ◽  
High Resolution ◽  
Feature Fusion ◽  
Low Frequency ◽  
Photometric Stereo ◽  
Surface Orientation ◽  
Smooth Functions ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
Deep Feature Extraction

AbstractPhotometric stereo aims to reconstruct 3D geometry by recovering the dense surface orientation of a 3D object from multiple images under differing illumination. Traditional methods normally adopt simplified reflectance models to make the surface orientation computable. However, the real reflectances of surfaces greatly limit applicability of such methods to real-world objects. While deep neural networks have been employed to handle non-Lambertian surfaces, these methods are subject to blurring and errors, especially in high-frequency regions (such as crinkles and edges), caused by spectral bias: neural networks favor low-frequency representations so exhibit a bias towards smooth functions. In this paper, therefore, we propose a self-learning conditional network with multi-scale features for photometric stereo, avoiding blurred reconstruction in such regions. Our explorations include: (i) a multi-scale feature fusion architecture, which keeps high-resolution representations and deep feature extraction, simultaneously, and (ii) an improved gradient-motivated conditionally parameterized convolution (GM-CondConv) in our photometric stereo network, with different combinations of convolution kernels for varying surfaces. Extensive experiments on public benchmark datasets show that our calibrated photometric stereo method outperforms the state-of-the-art.

scholarly journals Classification of benign and malignant pulmonary nodules based on multi-scale feature fusion

2020 ◽  
Vol 37 (4) ◽  
pp. 417-424
Author(s):  
Junhua GU ◽  
Zheran SUN ◽  
Feng WANG ◽  
Yongjun QI ◽  
Yajuan ZHANG
Keyword(s):  
Feature Fusion ◽  
Pulmonary Nodules ◽  
Scale Feature ◽  
Multi Scale

scholarly journals 3D multi-scale deep convolutional neural networks for pulmonary nodule detection

PLoS ONE ◽  
2021 ◽  
Vol 16 (1) ◽  
pp. e0244406
Author(s):  
Haixin Peng ◽  
Huacong Sun ◽  
Yanfei Guo
Keyword(s):  
Neural Networks ◽  
Pulmonary Nodule ◽  
False Positive ◽  
Rapid Development ◽  
Pulmonary Nodules ◽  
Multi Scale ◽  
False Positive Reduction ◽  
Prediction Probability ◽  
Nodule Detection ◽  
Pulmonary Nodule Detection

With the rapid development of big data and artificial intelligence technology, computer-aided pulmonary nodule detection based on deep learning has achieved some successes. However, the sizes of pulmonary nodules vary greatly, and the pulmonary nodules have visual similarity with structures such as blood vessels and shadows around pulmonary nodules, which make the quick and accurate detection of pulmonary nodules in CT image still a challenging task. In this paper, we propose two kinds of 3D multi-scale deep convolution neural networks for nodule candidate detection and false positive reduction respectively. Among them, the nodule candidate detection network consists of two parts: 1) the backbone network part Res2SENet, which is used to extract multi-scale feature information of pulmonary nodules, it is composed of the multi-scale Res2Net modules of multiple available receptive fields at a granular level and the squeeze-and-excitation units; 2) the detection part, which uses a region proposal network structure to determine region candidates, and introduces context enhancement module and spatial attention module to improve detection performance. The false positive reduction network, also composed of the multi-scale Res2Net modules and the squeeze-and-excitation units, can further classify the nodule candidates generated by the nodule candidate detection network and screen out the ground truth positive nodules. Finally, the prediction probability generated by the nodule candidate detection network is weighted average with the prediction probability generated by the false positive reduction network to obtain the final results. The experimental results on the publicly available LUNA16 dataset showed that the proposed method has a superior ability to detect pulmonary nodules in CT images.

scholarly journals Controllably Deep Supervision and Multi-Scale Feature Fusion Network for Cloud and Snow Detection Based on Medium- and High-Resolution Imagery Dataset

2021 ◽  
Vol 13 (23) ◽  
pp. 4805
Author(s):  
Guangbin Zhang ◽  
Xianjun Gao ◽  
Yuanwei Yang ◽  
Mingwei Wang ◽  
Shuhao Ran
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
High Resolution ◽  
Feature Fusion ◽  
Detection Accuracy ◽  
Learning Methods ◽  
Training Requirements ◽  
Scale Feature ◽  
Multi Scale ◽  
Hinge Loss

Clouds and snow in remote sensing imageries cover underlying surface information, reducing image availability. Moreover, they interact with each other, decreasing the cloud and snow detection accuracy. In this study, we propose a convolutional neural network for cloud and snow detection, named the cloud and snow detection network (CSD-Net). It incorporates the multi-scale feature fusion module (MFF) and the controllably deep supervision and feature fusion structure (CDSFF). MFF can capture and aggregate features at various scales, ensuring that the extracted high-level semantic features of clouds and snow are more distinctive. CDSFF can provide a deeply supervised mechanism with hinge loss and combine information from adjacent layers to gain more representative features. It ensures the gradient flow is more oriented and error-less, while retaining more effective information. Additionally, a high-resolution cloud and snow dataset based on WorldView2 (CSWV) was created and released. This dataset meets the training requirements of deep learning methods for clouds and snow in high-resolution remote sensing images. Based on the datasets with varied resolutions, CSD-Net is compared to eight state-of-the-art deep learning methods. The experiment results indicate that CSD-Net has an excellent detection accuracy and efficiency. Specifically, the mean intersection over the union (MIoU) of CSD-Net is the highest in the corresponding experiment. Furthermore, the number of parameters in our proposed network is just 7.61 million, which is the lowest of the tested methods. It only has 88.06 GFLOPs of floating point operations, which is less than the U-Net, DeepLabV3+, PSPNet, SegNet-Modified, MSCFF, and GeoInfoNet. Meanwhile, CSWV has a higher annotation quality since the same method can obtain a greater accuracy on it.

scholarly journals A DETECTION METHOD OF ARTIFICIAL AREA FROM HIGH RESOLUTION REMOTE SENSING IMAGES BASED ON MULTI SCALE AND MULTI FEATURE FUSION

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 387-392
Author(s):  
P. Li ◽  
X. Hu ◽  
Y. Hu ◽  
Y. Ding ◽  
L. Wang ◽  
...  
Keyword(s):  
Remote Sensing ◽  
High Resolution ◽  
Multiple Scales ◽  
Feature Fusion ◽  
Visual Saliency ◽  
Geometric Feature ◽  
Remote Sensing Images ◽  
Multi Scale ◽  
Urban City ◽  
Intensity Map

In order to solve the problem of automatic detection of artificial objects in high resolution remote sensing images, a method for detection of artificial areas in high resolution remote sensing images based on multi-scale and multi feature fusion is proposed. Firstly, the geometric features such as corner, straight line and right angle are extracted from the original resolution, and the pseudo corner points, pseudo linear features and pseudo orthogonal angles are filtered out by the self-constraint and mutual restraint between them. Then the radiation intensity map of the image with high geometric characteristics is obtained by the linear inverse distance weighted method. Secondly, the original image is reduced to multiple scales and the visual saliency image of each scale is obtained by adaptive weighting of the orthogonal saliency, the local brightness and contrast which are calculated at the corresponding scale. Then the final visual saliency image is obtained by fusing all scales’ visual saliency images. Thirdly, the visual saliency images of artificial areas based on multi scales and multi features are obtained by fusing the geometric feature energy intensity map and visual saliency image obtained in previous decision level. Finally, the artificial areas can be segmented based on the method called OTSU. Experiments show that the method in this paper not only can detect large artificial areas such as urban city, residential district, but also detect the single family house in the countryside correctly. The detection rate of artificial areas reached 92 %.

Sign in / Sign up

Export Citation Format

Share Document