Densely Connected Pyramidal Dilated Convolutional Network for Hyperspectral Image Classification

Recently, with the extensive application of deep learning techniques in the hyperspectral image (HSI) field, particularly convolutional neural network (CNN), the research of HSI classification has stepped into a new stage. To avoid the problem that the receptive field of naive convolution is small, the dilated convolution is introduced into the field of HSI classification. However, the dilated convolution usually generates blind spots in the receptive field, resulting in discontinuous spatial information obtained. In order to solve the above problem, a densely connected pyramidal dilated convolutional network (PDCNet) is proposed in this paper. Firstly, a pyramidal dilated convolutional (PDC) layer integrates different numbers of sub-dilated convolutional layers is proposed, where the dilated factor of the sub-dilated convolution increases exponentially, achieving multi-sacle receptive fields. Secondly, the number of sub-dilated convolutional layers increases in a pyramidal pattern with the depth of the network, thereby capturing more comprehensive hyperspectral information in the receptive field. Furthermore, a feature fusion mechanism combining pixel-by-pixel addition and channel stacking is adopted to extract more abstract spectral–spatial features. Finally, in order to reuse the features of the previous layers more effectively, dense connections are applied in densely pyramidal dilated convolutional (DPDC) blocks. Experiments on three well-known HSI datasets indicate that PDCNet proposed in this paper has good classification performance compared with other popular models.

Download Full-text

Multiscale Spatial-Spectral Convolutional Network with Image-Based Framework for Hyperspectral Imagery Classification

Remote Sensing ◽

10.3390/rs11192220 ◽

2019 ◽

Vol 11 (19) ◽

pp. 2220 ◽

Cited By ~ 5

Author(s):

Ximin Cui ◽

Ke Zheng ◽

Lianru Gao ◽

Bing Zhang ◽

Dong Yang ◽

...

Keyword(s):

Receptive Field ◽

Hyperspectral Image ◽

Receptive Fields ◽

Feature Representation ◽

Superior Performance ◽

Processing Efficiency ◽

Convolutional Network ◽

Spatial Features ◽

Classification Framework ◽

Field Feature

Jointly using spatial and spectral information has been widely applied to hyperspectral image (HSI) classification. Especially, convolutional neural networks (CNN) have gained attention in recent years due to their detailed representation of features. However, most of CNN-based HSI classification methods mainly use patches as input classifier. This limits the range of use for spatial neighbor information and reduces processing efficiency in training and testing. To overcome this problem, we propose an image-based classification framework that is efficient and straightforward. Based on this framework, we propose a multiscale spatial-spectral CNN for HSIs (HyMSCN) to integrate both multiple receptive fields fused features and multiscale spatial features at different levels. The fused features are exploited using a lightweight block called the multiple receptive field feature block (MRFF), which contains various types of dilation convolution. By fusing multiple receptive field features and multiscale spatial features, the HyMSCN has comprehensive feature representation for classification. Experimental results from three real hyperspectral images prove the efficiency of the proposed framework. The proposed method also achieves superior performance for HSI classification.

Download Full-text

A 3D-2D Multibranch Feature Fusion and Dense Attention Network for Hyperspectral Image Classification

Micromachines ◽

10.3390/mi12101271 ◽

2021 ◽

Vol 12 (10) ◽

pp. 1271

Author(s):

Hongmin Gao ◽

Yiyan Zhang ◽

Yunfei Zhang ◽

Zhonghao Chen ◽

Chenming Li ◽

...

Keyword(s):

Image Classification ◽

Hyperspectral Image ◽

Feature Fusion ◽

Receptive Fields ◽

Classification Performance ◽

Redundant Information ◽

Hyperspectral Image Classification ◽

Training Process ◽

Attention Network ◽

Spatial Features

In recent years, hyperspectral image classification (HSI) has attracted considerable attention. Various methods based on convolution neural networks have achieved outstanding classification results. However, most of them exited the defects of underutilization of spectral-spatial features, redundant information, and convergence difficulty. To address these problems, a novel 3D-2D multibranch feature fusion and dense attention network are proposed for HSI classification. Specifically, the 3D multibranch feature fusion module integrates multiple receptive fields in spatial and spectral dimensions to obtain shallow features. Then, a 2D densely connected attention module consists of densely connected layers and spatial-channel attention block. The former is used to alleviate the gradient vanishing and enhance the feature reuse during the training process. The latter emphasizes meaningful features and suppresses the interfering information along the two principal dimensions: channel and spatial axes. The experimental results on four benchmark hyperspectral images datasets demonstrate that the model can effectively improve the classification performance with great robustness.

Download Full-text

Modifications of the Multi-Layer Perceptron for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13173547 ◽

2021 ◽

Vol 13 (17) ◽

pp. 3547

Author(s):

Xin He ◽

Yushi Chen

Keyword(s):

Spatial Information ◽

Hyperspectral Image ◽

Classification Performance ◽

Multi Layer Perceptron ◽

Feature Mapping ◽

Spatial Feature ◽

Spatial Features ◽

Linear Layer ◽

Input Sample ◽

Good Classification Performance

Recently, many convolutional neural network (CNN)-based methods have been proposed to tackle the classification task of hyperspectral images (HSI). In fact, CNN has become the de-facto standard for HSI classification. It seems that the traditional neural networks such as multi-layer perceptron (MLP) are not competitive for HSI classification. However, in this study, we try to prove that the MLP can achieve good classification performance of HSI if it is properly designed and improved. The proposed Modified-MLP for HSI classification contains two special parts: spectral–spatial feature mapping and spectral–spatial information mixing. Specifically, for spectral–spatial feature mapping, each input sample of HSI is divided into a sequence of 3D patches with fixed length and then a linear layer is used to map the 3D patches to spectral–spatial features. For spectral–spatial information mixing, all the spectral–spatial features within a single sample are feed into the solely MLP architecture to model the spectral–spatial information across patches for following HSI classification. Furthermore, to obtain the abundant spectral–spatial information with different scales, Multiscale-MLP is proposed to aggregate neighboring patches with multiscale shapes for acquiring abundant spectral–spatial information. In addition, the Soft-MLP is proposed to further enhance the classification performance by applying soft split operation, which flexibly capture the global relations of patches at different positions in the input HSI sample. Finally, label smoothing is introduced to mitigate the overfitting problem in the Soft-MLP (Soft-MLP-L), which greatly improves the classification performance of MLP-based method. The proposed Modified-MLP, Multiscale-MLP, Soft-MLP, and Soft-MLP-L are tested on the three widely used hyperspectral datasets. The proposed Soft-MLP-L leads to the highest OA, which outperforms CNN by 5.76%, 2.55%, and 2.5% on the Salinas, Pavia, and Indian Pines datasets, respectively. The proposed Modified-MLP, Multiscale-MLP, and Soft-MLP are tested on the three widely used hyperspectral datasets (i.e., Salinas, Pavia, and Indian Pines). The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the MLP-based methods are still competitive for HSI classification.

Download Full-text

Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution

Remote Sensing ◽

10.3390/rs12101660 ◽

2020 ◽

Vol 12 (10) ◽

pp. 1660 ◽

Cited By ~ 2

Author(s):

Qiang Li ◽

Qi Wang ◽

Xuelong Li

Keyword(s):

Spatial Information ◽

Hyperspectral Image ◽

Feature Fusion ◽

Super Resolution ◽

Superior Performance ◽

Spectral Information ◽

Great Success ◽

Convolutional Network ◽

Training Time ◽

Image Super Resolution

Deep learning-based hyperspectral image super-resolution (SR) methods have achieved great success recently. However, there are two main problems in the previous works. One is to use the typical three-dimensional convolution analysis, resulting in more parameters of the network. The other is not to pay more attention to the mining of hyperspectral image spatial information, when the spectral information can be extracted. To address these issues, in this paper, we propose a mixed convolutional network (MCNet) for hyperspectral image super-resolution. We design a novel mixed convolutional module (MCM) to extract the potential features by 2D/3D convolution instead of one convolution, which enables the network to more mine spatial features of hyperspectral image. To explore the effective features from 2D unit, we design the local feature fusion to adaptively analyze from all the hierarchical features in 2D units. In 3D unit, we employ spatial and spectral separable 3D convolution to extract spatial and spectral information, which reduces unaffordable memory usage and training time. Extensive evaluations and comparisons on three benchmark datasets demonstrate that the proposed approach achieves superior performance in comparison to existing state-of-the-art methods.

Download Full-text

Variational Generative Adversarial Network with Crossed Spatial and Spectral Interactions for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13163131 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3131

Author(s):

Zhongwei Li ◽

Xue Zhu ◽

Ziqi Xin ◽

Fangming Guo ◽

Xingshuai Cui ◽

...

Keyword(s):

Image Classification ◽

Spatial Information ◽

Hyperspectral Image ◽

Classification Performance ◽

Generative Adversarial Networks ◽

Hyperspectral Image Classification ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Spatial Features ◽

Virtual Samples

Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have been widely used in hyperspectral image classification (HSIC) tasks. However, the generated HSI virtual samples by VAEs are often ambiguous, and GANs are prone to the mode collapse, which lead the poor generalization abilities ultimately. Moreover, most of these models only consider the extraction of spectral or spatial features. They fail to combine the two branches interactively and ignore the correlation between them. Consequently, the variational generative adversarial network with crossed spatial and spectral interactions (CSSVGAN) was proposed in this paper, which includes a dual-branch variational Encoder to map spectral and spatial information to different latent spaces, a crossed interactive Generator to improve the quality of generated virtual samples, and a Discriminator stuck with a classifier to enhance the classification performance. Combining these three subnetworks, the proposed CSSVGAN achieves excellent classification by ensuring the diversity and interacting spectral and spatial features in a crossed manner. The superior experimental results on three datasets verify the effectiveness of this method.

Download Full-text

Multi-Feature Manifold Discriminant Analysis for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs11060651 ◽

2019 ◽

Vol 11 (6) ◽

pp. 651 ◽

Cited By ~ 10

Author(s):

Hong Huang ◽

Zhengying Li ◽

Yinsong Pan

Keyword(s):

Discriminant Analysis ◽

Spatial Information ◽

Hyperspectral Image ◽

Feature Fusion ◽

Local Binary Patterns ◽

Hyperspectral Data ◽

Spectral Features ◽

Spatial Features ◽

Combined Features ◽

Low Dimensional

Hyperspectral image (HSI) provides both spatial structure and spectral information for classification, but many traditional methods simply concatenate spatial features and spectral features together that usually lead to the curse-of-dimensionality and unbalanced representation of different features. To address this issue, a new dimensionality reduction (DR) method, termed multi-feature manifold discriminant analysis (MFMDA), was proposed in this paper. At first, MFMDA explores local binary patterns (LBP) operator to extract textural features for encoding the spatial information in HSI. Then, under graph embedding framework, the intrinsic and penalty graphs of LBP and spectral features are constructed to explore the discriminant manifold structure in both spatial and spectral domains, respectively. After that, a new spatial-spectral DR model for multi-feature fusion is built to extract discriminant spatial-spectral combined features, and it not only preserves the similarity relationship between spectral features and LBP features but also possesses strong discriminating ability in the low-dimensional embedding space. Experiments on Indian Pines, Heihe and Pavia University (PaviaU) hyperspectral data sets demonstrate that the proposed MFMDA method performs significantly better than some state-of-the-art methods using only single feature or simply stacking spectral features and spatial features together, and the classification accuracies of it can reach 95.43%, 97.19% and 96.60%, respectively.

Download Full-text

Hybrid Dilated Convolution with Multi-Scale Residual Fusion Network for Hyperspectral Image Classification

Micromachines ◽

10.3390/mi12050545 ◽

2021 ◽

Vol 12 (5) ◽

pp. 545

Author(s):

Chenming Li ◽

Zelin Qiu ◽

Xueying Cao ◽

Zhonghao Chen ◽

Hongmin Gao ◽

...

Keyword(s):

Image Classification ◽

Spatial Information ◽

Hyperspectral Image ◽

Local Information ◽

Hyperspectral Image Classification ◽

Multi Scale ◽

Dilated Convolution ◽

Spatial Features ◽

Network Methods ◽

State Of Art

The convolutional neural network (CNN) has been proven to have better performance in hyperspectral image (HSI) classification than traditional methods. Traditional CNN on hyperspectral image classification is used to pay more attention to spectral features and ignore spatial information. In this paper, a new HSI model called local and hybrid dilated convolution fusion network (LDFN) was proposed, which fuses the local information of details and rich spatial features by expanding the perception field. The details of our local and hybrid dilated convolution fusion network methods are as follows. First, many operations are selected, such as standard convolution, average pooling, dropout and batch normalization. Then, fusion operations of local and hybrid dilated convolution are included to extract rich spatial-spectral information. Last, different convolution layers are gathered into residual fusion networks and finally input into the softmax layer to classify. Three widely hyperspectral datasets (i.e., Salinas, Pavia University and Indian Pines) have been used in the experiments, which show that LDFN outperforms state-of-art classifiers.

Download Full-text

3D Spatial Pyramid Dilated Network for Pulmonary Nodule Classification

Symmetry ◽

10.3390/sym10090376 ◽

2018 ◽

Vol 10 (9) ◽

pp. 376 ◽

Cited By ~ 2

Author(s):

Guokai Zhang ◽

Xiao Liu ◽

Dandan Zhu ◽

Pengcheng He ◽

Lipeng Liang ◽

...

Keyword(s):

Pulmonary Nodule ◽

Early Stage ◽

Receptive Fields ◽

Small Diameter ◽

Pulmonary Nodules ◽

Lung Cancer Mortality ◽

Classification Performance ◽

Dilated Convolution ◽

Detailed Characteristic ◽

Spatial Pyramid

Lung cancer mortality is currently the highest among all kinds of fatal cancers. With the help of computer-aided detection systems, a timely detection of malignant pulmonary nodule at early stage could improve the patient survival rate efficiently. However, the sizes of the pulmonary nodules are usually various, and it is more difficult to detect small diameter nodules. The traditional convolution neural network uses pooling layers to reduce the resolution progressively, but it hampers the network’s ability to capture the tiny but vital features of the pulmonary nodules. To tackle this problem, we propose a novel 3D spatial pyramid dilated convolution network to classify the malignancy of the pulmonary nodules. Instead of using the pooling layers, we use 3D dilated convolution to learn the detailed characteristic information of the pulmonary nodules. Furthermore, we show that the fusion of multiple receptive fields from different dilated convolutions could further improve the classification performance of the model. Extensive experimental results demonstrate that our model achieves a better result with an accuracy of 88 . 6 % , which outperforms other state-of-the- art methods.

Download Full-text

Small-Scale Face Detection Based on Improved R-FCN

Applied Sciences ◽

10.3390/app10124177 ◽

2020 ◽

Vol 10 (12) ◽

pp. 4177

Author(s):

Chaowei Tang ◽

Shiyu Chen ◽

Xu Zhou ◽

Shuai Ruan ◽

Haotian Wen

Keyword(s):

Receptive Field ◽

Face Detection ◽

Feature Fusion ◽

Selection Method ◽

Local Information ◽

Small Scale ◽

Average Precision ◽

Convolutional Network ◽

High Layer ◽

Novel Method

Face detection is an important basic technique for face-related applications, such as face analysis, recognition, and reconstruction. Images in unconstrained scenes may contain many small-scale faces. The features that the detector can extract from small-scale faces are limited, which will cause missed detection and greatly reduce the precision of face detection. Therefore, this study proposes a novel method to detect small-scale faces based on region-based fully convolutional network (R-FCN). First, we propose a novel R-FCN framework with the ability of feature fusion and receptive field adaptation. Second, a bottom-up feature fusion branch is established to enrich the local information of high-layer features. Third, a receptive field adaptation block (RFAB) is proposed to ensure that the receptive field can be adaptively selected to strengthen the expression ability of features. Finally, we improve the anchor setting method and adopt soft non-maximum suppression (SoftNMS) as the selection method of candidate boxes. Experimental results show that average precision for small-scale face detection of R-FCN with feature fusion branch and RFAB (RFAB-f-R-FCN) is improved by 0.8%, 2.9%, and 11% on three subsets of Wider Face compared with that of R-FCN.

Download Full-text

From Local to Global: Class Feature Fused Fully Convolutional Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13245043 ◽

2021 ◽

Vol 13 (24) ◽

pp. 5043

Author(s):

Qian Liu ◽

Zebin Wu ◽

Xiuping Jia ◽

Yang Xu ◽

Zhihui Wei

Keyword(s):

Feature Extraction ◽

Spatial Information ◽

Hyperspectral Image ◽

Global Perspective ◽

Global Information ◽

Global Features ◽

Convolutional Network ◽

Local Networks ◽

Common Component ◽

Fully Convolutional Network

Current mainstream networks for hyperspectral image (HSI) classification employ image patches as inputs for feature extraction. Spatial information extraction is limited by the size of inputs, which makes networks unable to perform effective learning and reasoning from the global perspective. As a common component for capturing long-range dependencies, non-local networks with pixel-by-pixel information interaction bring unaffordable computational costs and information redundancy. To address the above issues, we propose a class feature fused fully convolutional network (CFF-FCN) with a local feature extraction block (LFEB) and a class feature fusion block (CFFB) to jointly utilize local and global information. LFEB based on dilated convolutions and reverse loop mechanism can acquire the local spectral–spatial features at multiple levels and deliver shallower layer features for coarse classification. CFFB calculates global class representation to enhance pixel features. Robust global information is propagated to every pixel with low computational cost. CFF-FCN considers a fully global class context and obtains more discriminative representation by concatenating high-level local features and re-integrated global features. Experimental results conducted on three real HSI data sets demonstrate that the proposed fully convolutional network is superior to multiple state-of-the-art deep learning-based approaches, especially in the case of a small number of training samples.

Download Full-text