A Lightweight Spectral–Spatial Feature Extraction and Fusion Network for Hyperspectral Image Classification

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Download Full-text

A Multi-Scale and Multi-Level Spectral-Spatial Feature Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12010125 ◽

2020 ◽

Vol 12 (1) ◽

pp. 125 ◽

Cited By ~ 5

Author(s):

Mu ◽

Guo ◽

Liu

Keyword(s):

Neural Network ◽

Hyperspectral Image ◽

Feature Fusion ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Multi Scale ◽

Spatial Features ◽

Multi Level

Extracting spatial and spectral features through deep neural networks has become an effective means of classification of hyperspectral images. However, most networks rarely consider the extraction of multi-scale spatial features and cannot fully integrate spatial and spectral features. In order to solve these problems, this paper proposes a multi-scale and multi-level spectral-spatial feature fusion network (MSSN) for hyperspectral image classification. The network uses the original 3D cube as input data and does not need to use feature engineering. In the MSSN, using different scale neighborhood blocks as the input of the network, the spectral-spatial features of different scales can be effectively extracted. The proposed 3D–2D alternating residual block combines the spectral features extracted by the three-dimensional convolutional neural network (3D-CNN) with the spatial features extracted by the two-dimensional convolutional neural network (2D-CNN). It not only achieves the fusion of spectral features and spatial features but also achieves the fusion of high-level features and low-level features. Experimental results on four hyperspectral datasets show that this method is superior to several state-of-the-art classification methods for hyperspectral images.

Download Full-text

HYPERSPECTRAL IMAGE CLASSIFICATION USING RESIDUAL 2D AND 3D CONVOLUTIONAL NEURAL NETWORK JOINT ATTENTION MODEL

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliv-m-3-2021-187-2021 ◽

2021 ◽

Vol XLIV-M-3-2021 ◽

pp. 187-193

Author(s):

Q. Yuan ◽

Y. Ang ◽

H. Z. M. Shafri

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Features ◽

3D Cnn ◽

2D And 3D

Abstract. Hyperspectral image classification (HSIC) is a challenging task in remote sensing data analysis, which has been applied in many domains for better identification and inspection of the earth surface by extracting spectral and spatial information. The combination of abundant spectral features and accurate spatial information can improve classification accuracy. However, many traditional methods are based on handcrafted features, which brings difficulties for multi-classification tasks due to spectral intra-class heterogeneity and similarity of inter-class. The deep learning algorithm, especially the convolutional neural network (CNN), has been perceived promising feature extractor and classification for processing hyperspectral remote sensing images. Although 2D CNN can extract spatial features, the specific spectral properties are not used effectively. While 3D CNN has the capability for them, but the computational burden increases as stacking layers. To address these issues, we propose a novel HSIC framework based on the residual CNN network by integrating the advantage of 2D and 3D CNN. First, 3D convolutions focus on extracting spectral features with feature recalibration and refinement by channel attention mechanism. The 2D depth-wise separable convolution approach with different size kernels concentrates on obtaining multi-scale spatial features and reducing model parameters. Furthermore, the residual structure optimizes the back-propagation for network training. The results and analysis of extensive HSIC experiments show that the proposed residual 2D-3D CNN network can effectively extract spectral and spatial features and improve classification accuracy.

Download Full-text

Multiscale Weighted Adjacent Superpixel-Based Composite Kernel for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13040820 ◽

2021 ◽

Vol 13 (4) ◽

pp. 820

Author(s):

Yaokang Zhang ◽

Yunjie Chen

Keyword(s):

Image Classification ◽

Hyperspectral Image ◽

Kernel Method ◽

Superior Performance ◽

Spectral Features ◽

Classification Methods ◽

Hyperspectral Image Classification ◽

Superpixel Segmentation ◽

Spatial Features ◽

Composite Kernel

This paper presents a composite kernel method (MWASCK) based on multiscale weighted adjacent superpixels (ASs) to classify hyperspectral image (HSI). The MWASCK adequately exploits spatial-spectral features of weighted adjacent superpixels to guarantee that more accurate spectral features can be extracted. Firstly, we use a superpixel segmentation algorithm to divide HSI into multiple superpixels. Secondly, the similarities between each target superpixel and its ASs are calculated to construct the spatial features. Finally, a weighted AS-based composite kernel (WASCK) method for HSI classification is proposed. In order to avoid seeking for the optimal superpixel scale and fuse the multiscale spatial features, the MWASCK method uses multiscale weighted superpixel neighbor information. Experiments from two real HSIs indicate that superior performance of the WASCK and MWASCK methods compared with some popular classification methods.

Download Full-text

Hyperspectral Image Classification Based on Multi-Scale Residual Network with Attention Mechanism

Remote Sensing ◽

10.3390/rs13030335 ◽

2021 ◽

Vol 13 (3) ◽

pp. 335

Author(s):

Yuhao Qing ◽

Wenyi Liu

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Image Classification ◽

Classification Accuracy ◽

Hyperspectral Image ◽

Principal Component ◽

Hyperspectral Image Classification ◽

Deep Network ◽

Multi Scale

In recent years, image classification on hyperspectral imagery utilizing deep learning algorithms has attained good results. Thus, spurred by that finding and to further improve the deep learning classification accuracy, we propose a multi-scale residual convolutional neural network model fused with an efficient channel attention network (MRA-NET) that is appropriate for hyperspectral image classification. The suggested technique comprises a multi-staged architecture, where initially the spectral information of the hyperspectral image is reduced into a two-dimensional tensor, utilizing a principal component analysis (PCA) scheme. Then, the constructed low-dimensional image is input to our proposed ECA-NET deep network, which exploits the advantages of its core components, i.e., multi-scale residual structure and attention mechanisms. We evaluate the performance of the proposed MRA-NET on three public available hyperspectral datasets and demonstrate that, overall, the classification accuracy of our method is 99.82 %, 99.81%, and 99.37, respectively, which is higher compared to the corresponding accuracy of current networks such as 3D convolutional neural network (CNN), three-dimensional residual convolution structure (RES-3D-CNN), and space–spectrum joint deep network (SSRN).

Download Full-text

A Spectral Spatial Attention Fusion with Deformable Convolutional Residual Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13183590 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3590

Author(s):

Tianyu Zhang ◽

Cuiping Shi ◽

Diling Liao ◽

Liguo Wang

Keyword(s):

Image Classification ◽

Spatial Attention ◽

Hyperspectral Image ◽

Spectral Feature ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Residual Network ◽

Spatial Features ◽

Training Samples

Convolutional neural networks (CNNs) have exhibited excellent performance in hyperspectral image classification. However, due to the lack of labeled hyperspectral data, it is difficult to achieve high classification accuracy of hyperspectral images with fewer training samples. In addition, although some deep learning techniques have been used in hyperspectral image classification, due to the abundant information of hyperspectral images, the problem of insufficient spatial spectral feature extraction still exists. To address the aforementioned issues, a spectral–spatial attention fusion with a deformable convolution residual network (SSAF-DCR) is proposed for hyperspectral image classification. The proposed network is composed of three parts, and each part is connected sequentially to extract features. In the first part, a dense spectral block is utilized to reuse spectral features as much as possible, and a spectral attention block that can refine and optimize the spectral features follows. In the second part, spatial features are extracted and selected by a dense spatial block and attention block, respectively. Then, the results of the first two parts are fused and sent to the third part, and deep spatial features are extracted by the DCR block. The above three parts realize the effective extraction of spectral–spatial features, and the experimental results for four commonly used hyperspectral datasets demonstrate that the proposed SSAF-DCR method is superior to some state-of-the-art methods with very few training samples.

Download Full-text

Hyperspectral Image Classification Based on Two-Branch Spectral–Spatial-Feature Attention Network

Remote Sensing ◽

10.3390/rs13214262 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4262

Author(s):

Hanjie Wu ◽

Dan Li ◽

Yujian Wang ◽

Xiaojun Li ◽

Fanqiang Kong ◽

...

Keyword(s):

Hyperspectral Image ◽

Classification Performance ◽

Hybrid Architecture ◽

Classification Methods ◽

Attention Network ◽

Spatial Feature ◽

Spatial Features ◽

Original Dataset ◽

Training Samples ◽

Feature Attention

Although most of deep-learning-based hyperspectral image (HSI) classification methods achieve great performance, there still remains a challenge to utilize small-size training samples to remarkably enhance the classification accuracy. To tackle this challenge, a novel two-branch spectral–spatial-feature attention network (TSSFAN) for HSI classification is proposed in this paper. Firstly, two inputs with different spectral dimensions and spatial sizes are constructed, which can not only reduce the redundancy of the original dataset but also accurately explore the spectral and spatial features. Then, we design two parallel 3DCNN branches with attention modules, in which one focuses on extracting spectral features and adaptively learning the more discriminative spectral channels, and the other focuses on exploring spatial features and adaptively learning the more discriminative spatial structures. Next, the feature attention module is constructed to automatically adjust the weights of different features based on their contributions for classification to remarkably improve the classification performance. Finally, we design the hybrid architecture of 3D–2DCNN to acquire the final classification result, which can significantly decrease the sophistication of the network. Experimental results on three HSI datasets indicate that our presented TSSFAN method outperforms several of the most advanced classification methods.

Download Full-text

HSI-GCN: hyperspectral image classification algorithm based on Gabor convolutional networks

World Journal of Engineering ◽

10.1108/wje-09-2020-0460 ◽

2021 ◽

Vol ahead-of-print (ahead-of-print) ◽

Author(s):

Houari Youcef Moudjib ◽

Duan Haibin ◽

Baochang Zhang ◽

Mohammed Salah Ahmed Ghaleb

Keyword(s):

Image Classification ◽

Classification Accuracy ◽

Hyperspectral Image ◽

Feature Learning ◽

Hyperspectral Data ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Content Type ◽

Convolutional Networks ◽

The Rich

Purpose Hyperspectral imaging (HSI) systems are becoming potent technologies for computer vision tasks due to the rich information they uncover, where each substance exhibits a distinct spectral distribution. Although the high spectral dimensionality of the data empowers feature learning, the joint spatial–spectral features have not been well explored yet. Gabor convolutional networks (GCNs) incorporate Gabor filters into a deep convolutional neural network (CNN) to extract discriminative features of different orientations and frequencies. To the best if the authors’ knowledge, this paper introduces the exploitation of GCNs for hyperspectral image classification (HSI-GCN) for the first time. HSI-GCN is able to extract deep joint spatial–spectral features more rapidly and accurately despite the shortage of training samples. The authors thoroughly evaluate the effectiveness of used method on different hyperspectral data sets, where promising results and high classification accuracy have been achieved compared to the previously proposed CNN-based and Gabor-based methods. Design/methodology/approach The authors have implemented the new algorithm of Gabor convolution network on the hyperspectral images for classification purposes. Findings Implementing the new GCN has shown unexpectable results with an excellent classification accuracy. Originality/value To the best of the authors’ knowledge, this work is the first one that implements this approach.

Download Full-text

Double-Branch Network with Pyramidal Convolution and Iterative Attention for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13071403 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1403

Author(s):

Hao Shi ◽

Guo Cao ◽

Zixian Ge ◽

Youqiang Zhang ◽

Peng Fu

Keyword(s):

Hyperspectral Image ◽

State Of The Art ◽

Classification Performance ◽

Attention Mechanism ◽

Hyperspectral Images ◽

Hyperspectral Image Classification ◽

Excellent Performance ◽

Common Procedure ◽

Spatial Features ◽

First Choice

Deep-learning methods, especially convolutional neural networks (CNN), have become the first choice for hyperspectral image (HSI) classification to date. It is a common procedure that small cubes are cropped from hyperspectral images and then fed into CNNs. However, standard CNNs find it difficult to extract discriminative spectral–spatial features. How to obtain finer spectral–spatial features to improve the classification performance is now a hot topic of research. In this regard, the attention mechanism, which has achieved excellent performance in other computer vision, holds the exciting prospect. In this paper, we propose a double-branch network consisting of a novel convolution named pyramidal convolution (PyConv) and an iterative attention mechanism. Each branch concentrates on exploiting spectral or spatial features with different PyConvs, supplemented by the attention module for refining the feature map. Experimental results demonstrate that our model can yield competitive performance compared to other state-of-the-art models.

Download Full-text

Multipath Residual Network for Spectral-Spatial Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs11161896 ◽

2019 ◽

Vol 11 (16) ◽

pp. 1896 ◽

Cited By ~ 4

Author(s):

Zhe Meng ◽

Lingling Li ◽

Xu Tang ◽

Zhixi Feng ◽

Licheng Jiao ◽

...

Keyword(s):

Hyperspectral Image ◽

State Of The Art ◽

Gradient Flow ◽

Hyperspectral Data ◽

Data Sets ◽

Classification Models ◽

Classification Methods ◽

Hyperspectral Image Classification ◽

Residual Network ◽

Residual Functions

Convolutional neural networks (CNNs) have recently shown outstanding capability for hyperspectral image (HSI) classification. In this work, a novel CNN model is proposed, which is wider than other existing deep learning-based HSI classification models. Based on the fact that very deep residual networks (ResNets) behave like an ensemble of relatively shallow networks, our proposed network, called multipath ResNet (MPRN), employs multiple residual functions in the residual blocks to make the network wider, rather than deeper. The proposed network consists of shorter-medium paths for efficient gradient flow and replaces the stacking of multiple residual blocks in ResNet with fewer residual blocks but more parallel residual functions in each of it. Experimental results on three real hyperspectral data sets demonstrate the superiority of the proposed method over several state-of-the-art classification methods.

Download Full-text

Double-Branch Multi-Attention Mechanism Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs11111307 ◽

2019 ◽

Vol 11 (11) ◽

pp. 1307 ◽

Cited By ~ 11

Author(s):

Wenping Ma ◽

Qifan Yang ◽

Yue Wu ◽

Wei Zhao ◽

Xiangrong Zhang

Keyword(s):

Image Classification ◽

Spatial Information ◽

Hyperspectral Image ◽

State Of The Art ◽

Attention Mechanism ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Types Of Information ◽

Different Characteristics ◽

Better Than

Recently, Hyperspectral Image (HSI) classification has gradually been getting attention from more and more researchers. HSI has abundant spectral and spatial information; thus, how to fuse these two types of information is still a problem worth studying. In this paper, to extract spectral and spatial feature, we propose a Double-Branch Multi-Attention mechanism network (DBMA) for HSI classification. This network has two branches to extract spectral and spatial feature respectively which can reduce the interference between the two types of feature. Furthermore, with respect to the different characteristics of these two branches, two types of attention mechanism are applied in the two branches respectively, which ensures to extract more discriminative spectral and spatial feature. The extracted features are then fused for classification. A lot of experiment results on three hyperspectral datasets shows that the proposed method performs better than the state-of-the-art method.

Download Full-text