DBMF: A Novel Method for Tree Species Fusion Classification Based on Multi-Source Images

Multi-source data remote sensing provides innovative technical support for tree species recognition. Tree species recognition is relatively poor despite noteworthy advancements in image fusion methods because the features from multi-source data for each pixel in the same region cannot be deeply exploited. In the present paper, a novel deep learning approach for hyperspectral imagery is proposed to improve accuracy for the classification of tree species. The proposed method, named the double branch multi-source fusion (DBMF) method, could more deeply determine the relationship between multi-source data and provide more effective information. The DBMF method does this by fusing spectral features extracted from a hyperspectral image (HSI) captured by the HJ-1A satellite and spatial features extracted from a multispectral image (MSI) captured by the Sentinel-2 satellite. The network has two branches in the spatial branch to avoid the risk of information loss, of which, sandglass blocks are embedded into a convolutional neural network (CNN) to extract the corresponding spatial neighborhood features from the MSI. Simultaneously, to make the useful spectral feature transfer more effective in the spectral branch, we employed bidirectional long short-term memory (Bi-LSTM) with a triple attention mechanism to extract the spectral features of each pixel in the HSI with low resolution. The feature information is fused to classify the tree species after the addition of a fusion activation function, which could allow the network to obtain more interactive information. Finally, the fusion strategy allows for the prediction of the full classification map of three study areas. Experimental results on a multi-source dataset show that DBMF has a significant advantage over other state-of-the-art frameworks.

Download Full-text

A Spectral Spatial Attention Fusion with Deformable Convolutional Residual Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13183590 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3590

Author(s):

Tianyu Zhang ◽

Cuiping Shi ◽

Diling Liao ◽

Liguo Wang

Keyword(s):

Image Classification ◽

Spatial Attention ◽

Hyperspectral Image ◽

Spectral Feature ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Residual Network ◽

Spatial Features ◽

Training Samples

Convolutional neural networks (CNNs) have exhibited excellent performance in hyperspectral image classification. However, due to the lack of labeled hyperspectral data, it is difficult to achieve high classification accuracy of hyperspectral images with fewer training samples. In addition, although some deep learning techniques have been used in hyperspectral image classification, due to the abundant information of hyperspectral images, the problem of insufficient spatial spectral feature extraction still exists. To address the aforementioned issues, a spectral–spatial attention fusion with a deformable convolution residual network (SSAF-DCR) is proposed for hyperspectral image classification. The proposed network is composed of three parts, and each part is connected sequentially to extract features. In the first part, a dense spectral block is utilized to reuse spectral features as much as possible, and a spectral attention block that can refine and optimize the spectral features follows. In the second part, spatial features are extracted and selected by a dense spatial block and attention block, respectively. Then, the results of the first two parts are fused and sent to the third part, and deep spatial features are extracted by the DCR block. The above three parts realize the effective extraction of spectral–spatial features, and the experimental results for four commonly used hyperspectral datasets demonstrate that the proposed SSAF-DCR method is superior to some state-of-the-art methods with very few training samples.

Download Full-text

Spatial-Spectral Network for Hyperspectral Image Classification: A 3-D CNN and Bi-LSTM Framework

Remote Sensing ◽

10.3390/rs13122353 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2353

Author(s):

Junru Yin ◽

Changsheng Qi ◽

Qiqiang Chen ◽

Jiantao Qu

Keyword(s):

Short Term Memory ◽

Hyperspectral Image ◽

Sequence Data ◽

Three Dimensional ◽

Spectral Feature ◽

Spectral Features ◽

Spectral Bands ◽

Current State ◽

Feature Extractor ◽

Lstm Network

Recently, deep learning methods based on the combination of spatial and spectral features have been successfully applied in hyperspectral image (HSI) classification. To improve the utilization of the spatial and spectral information from the HSI, this paper proposes a unified network framework using a three-dimensional convolutional neural network (3-D CNN) and a band grouping-based bidirectional long short-term memory (Bi-LSTM) network for HSI classification. In the framework, extracting spectral features is regarded as a procedure of processing sequence data, and the Bi-LSTM network acts as the spectral feature extractor of the unified network to fully exploit the close relationships between spectral bands. The 3-D CNN has a unique advantage in processing the 3-D data; therefore, it is used as the spatial-spectral feature extractor in this unified network. Finally, in order to optimize the parameters of both feature extractors simultaneously, the Bi-LSTM and 3-D CNN share a loss function to form a unified network. To evaluate the performance of the proposed framework, three datasets were tested for HSI classification. The results demonstrate that the performance of the proposed method is better than the current state-of-the-art HSI classification methods.

Download Full-text

Hierarchical Spatial-Spectral Feature Extraction with Long Short Term Memory (LSTM) for Mineral Identification Using Hyperspectral Imagery

Sensors ◽

10.3390/s20236854 ◽

2020 ◽

Vol 20 (23) ◽

pp. 6854

Author(s):

Huijie Zhao ◽

Kewang Deng ◽

Na Li ◽

Ziwei Wang ◽

Wei Wei

Keyword(s):

Feature Extraction ◽

Short Term Memory ◽

Spectral Feature ◽

Spectral Features ◽

Short Term ◽

Term Memory ◽

Spatial Features ◽

Proposed Model ◽

Long Short Term Memory ◽

Mineral Identification

Deep learning models are widely employed in hyperspectral image processing to integrate both spatial features and spectral features, but the correlations between them are rarely taken into consideration. However, in hyperspectral mineral identification, not only the spectral and spatial features of minerals need to be considered, but also the correlations between them are crucial to further promote identification accuracy. In this paper, we propose hierarchical spatial-spectral feature extraction with long short term memory (HSS-LSTM) to explore correlations between spatial features and spectral features and obtain hierarchical intrinsic features for mineral identification. In the proposed model, the fusion spatial-spectral feature is primarily extracted by stacking local spatial features obtained by a convolution neural network (CNN)-based model and spectral information together. To better exploit spatial features and spectral features, an LSTM-based model is proposed to capture correlations and obtain hierarchical features for accurate mineral identification. Specifically, the proposed model shares a uniform objective function, so that all the parameters in the network can be optimized in the meantime. Experimental results on the hyperspectral data collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) in the Nevada mining area show that HSS-LSTM achieves an overall accuracy of 94.70% and outperforms other commonly used identification methods.

Download Full-text

A Spatial-Spectral Feature Descriptor for Hyperspectral Image Matching

Remote Sensing ◽

10.3390/rs13234912 ◽

2021 ◽

Vol 13 (23) ◽

pp. 4912

Author(s):

Yang Yu ◽

Yong Ma ◽

Xiaoguang Mei ◽

Fan Fan ◽

Jun Huang ◽

...

Keyword(s):

Image Matching ◽

Hyperspectral Image ◽

Spectral Feature ◽

Feature Descriptor ◽

Spectral Features ◽

Grayscale Image ◽

Critical Problem ◽

Spatial Features ◽

Feature Descriptors ◽

Wide Range

Hyperspectral Images (HSIs) have been utilized in many fields which contain spatial and spectral features of objects simultaneously. Hyperspectral image matching is a fundamental and critical problem in a wide range of HSI applications. Feature descriptors for grayscale image matching are well studied, but few descriptors are elaborately designed for HSI matching. HSI descriptors, which should have made good use of the spectral feature, are essential in HSI matching tasks. Therefore, this paper presents a descriptor for HSI matching, called HOSG-SIFT, which ensembles spectral features with spatial features of objects. First, we obtain the grayscale image by dimensional reduction from HSI and apply it to extract keypoints and descriptors of spatial features. Second, the descriptors of spectral features are designed based on the histogram of the spectral gradient (HOSG), which effectively preserves the physical significance of the spectral profile. Third, we concatenate the spatial descriptors and spectral descriptors with the same weights into a new descriptor and apply it for HSI matching. Experimental results demonstrate that the proposed HOSG-SIFT performs superior against traditional feature descriptors.

Download Full-text

Random Collective Representation-Based Detector with Multiple Features for Hyperspectral Images

Remote Sensing ◽

10.3390/rs13040721 ◽

2021 ◽

Vol 13 (4) ◽

pp. 721

Author(s):

Zhongheng Li ◽

Fang He ◽

Haojie Hu ◽

Fei Wang ◽

Weizhong Yu

Keyword(s):

Anomaly Detection ◽

Hyperspectral Image ◽

Spectral Feature ◽

Collaborative Representation ◽

Detection Accuracy ◽

Spatial Features ◽

Collective Representation ◽

Detection Approach ◽

Adaptive Weight ◽

Gabor Feature

Collaborative representation-based detector (CRD), as the most representative anomaly detection method, has been widely applied in the field of hyperspectral anomaly detection (HAD). However, the sliding dual window of the original CRD introduces high computational complexity. Moreover, most HAD models only consider a single spectral or spatial feature of the hyperspectral image (HSI), which is unhelpful for improving detection accuracy. To solve these problems, in terms of speed and accuracy, we propose a novel anomaly detection approach, named Random Collective Representation-based Detector with Multiple Feature (RCRDMF). This method includes the following steps. This method first extract the different features include spectral feature, Gabor feature, extended multiattribute profile (EMAP) feature, and extended morphological profile (EMP) feature matrix from the HSI image, which enables us to improve the accuracy of HAD by combining the multiple spectral and spatial features. The ensemble and random collaborative representation detector (ERCRD) method is then applied, which can improve the anomaly detection speed. Finally, an adaptive weight approach is proposed to calculate the weight for each feature. Experimental results on six hyperspectral datasets demonstrate that the proposed approach has the superiority over accuracy and speed.

Download Full-text

A Lightweight Spectral–Spatial Feature Extraction and Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12091395 ◽

2020 ◽

Vol 12 (9) ◽

pp. 1395

Author(s):

Linlin Chen ◽

Zhihui Wei ◽

Yang Xu

Keyword(s):

Classification Accuracy ◽

Hyperspectral Image ◽

State Of The Art ◽

Spectral Features ◽

Classification Methods ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Deep Network ◽

Spatial Features ◽

Reduced Parameters

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Download Full-text

A Multi-Scale and Multi-Level Spectral-Spatial Feature Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12010125 ◽

2020 ◽

Vol 12 (1) ◽

pp. 125 ◽

Cited By ~ 5

Author(s):

Mu ◽

Guo ◽

Liu

Keyword(s):

Neural Network ◽

Hyperspectral Image ◽

Feature Fusion ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Multi Scale ◽

Spatial Features ◽

Multi Level

Extracting spatial and spectral features through deep neural networks has become an effective means of classification of hyperspectral images. However, most networks rarely consider the extraction of multi-scale spatial features and cannot fully integrate spatial and spectral features. In order to solve these problems, this paper proposes a multi-scale and multi-level spectral-spatial feature fusion network (MSSN) for hyperspectral image classification. The network uses the original 3D cube as input data and does not need to use feature engineering. In the MSSN, using different scale neighborhood blocks as the input of the network, the spectral-spatial features of different scales can be effectively extracted. The proposed 3D–2D alternating residual block combines the spectral features extracted by the three-dimensional convolutional neural network (3D-CNN) with the spatial features extracted by the two-dimensional convolutional neural network (2D-CNN). It not only achieves the fusion of spectral features and spatial features but also achieves the fusion of high-level features and low-level features. Experimental results on four hyperspectral datasets show that this method is superior to several state-of-the-art classification methods for hyperspectral images.

Download Full-text

Spatial–Spectral Feature Refinement for Hyperspectral Image Classification Based on Attention-Dense 3D-2D-CNN

Sensors ◽

10.3390/s20185191 ◽

2020 ◽

Vol 20 (18) ◽

pp. 5191

Author(s):

Jin Zhang ◽

Fengyuan Wei ◽

Fan Feng ◽

Chunyang Wang

Keyword(s):

Spatial Information ◽

Hyperspectral Image ◽

Spectral Feature ◽

Small Sample ◽

Training Data ◽

Spectral Features ◽

Training Samples ◽

Proposed Model ◽

Limited Training Samples ◽

Hyperspectral Classification

Convolutional neural networks provide an ideal solution for hyperspectral image (HSI) classification. However, the classification effect is not satisfactory when limited training samples are available. Focused on “small sample” hyperspectral classification, we proposed a novel 3D-2D-convolutional neural network (CNN) model named AD-HybridSN (Attention-Dense-HybridSN). In our proposed model, a dense block was used to reuse shallow features and aimed at better exploiting hierarchical spatial–spectral features. Subsequent depth separable convolutional layers were used to discriminate the spatial information. Further refinement of spatial–spectral features was realized by the channel attention method and spatial attention method, which were performed behind every 3D convolutional layer and every 2D convolutional layer, respectively. Experiment results indicate that our proposed model can learn more discriminative spatial–spectral features using very few training data. In Indian Pines, Salinas and the University of Pavia, AD-HybridSN obtain 97.02%, 99.59% and 98.32% overall accuracy using only 5%, 1% and 1% labeled data for training, respectively, which are far better than all the contrast models.

Download Full-text

Hyperspectral Image Classification via a Novel Spectral–Spatial 3D ConvLSTM-CNN

Remote Sensing ◽

10.3390/rs13214348 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4348

Author(s):

Ghulam Farooque ◽

Liang Xiao ◽

Jingxiang Yang ◽

Allah Bux Sargano

Keyword(s):

Spatial Information ◽

Short Term Memory ◽

Hyperspectral Image ◽

Fast Learning ◽

Spatial Features ◽

Learning Capabilities ◽

Benchmark Datasets ◽

3D Cnn ◽

Local Feature Extraction

In recent years, deep learning-based models have produced encouraging results for hyperspectral image (HSI) classification. Specifically, Convolutional Long Short-Term Memory (ConvLSTM) has shown good performance for learning valuable features and modeling long-term dependencies in spectral data. However, it is less effective for learning spatial features, which is an integral part of hyperspectral images. Alternatively, convolutional neural networks (CNNs) can learn spatial features, but they possess limitations in handling long-term dependencies due to the local feature extraction in these networks. Considering these factors, this paper proposes an end-to-end Spectral-Spatial 3D ConvLSTM-CNN based Residual Network (SSCRN), which combines 3D ConvLSTM and 3D CNN for handling both spectral and spatial information, respectively. The contribution of the proposed network is twofold. Firstly, it addresses the long-term dependencies of spectral dimension using 3D ConvLSTM to capture the information related to various ground materials effectively. Secondly, it learns the discriminative spatial features using 3D CNN by employing the concept of the residual blocks to accelerate the training process and alleviate the overfitting. In addition, SSCRN uses batch normalization and dropout to regularize the network for smooth learning. The proposed framework is evaluated on three benchmark datasets widely used by the research community. The results confirm that SSCRN outperforms state-of-the-art methods with an overall accuracy of 99.17%, 99.67%, and 99.31% over Indian Pines, Salinas, and Pavia University datasets, respectively. Moreover, it is worth mentioning that these excellent results were achieved with comparatively fewer epochs, which also confirms the fast learning capabilities of the SSCRN.

Download Full-text

HYPERSPECTRAL IMAGE CLASSIFICATION USING RESIDUAL 2D AND 3D CONVOLUTIONAL NEURAL NETWORK JOINT ATTENTION MODEL

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliv-m-3-2021-187-2021 ◽

2021 ◽

Vol XLIV-M-3-2021 ◽

pp. 187-193

Author(s):

Q. Yuan ◽

Y. Ang ◽

H. Z. M. Shafri

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Features ◽

3D Cnn ◽

2D And 3D

Abstract. Hyperspectral image classification (HSIC) is a challenging task in remote sensing data analysis, which has been applied in many domains for better identification and inspection of the earth surface by extracting spectral and spatial information. The combination of abundant spectral features and accurate spatial information can improve classification accuracy. However, many traditional methods are based on handcrafted features, which brings difficulties for multi-classification tasks due to spectral intra-class heterogeneity and similarity of inter-class. The deep learning algorithm, especially the convolutional neural network (CNN), has been perceived promising feature extractor and classification for processing hyperspectral remote sensing images. Although 2D CNN can extract spatial features, the specific spectral properties are not used effectively. While 3D CNN has the capability for them, but the computational burden increases as stacking layers. To address these issues, we propose a novel HSIC framework based on the residual CNN network by integrating the advantage of 2D and 3D CNN. First, 3D convolutions focus on extracting spectral features with feature recalibration and refinement by channel attention mechanism. The 2D depth-wise separable convolution approach with different size kernels concentrates on obtaining multi-scale spatial features and reducing model parameters. Furthermore, the residual structure optimizes the back-propagation for network training. The results and analysis of extensive HSIC experiments show that the proposed residual 2D-3D CNN network can effectively extract spectral and spatial features and improve classification accuracy.

Download Full-text