A Spatial-Spectral Feature Descriptor for Hyperspectral Image Matching

Hyperspectral Images (HSIs) have been utilized in many fields which contain spatial and spectral features of objects simultaneously. Hyperspectral image matching is a fundamental and critical problem in a wide range of HSI applications. Feature descriptors for grayscale image matching are well studied, but few descriptors are elaborately designed for HSI matching. HSI descriptors, which should have made good use of the spectral feature, are essential in HSI matching tasks. Therefore, this paper presents a descriptor for HSI matching, called HOSG-SIFT, which ensembles spectral features with spatial features of objects. First, we obtain the grayscale image by dimensional reduction from HSI and apply it to extract keypoints and descriptors of spatial features. Second, the descriptors of spectral features are designed based on the histogram of the spectral gradient (HOSG), which effectively preserves the physical significance of the spectral profile. Third, we concatenate the spatial descriptors and spectral descriptors with the same weights into a new descriptor and apply it for HSI matching. Experimental results demonstrate that the proposed HOSG-SIFT performs superior against traditional feature descriptors.

Download Full-text

A Spectral Spatial Attention Fusion with Deformable Convolutional Residual Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13183590 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3590

Author(s):

Tianyu Zhang ◽

Cuiping Shi ◽

Diling Liao ◽

Liguo Wang

Keyword(s):

Image Classification ◽

Spatial Attention ◽

Hyperspectral Image ◽

Spectral Feature ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Residual Network ◽

Spatial Features ◽

Training Samples

Convolutional neural networks (CNNs) have exhibited excellent performance in hyperspectral image classification. However, due to the lack of labeled hyperspectral data, it is difficult to achieve high classification accuracy of hyperspectral images with fewer training samples. In addition, although some deep learning techniques have been used in hyperspectral image classification, due to the abundant information of hyperspectral images, the problem of insufficient spatial spectral feature extraction still exists. To address the aforementioned issues, a spectral–spatial attention fusion with a deformable convolution residual network (SSAF-DCR) is proposed for hyperspectral image classification. The proposed network is composed of three parts, and each part is connected sequentially to extract features. In the first part, a dense spectral block is utilized to reuse spectral features as much as possible, and a spectral attention block that can refine and optimize the spectral features follows. In the second part, spatial features are extracted and selected by a dense spatial block and attention block, respectively. Then, the results of the first two parts are fused and sent to the third part, and deep spatial features are extracted by the DCR block. The above three parts realize the effective extraction of spectral–spatial features, and the experimental results for four commonly used hyperspectral datasets demonstrate that the proposed SSAF-DCR method is superior to some state-of-the-art methods with very few training samples.

Download Full-text

DBMF: A Novel Method for Tree Species Fusion Classification Based on Multi-Source Images

Forests ◽

10.3390/f13010033 ◽

2021 ◽

Vol 13 (1) ◽

pp. 33

Author(s):

Xueliang Wang ◽

Honge Ren

Keyword(s):

Tree Species ◽

Short Term Memory ◽

Hyperspectral Image ◽

Species Recognition ◽

Spectral Feature ◽

Activation Function ◽

Spectral Features ◽

Spatial Features ◽

Source Data ◽

Fusion Methods

Multi-source data remote sensing provides innovative technical support for tree species recognition. Tree species recognition is relatively poor despite noteworthy advancements in image fusion methods because the features from multi-source data for each pixel in the same region cannot be deeply exploited. In the present paper, a novel deep learning approach for hyperspectral imagery is proposed to improve accuracy for the classification of tree species. The proposed method, named the double branch multi-source fusion (DBMF) method, could more deeply determine the relationship between multi-source data and provide more effective information. The DBMF method does this by fusing spectral features extracted from a hyperspectral image (HSI) captured by the HJ-1A satellite and spatial features extracted from a multispectral image (MSI) captured by the Sentinel-2 satellite. The network has two branches in the spatial branch to avoid the risk of information loss, of which, sandglass blocks are embedded into a convolutional neural network (CNN) to extract the corresponding spatial neighborhood features from the MSI. Simultaneously, to make the useful spectral feature transfer more effective in the spectral branch, we employed bidirectional long short-term memory (Bi-LSTM) with a triple attention mechanism to extract the spectral features of each pixel in the HSI with low resolution. The feature information is fused to classify the tree species after the addition of a fusion activation function, which could allow the network to obtain more interactive information. Finally, the fusion strategy allows for the prediction of the full classification map of three study areas. Experimental results on a multi-source dataset show that DBMF has a significant advantage over other state-of-the-art frameworks.

Download Full-text

Random Collective Representation-Based Detector with Multiple Features for Hyperspectral Images

Remote Sensing ◽

10.3390/rs13040721 ◽

2021 ◽

Vol 13 (4) ◽

pp. 721

Author(s):

Zhongheng Li ◽

Fang He ◽

Haojie Hu ◽

Fei Wang ◽

Weizhong Yu

Keyword(s):

Anomaly Detection ◽

Hyperspectral Image ◽

Spectral Feature ◽

Collaborative Representation ◽

Detection Accuracy ◽

Spatial Features ◽

Collective Representation ◽

Detection Approach ◽

Adaptive Weight ◽

Gabor Feature

Collaborative representation-based detector (CRD), as the most representative anomaly detection method, has been widely applied in the field of hyperspectral anomaly detection (HAD). However, the sliding dual window of the original CRD introduces high computational complexity. Moreover, most HAD models only consider a single spectral or spatial feature of the hyperspectral image (HSI), which is unhelpful for improving detection accuracy. To solve these problems, in terms of speed and accuracy, we propose a novel anomaly detection approach, named Random Collective Representation-based Detector with Multiple Feature (RCRDMF). This method includes the following steps. This method first extract the different features include spectral feature, Gabor feature, extended multiattribute profile (EMAP) feature, and extended morphological profile (EMP) feature matrix from the HSI image, which enables us to improve the accuracy of HAD by combining the multiple spectral and spatial features. The ensemble and random collaborative representation detector (ERCRD) method is then applied, which can improve the anomaly detection speed. Finally, an adaptive weight approach is proposed to calculate the weight for each feature. Experimental results on six hyperspectral datasets demonstrate that the proposed approach has the superiority over accuracy and speed.

Download Full-text

A Lightweight Spectral–Spatial Feature Extraction and Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12091395 ◽

2020 ◽

Vol 12 (9) ◽

pp. 1395

Author(s):

Linlin Chen ◽

Zhihui Wei ◽

Yang Xu

Keyword(s):

Classification Accuracy ◽

Hyperspectral Image ◽

State Of The Art ◽

Spectral Features ◽

Classification Methods ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Deep Network ◽

Spatial Features ◽

Reduced Parameters

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Download Full-text

A Multi-Scale and Multi-Level Spectral-Spatial Feature Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12010125 ◽

2020 ◽

Vol 12 (1) ◽

pp. 125 ◽

Cited By ~ 5

Author(s):

Mu ◽

Guo ◽

Liu

Keyword(s):

Neural Network ◽

Hyperspectral Image ◽

Feature Fusion ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Multi Scale ◽

Spatial Features ◽

Multi Level

Extracting spatial and spectral features through deep neural networks has become an effective means of classification of hyperspectral images. However, most networks rarely consider the extraction of multi-scale spatial features and cannot fully integrate spatial and spectral features. In order to solve these problems, this paper proposes a multi-scale and multi-level spectral-spatial feature fusion network (MSSN) for hyperspectral image classification. The network uses the original 3D cube as input data and does not need to use feature engineering. In the MSSN, using different scale neighborhood blocks as the input of the network, the spectral-spatial features of different scales can be effectively extracted. The proposed 3D–2D alternating residual block combines the spectral features extracted by the three-dimensional convolutional neural network (3D-CNN) with the spatial features extracted by the two-dimensional convolutional neural network (2D-CNN). It not only achieves the fusion of spectral features and spatial features but also achieves the fusion of high-level features and low-level features. Experimental results on four hyperspectral datasets show that this method is superior to several state-of-the-art classification methods for hyperspectral images.

Download Full-text

Spatial–Spectral Feature Refinement for Hyperspectral Image Classification Based on Attention-Dense 3D-2D-CNN

Sensors ◽

10.3390/s20185191 ◽

2020 ◽

Vol 20 (18) ◽

pp. 5191

Author(s):

Jin Zhang ◽

Fengyuan Wei ◽

Fan Feng ◽

Chunyang Wang

Keyword(s):

Spatial Information ◽

Hyperspectral Image ◽

Spectral Feature ◽

Small Sample ◽

Training Data ◽

Spectral Features ◽

Training Samples ◽

Proposed Model ◽

Limited Training Samples ◽

Hyperspectral Classification

Convolutional neural networks provide an ideal solution for hyperspectral image (HSI) classification. However, the classification effect is not satisfactory when limited training samples are available. Focused on “small sample” hyperspectral classification, we proposed a novel 3D-2D-convolutional neural network (CNN) model named AD-HybridSN (Attention-Dense-HybridSN). In our proposed model, a dense block was used to reuse shallow features and aimed at better exploiting hierarchical spatial–spectral features. Subsequent depth separable convolutional layers were used to discriminate the spatial information. Further refinement of spatial–spectral features was realized by the channel attention method and spatial attention method, which were performed behind every 3D convolutional layer and every 2D convolutional layer, respectively. Experiment results indicate that our proposed model can learn more discriminative spatial–spectral features using very few training data. In Indian Pines, Salinas and the University of Pavia, AD-HybridSN obtain 97.02%, 99.59% and 98.32% overall accuracy using only 5%, 1% and 1% labeled data for training, respectively, which are far better than all the contrast models.

Download Full-text

HYPERSPECTRAL IMAGE CLASSIFICATION USING RESIDUAL 2D AND 3D CONVOLUTIONAL NEURAL NETWORK JOINT ATTENTION MODEL

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliv-m-3-2021-187-2021 ◽

2021 ◽

Vol XLIV-M-3-2021 ◽

pp. 187-193

Author(s):

Q. Yuan ◽

Y. Ang ◽

H. Z. M. Shafri

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Features ◽

3D Cnn ◽

2D And 3D

Abstract. Hyperspectral image classification (HSIC) is a challenging task in remote sensing data analysis, which has been applied in many domains for better identification and inspection of the earth surface by extracting spectral and spatial information. The combination of abundant spectral features and accurate spatial information can improve classification accuracy. However, many traditional methods are based on handcrafted features, which brings difficulties for multi-classification tasks due to spectral intra-class heterogeneity and similarity of inter-class. The deep learning algorithm, especially the convolutional neural network (CNN), has been perceived promising feature extractor and classification for processing hyperspectral remote sensing images. Although 2D CNN can extract spatial features, the specific spectral properties are not used effectively. While 3D CNN has the capability for them, but the computational burden increases as stacking layers. To address these issues, we propose a novel HSIC framework based on the residual CNN network by integrating the advantage of 2D and 3D CNN. First, 3D convolutions focus on extracting spectral features with feature recalibration and refinement by channel attention mechanism. The 2D depth-wise separable convolution approach with different size kernels concentrates on obtaining multi-scale spatial features and reducing model parameters. Furthermore, the residual structure optimizes the back-propagation for network training. The results and analysis of extensive HSIC experiments show that the proposed residual 2D-3D CNN network can effectively extract spectral and spatial features and improve classification accuracy.

Download Full-text

Multi-Feature Manifold Discriminant Analysis for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs11060651 ◽

2019 ◽

Vol 11 (6) ◽

pp. 651 ◽

Cited By ~ 10

Author(s):

Hong Huang ◽

Zhengying Li ◽

Yinsong Pan

Keyword(s):

Discriminant Analysis ◽

Spatial Information ◽

Hyperspectral Image ◽

Feature Fusion ◽

Local Binary Patterns ◽

Hyperspectral Data ◽

Spectral Features ◽

Spatial Features ◽

Combined Features ◽

Low Dimensional

Hyperspectral image (HSI) provides both spatial structure and spectral information for classification, but many traditional methods simply concatenate spatial features and spectral features together that usually lead to the curse-of-dimensionality and unbalanced representation of different features. To address this issue, a new dimensionality reduction (DR) method, termed multi-feature manifold discriminant analysis (MFMDA), was proposed in this paper. At first, MFMDA explores local binary patterns (LBP) operator to extract textural features for encoding the spatial information in HSI. Then, under graph embedding framework, the intrinsic and penalty graphs of LBP and spectral features are constructed to explore the discriminant manifold structure in both spatial and spectral domains, respectively. After that, a new spatial-spectral DR model for multi-feature fusion is built to extract discriminant spatial-spectral combined features, and it not only preserves the similarity relationship between spectral features and LBP features but also possesses strong discriminating ability in the low-dimensional embedding space. Experiments on Indian Pines, Heihe and Pavia University (PaviaU) hyperspectral data sets demonstrate that the proposed MFMDA method performs significantly better than some state-of-the-art methods using only single feature or simply stacking spectral features and spatial features together, and the classification accuracies of it can reach 95.43%, 97.19% and 96.60%, respectively.

Download Full-text

Spatial-spectral identification of abnormal leukocytes based on microscopic hyperspectral imaging technology

Journal of Innovative Optical Health Sciences ◽

10.1142/s1793545820500054 ◽

2020 ◽

Vol 13 (02) ◽

pp. 2050005

Author(s):

Xueqi Hu ◽

Jiahua Ou ◽

Mei Zhou ◽

Menghan Hu ◽

Li Sun ◽

...

Keyword(s):

Hyperspectral Imaging ◽

Lymphoblastic Leukemia ◽

Spectral Feature ◽

Support Vector ◽

Watershed Algorithm ◽

Spectral Features ◽

Accurate Identification ◽

Spatial Features ◽

Spectral Identification

Screening and diagnosing of abnormal Leukocytes are crucial for the diagnosis of immune diseases and Acute Lymphoblastic Leukemia (ALL). As the deterioration of abnormal leukocytes is mainly due to the changes in the chromatin distribution, which significantly affects the absorption and reflection of light, the spectral feature is proved to be important for leukocytes classification and identification. This paper proposes an accurate identification method for healthy and abnormal leukocytes based on microscopic hyperspectral imaging (HSI) technology which combines the spectral information. The segmentation of nucleus and cytoplasm is obtained by the morphological watershed algorithm. Then, the spectral features are extracted and combined with the spatial features. Based on this, the support vector machine (SVM) is applied for classification of five types of leukocytes and abnormal leukocytes. Compared with different classification methods, the proposed method utilizes spectral features which highlight the differences between healthy leukocytes and abnormal leukocytes, improving the accuracy in the classification and identification of leukocytes. This paper only selects one subtype of ALL for test, and the proposed method can be applied for detection of other leukemia in the future.

Download Full-text

Spatial-Spectral Network for Hyperspectral Image Classification: A 3-D CNN and Bi-LSTM Framework

Remote Sensing ◽

10.3390/rs13122353 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2353

Author(s):

Junru Yin ◽

Changsheng Qi ◽

Qiqiang Chen ◽

Jiantao Qu

Keyword(s):

Short Term Memory ◽

Hyperspectral Image ◽

Sequence Data ◽

Three Dimensional ◽

Spectral Feature ◽

Spectral Features ◽

Spectral Bands ◽

Current State ◽

Feature Extractor ◽

Lstm Network

Recently, deep learning methods based on the combination of spatial and spectral features have been successfully applied in hyperspectral image (HSI) classification. To improve the utilization of the spatial and spectral information from the HSI, this paper proposes a unified network framework using a three-dimensional convolutional neural network (3-D CNN) and a band grouping-based bidirectional long short-term memory (Bi-LSTM) network for HSI classification. In the framework, extracting spectral features is regarded as a procedure of processing sequence data, and the Bi-LSTM network acts as the spectral feature extractor of the unified network to fully exploit the close relationships between spectral bands. The 3-D CNN has a unique advantage in processing the 3-D data; therefore, it is used as the spatial-spectral feature extractor in this unified network. Finally, in order to optimize the parameters of both feature extractors simultaneously, the Bi-LSTM and 3-D CNN share a loss function to form a unified network. To evaluate the performance of the proposed framework, three datasets were tested for HSI classification. The results demonstrate that the performance of the proposed method is better than the current state-of-the-art HSI classification methods.

Download Full-text