Modifications of the Multi-Layer Perceptron for Hyperspectral Image Classification

Recently, many convolutional neural network (CNN)-based methods have been proposed to tackle the classification task of hyperspectral images (HSI). In fact, CNN has become the de-facto standard for HSI classification. It seems that the traditional neural networks such as multi-layer perceptron (MLP) are not competitive for HSI classification. However, in this study, we try to prove that the MLP can achieve good classification performance of HSI if it is properly designed and improved. The proposed Modified-MLP for HSI classification contains two special parts: spectral–spatial feature mapping and spectral–spatial information mixing. Specifically, for spectral–spatial feature mapping, each input sample of HSI is divided into a sequence of 3D patches with fixed length and then a linear layer is used to map the 3D patches to spectral–spatial features. For spectral–spatial information mixing, all the spectral–spatial features within a single sample are feed into the solely MLP architecture to model the spectral–spatial information across patches for following HSI classification. Furthermore, to obtain the abundant spectral–spatial information with different scales, Multiscale-MLP is proposed to aggregate neighboring patches with multiscale shapes for acquiring abundant spectral–spatial information. In addition, the Soft-MLP is proposed to further enhance the classification performance by applying soft split operation, which flexibly capture the global relations of patches at different positions in the input HSI sample. Finally, label smoothing is introduced to mitigate the overfitting problem in the Soft-MLP (Soft-MLP-L), which greatly improves the classification performance of MLP-based method. The proposed Modified-MLP, Multiscale-MLP, Soft-MLP, and Soft-MLP-L are tested on the three widely used hyperspectral datasets. The proposed Soft-MLP-L leads to the highest OA, which outperforms CNN by 5.76%, 2.55%, and 2.5% on the Salinas, Pavia, and Indian Pines datasets, respectively. The proposed Modified-MLP, Multiscale-MLP, and Soft-MLP are tested on the three widely used hyperspectral datasets (i.e., Salinas, Pavia, and Indian Pines). The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the MLP-based methods are still competitive for HSI classification.

Download Full-text

Densely Connected Pyramidal Dilated Convolutional Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13173396 ◽

2021 ◽

Vol 13 (17) ◽

pp. 3396

Author(s):

Feng Zhao ◽

Junjie Zhang ◽

Zhe Meng ◽

Hanqiang Liu

Keyword(s):

Receptive Field ◽

Spatial Information ◽

Hyperspectral Image ◽

Feature Fusion ◽

Receptive Fields ◽

Classification Performance ◽

Convolutional Network ◽

Dilated Convolution ◽

Spatial Features ◽

Good Classification Performance

Recently, with the extensive application of deep learning techniques in the hyperspectral image (HSI) field, particularly convolutional neural network (CNN), the research of HSI classification has stepped into a new stage. To avoid the problem that the receptive field of naive convolution is small, the dilated convolution is introduced into the field of HSI classification. However, the dilated convolution usually generates blind spots in the receptive field, resulting in discontinuous spatial information obtained. In order to solve the above problem, a densely connected pyramidal dilated convolutional network (PDCNet) is proposed in this paper. Firstly, a pyramidal dilated convolutional (PDC) layer integrates different numbers of sub-dilated convolutional layers is proposed, where the dilated factor of the sub-dilated convolution increases exponentially, achieving multi-sacle receptive fields. Secondly, the number of sub-dilated convolutional layers increases in a pyramidal pattern with the depth of the network, thereby capturing more comprehensive hyperspectral information in the receptive field. Furthermore, a feature fusion mechanism combining pixel-by-pixel addition and channel stacking is adopted to extract more abstract spectral–spatial features. Finally, in order to reuse the features of the previous layers more effectively, dense connections are applied in densely pyramidal dilated convolutional (DPDC) blocks. Experiments on three well-known HSI datasets indicate that PDCNet proposed in this paper has good classification performance compared with other popular models.

Download Full-text

Hyperspectral Image Classification Based on Two-Branch Spectral–Spatial-Feature Attention Network

Remote Sensing ◽

10.3390/rs13214262 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4262

Author(s):

Hanjie Wu ◽

Dan Li ◽

Yujian Wang ◽

Xiaojun Li ◽

Fanqiang Kong ◽

...

Keyword(s):

Hyperspectral Image ◽

Classification Performance ◽

Hybrid Architecture ◽

Classification Methods ◽

Attention Network ◽

Spatial Feature ◽

Spatial Features ◽

Original Dataset ◽

Training Samples ◽

Feature Attention

Although most of deep-learning-based hyperspectral image (HSI) classification methods achieve great performance, there still remains a challenge to utilize small-size training samples to remarkably enhance the classification accuracy. To tackle this challenge, a novel two-branch spectral–spatial-feature attention network (TSSFAN) for HSI classification is proposed in this paper. Firstly, two inputs with different spectral dimensions and spatial sizes are constructed, which can not only reduce the redundancy of the original dataset but also accurately explore the spectral and spatial features. Then, we design two parallel 3DCNN branches with attention modules, in which one focuses on extracting spectral features and adaptively learning the more discriminative spectral channels, and the other focuses on exploring spatial features and adaptively learning the more discriminative spatial structures. Next, the feature attention module is constructed to automatically adjust the weights of different features based on their contributions for classification to remarkably improve the classification performance. Finally, we design the hybrid architecture of 3D–2DCNN to acquire the final classification result, which can significantly decrease the sophistication of the network. Experimental results on three HSI datasets indicate that our presented TSSFAN method outperforms several of the most advanced classification methods.

Download Full-text

Variational Generative Adversarial Network with Crossed Spatial and Spectral Interactions for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13163131 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3131

Author(s):

Zhongwei Li ◽

Xue Zhu ◽

Ziqi Xin ◽

Fangming Guo ◽

Xingshuai Cui ◽

...

Keyword(s):

Image Classification ◽

Spatial Information ◽

Hyperspectral Image ◽

Classification Performance ◽

Generative Adversarial Networks ◽

Hyperspectral Image Classification ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Spatial Features ◽

Virtual Samples

Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have been widely used in hyperspectral image classification (HSIC) tasks. However, the generated HSI virtual samples by VAEs are often ambiguous, and GANs are prone to the mode collapse, which lead the poor generalization abilities ultimately. Moreover, most of these models only consider the extraction of spectral or spatial features. They fail to combine the two branches interactively and ignore the correlation between them. Consequently, the variational generative adversarial network with crossed spatial and spectral interactions (CSSVGAN) was proposed in this paper, which includes a dual-branch variational Encoder to map spectral and spatial information to different latent spaces, a crossed interactive Generator to improve the quality of generated virtual samples, and a Discriminator stuck with a classifier to enhance the classification performance. Combining these three subnetworks, the proposed CSSVGAN achieves excellent classification by ensuring the diversity and interacting spectral and spatial features in a crossed manner. The superior experimental results on three datasets verify the effectiveness of this method.

Download Full-text

Deep Spectral Spatial Inverted Residual Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13214472 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4472

Author(s):

Tianyu Zhang ◽

Cuiping Shi ◽

Diling Liao ◽

Liguo Wang

Keyword(s):

Image Classification ◽

Data Augmentation ◽

Hyperspectral Image ◽

Classification Performance ◽

Hyperspectral Data ◽

Hyperspectral Images ◽

Hyperspectral Image Classification ◽

Feature Maps ◽

Spatial Feature ◽

Global Context

Convolutional neural networks (CNNs) have been widely used in hyperspectral image classification in recent years. The training of CNNs relies on a large amount of labeled sample data. However, the number of labeled samples of hyperspectral data is relatively small. Moreover, for hyperspectral images, fully extracting spectral and spatial feature information is the key to achieve high classification performance. To solve the above issues, a deep spectral spatial inverted residuals network (DSSIRNet) is proposed. In this network, a data block random erasing strategy is introduced to alleviate the problem of limited labeled samples by data augmentation of small spatial blocks. In addition, a deep inverted residuals (DIR) module for spectral spatial feature extraction is proposed, which locks the effective features of each layer while avoiding network degradation. Furthermore, a global 3D attention module is proposed, which can realize the fine extraction of spectral and spatial global context information under the condition of the same number of input and output feature maps. Experiments are carried out on four commonly used hyperspectral datasets. A large number of experimental results show that compared with some state-of-the-art classification methods, the proposed method can provide higher classification accuracy for hyperspectral images.

Download Full-text

A 3D-2D Multibranch Feature Fusion and Dense Attention Network for Hyperspectral Image Classification

Micromachines ◽

10.3390/mi12101271 ◽

2021 ◽

Vol 12 (10) ◽

pp. 1271

Author(s):

Hongmin Gao ◽

Yiyan Zhang ◽

Yunfei Zhang ◽

Zhonghao Chen ◽

Chenming Li ◽

...

Keyword(s):

Image Classification ◽

Hyperspectral Image ◽

Feature Fusion ◽

Receptive Fields ◽

Classification Performance ◽

Redundant Information ◽

Hyperspectral Image Classification ◽

Training Process ◽

Attention Network ◽

Spatial Features

In recent years, hyperspectral image classification (HSI) has attracted considerable attention. Various methods based on convolution neural networks have achieved outstanding classification results. However, most of them exited the defects of underutilization of spectral-spatial features, redundant information, and convergence difficulty. To address these problems, a novel 3D-2D multibranch feature fusion and dense attention network are proposed for HSI classification. Specifically, the 3D multibranch feature fusion module integrates multiple receptive fields in spatial and spectral dimensions to obtain shallow features. Then, a 2D densely connected attention module consists of densely connected layers and spatial-channel attention block. The former is used to alleviate the gradient vanishing and enhance the feature reuse during the training process. The latter emphasizes meaningful features and suppresses the interfering information along the two principal dimensions: channel and spatial axes. The experimental results on four benchmark hyperspectral images datasets demonstrate that the model can effectively improve the classification performance with great robustness.

Download Full-text

A Lightweight Spectral–Spatial Feature Extraction and Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12091395 ◽

2020 ◽

Vol 12 (9) ◽

pp. 1395

Author(s):

Linlin Chen ◽

Zhihui Wei ◽

Yang Xu

Keyword(s):

Classification Accuracy ◽

Hyperspectral Image ◽

State Of The Art ◽

Spectral Features ◽

Classification Methods ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Deep Network ◽

Spatial Features ◽

Reduced Parameters

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Download Full-text

3-D Convolution-Recurrent Networks for Spectral-Spatial Classification of Hyperspectral Images

Remote Sensing ◽

10.3390/rs11070883 ◽

2019 ◽

Vol 11 (7) ◽

pp. 883 ◽

Cited By ~ 10

Author(s):

Majid Seydgar ◽

Amin Alizadeh Naeini ◽

Mengmeng Zhang ◽

Wei Li ◽

Mehran Satari

Keyword(s):

Spatial Information ◽

Short Term Memory ◽

Structural Data ◽

Experimental Results ◽

Spatial Feature ◽

Computational Burden ◽

Spatial Classification ◽

Spatial Features ◽

Proposed Model

Nowadays, 3-D convolutional neural networks (3-D CNN) have attracted lots of attention in the spectral-spatial classification of hyperspectral imageries (HSI). In this model, the feed-forward processing structure reduces the computational burden of 3-D structural processing. However, this model as a vector-based methodology cannot analyze the full content of the HSI information, and as a result, its features are not quite discriminative. On the other hand, convolutional long short-term memory (CLSTM) can recurrently analyze the 3-D structural data to extract more discriminative and abstract features. However, the computational burden of this model as a sequence-based methodology is extremely high. In the meanwhile, the robust spectral-spatial feature extraction with a reasonable computational burden is of great interest in HSI classification. For this purpose, a two-stage method based on the integration of CNN and CLSTM is proposed. In the first stage, 3-D CNN is applied to extract low-dimensional shallow spectral-spatial features from HSI, where information on the spatial features are less than that of the spectral information; consequently, in the second stage, the CLSTM, for the first time, is applied to recurrently analyze the spatial information while considering the spectral one. The experimental results obtained from three widely used HSI datasets indicate that the application of the recurrent analysis for spatial feature extractions makes the proposed model robust against different spatial sizes of the extracted patches. Moreover, applying the 3-D CNN prior to the CLSTM efficiently reduces the model’s computational burden. The experimental results also indicated that the proposed model led to a 1% to 2% improvement compared to its counterpart models.

Download Full-text

A Multi-Scale and Multi-Level Spectral-Spatial Feature Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12010125 ◽

2020 ◽

Vol 12 (1) ◽

pp. 125 ◽

Cited By ~ 5

Author(s):

Mu ◽

Guo ◽

Liu

Keyword(s):

Neural Network ◽

Hyperspectral Image ◽

Feature Fusion ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Multi Scale ◽

Spatial Features ◽

Multi Level

Extracting spatial and spectral features through deep neural networks has become an effective means of classification of hyperspectral images. However, most networks rarely consider the extraction of multi-scale spatial features and cannot fully integrate spatial and spectral features. In order to solve these problems, this paper proposes a multi-scale and multi-level spectral-spatial feature fusion network (MSSN) for hyperspectral image classification. The network uses the original 3D cube as input data and does not need to use feature engineering. In the MSSN, using different scale neighborhood blocks as the input of the network, the spectral-spatial features of different scales can be effectively extracted. The proposed 3D–2D alternating residual block combines the spectral features extracted by the three-dimensional convolutional neural network (3D-CNN) with the spatial features extracted by the two-dimensional convolutional neural network (2D-CNN). It not only achieves the fusion of spectral features and spatial features but also achieves the fusion of high-level features and low-level features. Experimental results on four hyperspectral datasets show that this method is superior to several state-of-the-art classification methods for hyperspectral images.

Download Full-text

Distance Transform-Based Spectral-Spatial Feature Vector for Hyperspectral Image Classification with Stacked Autoencoder

Remote Sensing ◽

10.3390/rs13091732 ◽

2021 ◽

Vol 13 (9) ◽

pp. 1732

Author(s):

Hadis Madani ◽

Kenneth McIsaac

Keyword(s):

Deep Learning ◽

Spatial Information ◽

Hyperspectral Image ◽

Feature Vector ◽

Remote Sensing Data ◽

Distance Transform ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Learning Techniques

Pixel-wise classification of hyperspectral images (HSIs) from remote sensing data is a common approach for extracting information about scenes. In recent years, approaches based on deep learning techniques have gained wide applicability. An HSI dataset can be viewed either as a collection of images, each one captured at a different wavelength, or as a collection of spectra, each one associated with a specific point (pixel). Enhanced classification accuracy is enabled if the spectral and spatial information are combined in the input vector. This allows simultaneous classification according to spectral type but also according to geometric relationships. In this study, we proposed a novel spatial feature vector which improves accuracies in pixel-wise classification. Our proposed feature vector is based on the distance transform of the pixels with respect to the dominant edges in the input HSI. In other words, we allow the location of pixels within geometric subdivisions of the dataset to modify the contribution of each pixel to the spatial feature vector. Moreover, we used the extended multi attribute profile (EMAP) features to add more geometric features to the proposed spatial feature vector. We have performed experiments with three hyperspectral datasets. In addition to the Salinas and University of Pavia datasets, which are commonly used in HSI research, we include samples from our Surrey BC dataset. Our proposed method results compares favorably to traditional algorithms as well as to some recently published deep learning-based algorithms.

Download Full-text

Fully Dense Multiscale Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs11222718 ◽

2019 ◽

Vol 11 (22) ◽

pp. 2718 ◽

Cited By ~ 5

Author(s):

Zhe Meng ◽

Lingling Li ◽

Licheng Jiao ◽

Zhixi Feng ◽

Xu Tang ◽

...

Keyword(s):

Hyperspectral Image ◽

Multiple Scales ◽

Feature Learning ◽

Classification Performance ◽

Connectivity Pattern ◽

Great Success ◽

Feature Representations ◽

Spatial Features ◽

Discriminative Feature ◽

Correlated Information

The convolutional neural network (CNN) can automatically extract hierarchical feature representations from raw data and has recently achieved great success in the classification of hyperspectral images (HSIs). However, most CNN based methods used in HSI classification neglect adequately utilizing the strong complementary yet correlated information from each convolutional layer and only employ the last convolutional layer features for classification. In this paper, we propose a novel fully dense multiscale fusion network (FDMFN) that takes full advantage of the hierarchical features from all the convolutional layers for HSI classification. In the proposed network, shortcut connections are introduced between any two layers in a feed-forward manner, enabling features learned by each layer to be accessed by all subsequent layers. This fully dense connectivity pattern achieves comprehensive feature reuse and enforces discriminative feature learning. In addition, various spectral-spatial features with multiple scales from all convolutional layers are fused to extract more discriminative features for HSI classification. Experimental results on three widely used hyperspectral scenes demonstrate that the proposed FDMFN can achieve better classification performance in comparison with several state-of-the-art approaches.

Download Full-text