Hybrid Dense Network with Dual Attention for Hyperspectral Image Classification

Hyperspectral images (HSIs) have been widely used in many fields of application, but it is still extremely challenging to obtain higher classification accuracy, especially when facing a smaller number of training samples in practical applications. It is very time-consuming and laborious to acquire enough labeled samples. Consequently, an efficient hybrid dense network was proposed based on a dual-attention mechanism, due to limited training samples and unsatisfactory classification accuracy. The stacked autoencoder was first used to reduce the dimensions of HSIs. A hybrid dense network framework with two feature-extraction branches was then established in order to extract abundant spectral–spatial features from HSIs, based on the 3D and 2D convolutional neural network models. In addition, spatial attention and channel attention were jointly introduced in order to achieve selective learning of features derived from HSIs. The feature maps were further refined, and more important features could be retained. To improve computational efficiency and prevent the overfitting, the batch normalization layer and the dropout layer were adopted. The Indian Pines, Pavia University, and Salinas datasets were selected to evaluate the classification performance; 5%, 1%, and 1% of classes were randomly selected as training samples, respectively. In comparison with the REF-SVM, 3D-CNN, HybridSN, SSRN, and R-HybridSN, the overall accuracy of our proposed method could still reach 96.80%, 98.28%, and 98.85%, respectively. Our results show that this method can achieve a satisfactory classification performance even in the case of fewer training samples.

Download Full-text

Multiscale Information Fusion for Hyperspectral Image Classification Based on Hybrid 2D-3D CNN

Remote Sensing ◽

10.3390/rs13122268 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2268

Author(s):

Hang Gong ◽

Qiuxia Li ◽

Chunlai Li ◽

Haishan Dai ◽

Zhiping He ◽

...

Keyword(s):

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Small Sample ◽

Hyperspectral Images ◽

Sample Problem ◽

Training Samples ◽

Small Sample Problem ◽

3D Cnn ◽

Hyperspectral Classification

Hyperspectral images are widely used for classification due to its rich spectral information along with spatial information. To process the high dimensionality and high nonlinearity of hyperspectral images, deep learning methods based on convolutional neural network (CNN) are widely used in hyperspectral classification applications. However, most CNN structures are stacked vertically in addition to using a onefold size of convolutional kernels or pooling layers, which cannot fully mine the multiscale information on the hyperspectral images. When such networks meet the practical challenge of a limited labeled hyperspectral image dataset—i.e., “small sample problem”—the classification accuracy and generalization ability would be limited. In this paper, to tackle the small sample problem, we apply the semantic segmentation function to the pixel-level hyperspectral classification due to their comparability. A lightweight, multiscale squeeze-and-excitation pyramid pooling network (MSPN) is proposed. It consists of a multiscale 3D CNN module, a squeezing and excitation module, and a pyramid pooling module with 2D CNN. Such a hybrid 2D-3D-CNN MSPN framework can learn and fuse deeper hierarchical spatial–spectral features with fewer training samples. The proposed MSPN was tested on three publicly available hyperspectral classification datasets: Indian Pine, Salinas, and Pavia University. Using 5%, 0.5%, and 0.5% training samples of the three datasets, the classification accuracies of the MSPN were 96.09%, 97%, and 96.56%, respectively. In addition, we also selected the latest dataset with higher spatial resolution, named WHU-Hi-LongKou, as the challenge object. Using only 0.1% of the training samples, we could achieve a 97.31% classification accuracy, which is far superior to the state-of-the-art hyperspectral classification methods.

Download Full-text

A Convolutional Neural Network with Fletcher–Reeves Algorithm for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs11111325 ◽

2019 ◽

Vol 11 (11) ◽

pp. 1325 ◽

Cited By ~ 6

Author(s):

Chen Chen ◽

Yi Ma ◽

Guangbo Ren

Keyword(s):

Image Classification ◽

Classification Accuracy ◽

Hyperspectral Image ◽

Training Sample ◽

Practical Applications ◽

Model Based ◽

Training Samples ◽

Proposed Model ◽

Convergence Performance ◽

Hyperspectral Classification

Deep learning models, especially the convolutional neural networks (CNNs), are very active in hyperspectral remote sensing image classification. In order to better apply the CNN model to hyperspectral classification, we propose a CNN model based on Fletcher–Reeves algorithm (F–R CNN), which uses the Fletcher–Reeves (F–R) algorithm for gradient updating to optimize the convergence performance of the model in classification. In view of the fact that there are fewer optional training samples in practical applications, we further propose a method of increasing the number of samples by adding a certain degree of perturbed samples, which can also test the anti-interference ability of classification methods. Furthermore, we analyze the anti-interference and convergence performance of the proposed model in terms of different training sample data sets, different batch training sample numbers and iteration time. In this paper, we describe the experimental process in detail and comprehensively evaluate the proposed model based on the classification of CHRIS hyperspectral imagery covering coastal wetlands, and further evaluate it on a commonly used hyperspectral image benchmark dataset. The experimental results show that the accuracy of the two models after increasing training samples and adjusting the number of batch training samples is improved. When the number of batch training samples is continuously increased to 350, the classification accuracy of the proposed method can still be maintained above 80.7%, which is 2.9% higher than the traditional one. And its time consumption is less than that of the traditional one while ensuring classification accuracy. It can be concluded that the proposed method has anti-interference ability and outperforms the traditional CNN in terms of batch computing adaptability and convergence speed.

Download Full-text

CSA-MSO3DCNN: Multiscale Octave 3D CNN with Channel and Spatial Attention for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12010188 ◽

2020 ◽

Vol 12 (1) ◽

pp. 188 ◽

Cited By ~ 2

Author(s):

Qin Xu ◽

Yong Xiao ◽

Dongyue Wang ◽

Bin Luo

Keyword(s):

Spatial Attention ◽

Hyperspectral Image ◽

Image Data ◽

Classification Performance ◽

Data Sets ◽

Feature Maps ◽

Contextual Feature ◽

Hyperspectral Image Data ◽

3D Cnn ◽

Spatial Redundancy

3D convolutional neural networks (CNNs) have been demonstrated to be a powerful tool in hyperspectral images (HSIs) classification. However, using the conventional 3D CNNs to extract the spectral–spatial feature for HSIs results in too many parameters as HSIs have plenty of spatial redundancy. To address this issue, in this paper, we first design multiscale convolution to extract the contextual feature of different scales for HSIs and then propose to employ the octave 3D CNN which factorizes the mixed feature maps by their frequency to replace the normal 3D CNN in order to reduce the spatial redundancy and enlarge the receptive field. To further explore the discriminative features, a channel attention module and a spatial attention module are adopted to optimize the feature maps and improve the classification performance. The experiments on four hyperspectral image data sets demonstrate that the proposed method outperforms other state-of-the-art deep learning methods.

Download Full-text

Residual Group Channel and Space Attention Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12122035 ◽

2020 ◽

Vol 12 (12) ◽

pp. 2035 ◽

Cited By ~ 4

Author(s):

Peida Wu ◽

Ziguan Cui ◽

Zongliang Gan ◽

Feng Liu

Keyword(s):

Hyperspectral Image ◽

Three Dimensional ◽

Classification Performance ◽

Training Process ◽

Feature Maps ◽

Attention Network ◽

Spatial Features ◽

Network Training ◽

Training Samples ◽

Classification Tasks

Recently, deep learning methods based on three-dimensional (3-D) convolution have been widely used in the hyperspectral image (HSI) classification tasks and shown good classification performance. However, affected by the irregular distribution of various classes in HSI datasets, most previous 3-D convolutional neural network (CNN)-based models require more training samples to obtain better classification accuracies. In addition, as the network deepens, which leads to the spatial resolution of feature maps gradually decreasing, much useful information may be lost during the training process. Therefore, how to ensure efficient network training is key to the HSI classification tasks. To address the issue mentioned above, in this paper, we proposed a 3-DCNN-based residual group channel and space attention network (RGCSA) for HSI classification. Firstly, the proposed bottom-up top-down attention structure with the residual connection can improve network training efficiency by optimizing channel-wise and spatial-wise features throughout the whole training process. Secondly, the proposed residual group channel-wise attention module can reduce the possibility of losing useful information, and the novel spatial-wise attention module can extract context information to strengthen the spatial features. Furthermore, our proposed RGCSA network only needs few training samples to achieve higher classification accuracies than previous 3-D-CNN-based networks. The experimental results on three commonly used HSI datasets demonstrate the superiority of our proposed network based on the attention mechanism and the effectiveness of the proposed channel-wise and spatial-wise attention modules for HSI classification. The code and configurations are released at Github.com.

Download Full-text

Data Augmentation and Spectral Structure Features for Limited Samples Hyperspectral Classification

Remote Sensing ◽

10.3390/rs13040547 ◽

2021 ◽

Vol 13 (4) ◽

pp. 547

Author(s):

Wenning Wang ◽

Xuebin Liu ◽

Xuanqin Mou

Keyword(s):

Classification Accuracy ◽

Data Augmentation ◽

Classification Problem ◽

Classification Performance ◽

Spectral Structure ◽

Limited Sample ◽

Sample Classification ◽

Training Samples ◽

Traditional Classification ◽

Hyperspectral Classification

For both traditional classification and current popular deep learning methods, the limited sample classification problem is very challenging, and the lack of samples is an important factor affecting the classification performance. Our work includes two aspects. First, the unsupervised data augmentation for all hyperspectral samples not only improves the classification accuracy greatly with the newly added training samples, but also further improves the classification accuracy of the classifier by optimizing the augmented test samples. Second, an effective spectral structure extraction method is designed, and the effective spectral structure features have a better classification accuracy than the true spectral features.

Download Full-text

A data-driven neural network architecture for sentiment analysis

Data Technologies and Applications ◽

10.1108/dta-03-2018-0017 ◽

2019 ◽

Vol 53 (1) ◽

pp. 2-19 ◽

Cited By ~ 1

Author(s):

Erion Çano ◽

Maurizio Morisio

Keyword(s):

Neural Network ◽

Sentiment Analysis ◽

Network Architecture ◽

Network Models ◽

Data Sets ◽

Feature Maps ◽

Neural Network Architecture ◽

Neural Network Models ◽

Content Type ◽

Max Pooling

Purpose The fabulous results of convolution neural networks in image-related tasks attracted attention of text mining, sentiment analysis and other text analysis researchers. It is, however, difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. The purpose of this paper is to present the creation steps of two big data sets of song emotions. The authors also explore usage of convolution and max-pooling neural layers on song lyrics, product and movie review text data sets. Three variants of a simple and flexible neural network architecture are also compared. Design/methodology/approach The intention was to spot any important patterns that can serve as guidelines for parameter optimization of similar models. The authors also wanted to identify architecture design choices which lead to high performing sentiment analysis models. To this end, the authors conducted a series of experiments with neural architectures of various configurations. Findings The results indicate that parallel convolutions of filter lengths up to 3 are usually enough for capturing relevant text features. Also, max-pooling region size should be adapted to the length of text documents for producing the best feature maps. Originality/value Top results the authors got are obtained with feature maps of lengths 6–18. An improvement on future neural network models for sentiment analysis could be generating sentiment polarity prediction of documents using aggregation of predictions on smaller excerpt of the entire text.

Download Full-text

Deep Spectral Spatial Inverted Residual Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs13214472 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4472

Author(s):

Tianyu Zhang ◽

Cuiping Shi ◽

Diling Liao ◽

Liguo Wang

Keyword(s):

Image Classification ◽

Data Augmentation ◽

Hyperspectral Image ◽

Classification Performance ◽

Hyperspectral Data ◽

Hyperspectral Images ◽

Hyperspectral Image Classification ◽

Feature Maps ◽

Spatial Feature ◽

Global Context

Convolutional neural networks (CNNs) have been widely used in hyperspectral image classification in recent years. The training of CNNs relies on a large amount of labeled sample data. However, the number of labeled samples of hyperspectral data is relatively small. Moreover, for hyperspectral images, fully extracting spectral and spatial feature information is the key to achieve high classification performance. To solve the above issues, a deep spectral spatial inverted residuals network (DSSIRNet) is proposed. In this network, a data block random erasing strategy is introduced to alleviate the problem of limited labeled samples by data augmentation of small spatial blocks. In addition, a deep inverted residuals (DIR) module for spectral spatial feature extraction is proposed, which locks the effective features of each layer while avoiding network degradation. Furthermore, a global 3D attention module is proposed, which can realize the fine extraction of spectral and spatial global context information under the condition of the same number of input and output feature maps. Experiments are carried out on four commonly used hyperspectral datasets. A large number of experimental results show that compared with some state-of-the-art classification methods, the proposed method can provide higher classification accuracy for hyperspectral images.

Download Full-text

A DIVERSIFIED DEEP BELIEF NETWORK FOR HYPERSPECTRAL IMAGE CLASSIFICATION

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b7-443-2016 ◽

2016 ◽

Vol XLI-B7 ◽

pp. 443-449

Author(s):

P. Zhong ◽

Z. Q. Gong ◽

C. Schönlieb

Keyword(s):

Image Classification ◽

Real World ◽

Hyperspectral Image ◽

Classification Performance ◽

Fine Tuning ◽

Hyperspectral Image Classification ◽

Latent Factors ◽

Deep Model ◽

Training Samples ◽

Limited Training Samples

In recent years, researches in remote sensing demonstrated that deep architectures with multiple layers can potentially extract abstract and invariant features for better hyperspectral image classification. Since the usual real-world hyperspectral image classification task cannot provide enough training samples for a supervised deep model, such as convolutional neural networks (CNNs), this work turns to investigate the deep belief networks (DBNs), which allow unsupervised training. The DBN trained over limited training samples usually has many “dead” (never responding) or “potential over-tolerant” (always responding) latent factors (neurons), which decrease the DBN’s description ability and thus finally decrease the hyperspectral image classification performance. This work proposes a new diversified DBN through introducing a diversity promoting prior over the latent factors during the DBN pre-training and fine-tuning procedures. The diversity promoting prior in the training procedures will encourage the latent factors to be uncorrelated, such that each latent factor focuses on modelling unique information, and all factors will be summed up to capture a large proportion of information and thus increase description ability and classification performance of the diversified DBNs. The proposed method was evaluated over the well-known real-world hyperspectral image dataset. The experiments demonstrate that the diversified DBNs can obtain much better results than original DBNs and comparable or even better performances compared with other recent hyperspectral image classification methods.

Download Full-text

Individual differences among deep neural network models

10.1101/2020.01.08.898288 ◽

2020 ◽

Cited By ~ 4

Author(s):

Johannes Mehrer ◽

Courtney J. Spoerer ◽

Nikolaus Kriegeskorte ◽

Tim C. Kietzmann

Keyword(s):

Individual Differences ◽

Visual Recognition ◽

Initial Conditions ◽

Network Models ◽

Classification Performance ◽

Neural Network Models ◽

Multiple Networks ◽

Modelling Framework ◽

Neural Computations ◽

Category Exemplars

AbstractDeep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modelling framework for neural computations in the primate brain. However, each DNN instance, just like each individual brain, has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using representational similarity analysis, we demonstrate that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations, despite achieving indistinguishable network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than a misalignment of category centroids. Furthermore, while network regularization can increase the consistency of learned representations, considerable differences remain. These results suggest that computational neuroscientists working with DNNs should base their inferences on multiple networks instances instead of single off-the-shelf networks.

Download Full-text

Individual differences among deep neural network models

Nature Communications ◽

10.1038/s41467-020-19632-w ◽

2020 ◽

Vol 11 (1) ◽

Author(s):

Johannes Mehrer ◽

Courtney J. Spoerer ◽

Nikolaus Kriegeskorte ◽

Tim C. Kietzmann

Keyword(s):

Individual Differences ◽

Visual Recognition ◽

Initial Conditions ◽

Network Models ◽

Classification Performance ◽

Neural Network Models ◽

Modeling Framework ◽

Neural Computations ◽

Neural Information ◽

Multiple Network

AbstractDeep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modeling framework for neural computations in the primate brain. Just like individual brains, each DNN has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using tools typically employed in systems neuroscience, we show that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations despite similar network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than misaligned category centroids. These results call into question the common practice of using single networks to derive insights into neural information processing and rather suggest that computational neuroscientists working with DNNs may need to base their inferences on groups of multiple network instances.

Download Full-text