HYPERSPECTRAL IMAGE CLASSIFICATION USING RESIDUAL 2D AND 3D CONVOLUTIONAL NEURAL NETWORK JOINT ATTENTION MODEL

Abstract. Hyperspectral image classification (HSIC) is a challenging task in remote sensing data analysis, which has been applied in many domains for better identification and inspection of the earth surface by extracting spectral and spatial information. The combination of abundant spectral features and accurate spatial information can improve classification accuracy. However, many traditional methods are based on handcrafted features, which brings difficulties for multi-classification tasks due to spectral intra-class heterogeneity and similarity of inter-class. The deep learning algorithm, especially the convolutional neural network (CNN), has been perceived promising feature extractor and classification for processing hyperspectral remote sensing images. Although 2D CNN can extract spatial features, the specific spectral properties are not used effectively. While 3D CNN has the capability for them, but the computational burden increases as stacking layers. To address these issues, we propose a novel HSIC framework based on the residual CNN network by integrating the advantage of 2D and 3D CNN. First, 3D convolutions focus on extracting spectral features with feature recalibration and refinement by channel attention mechanism. The 2D depth-wise separable convolution approach with different size kernels concentrates on obtaining multi-scale spatial features and reducing model parameters. Furthermore, the residual structure optimizes the back-propagation for network training. The results and analysis of extensive HSIC experiments show that the proposed residual 2D-3D CNN network can effectively extract spectral and spatial features and improve classification accuracy.

Download Full-text

Hyperspectral Image Classification Based on Spectral and Spatial Information Using Multi-Scale ResNet

Applied Sciences ◽

10.3390/app9224890 ◽

2019 ◽

Vol 9 (22) ◽

pp. 4890 ◽

Cited By ~ 2

Author(s):

Zong-Yue Wang ◽

Qi-Ming Xia ◽

Jing-Wen Yan ◽

Shu-Qi Xuan ◽

Jin-He Su ◽

...

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Computational Cost ◽

Data Cube ◽

Hyperspectral Image Classification ◽

Fixed Size ◽

Multi Scale

Hyperspectral imaging (HSI) contains abundant spectrums as well as spatial information, providing a great basis for classification in the field of remote sensing. In this paper, to make full use of HSI information, we combined spectral and spatial information into a two-dimension image in a particular order by extracting a data cube and unfolding it. Prior to the step of combining, principle component analysis (PCA) is utilized to decrease the dimensions of HSI so as to reduce computational cost. Moreover, the classification block used during the experiment is a convolutional neural network (CNN). Instead of using traditionally fixed-size kernels in CNN, we leverage a multi-scale kernel in the first convolutional layer so that it can scale to the receptive field. To attain higher classification accuracy with deeper layers, residual blocks are also applied to the network. Extensive experiments on the datasets from Pavia University and Salinas demonstrate that the proposed method significantly improves the accuracy in HSI classification.

Download Full-text

ODPA-CNN: One Dimensional Parallel Atrous Convolution Neural Network for Band-Selective Hyperspectral Image Classification

Applied Sciences ◽

10.3390/app12010174 ◽

2021 ◽

Vol 12 (1) ◽

pp. 174

Author(s):

Byungjin Kang ◽

Inho Park ◽

Changmin Ok ◽

Sungho Kim

Keyword(s):

Neural Network ◽

Spatial Information ◽

Hyperspectral Image ◽

Convolution Neural Network ◽

Spectral Information ◽

Hyperspectral Image Classification ◽

One Dimensional ◽

Proposed Model ◽

3D Cnn ◽

2D And 3D

Recently, hyperspectral image (HSI) classification using deep learning has been actively studied using 2D and 3D convolution neural networks (CNN). However, they learn spatial information as well as spectral information. These methods can increase the accuracy of classification, but do not only focus on the spectral information, which is a big advantage of HSI. In addition, the 1D-CNN, which learns only pure spectral information, has limitations because it uses adjacent spectral information. In this paper, we propose a One Dimensional Parellel Atrous Convolution Neural Network (ODPA-CNN) that learns not only adjacent spectral information for HSI classification, but also spectral information from a certain distance. It extracts features in parallel to account for bands of varying distances. The proposed method excludes spatial information such as the shape of an object and performs HSI classification only with spectral information about the material of the object. Atrous convolution is not a convolution of adjacent spectral information, but a convolution between spectral information separated by a certain distance. We compare the proposed model with various datasets to the other models. We also test with the data we have taken ourselves. Experimental results show a higher performance than some 3D-CNN models and other 1D-CNN methods. In addition, using datasets to which random space is applied, the vulnerabilities of 3D-CNN are identified, and the proposed model is shown to be robust to datasets with little spatial information.

Download Full-text

A Lightweight Spectral–Spatial Feature Extraction and Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12091395 ◽

2020 ◽

Vol 12 (9) ◽

pp. 1395

Author(s):

Linlin Chen ◽

Zhihui Wei ◽

Yang Xu

Keyword(s):

Classification Accuracy ◽

Hyperspectral Image ◽

State Of The Art ◽

Spectral Features ◽

Classification Methods ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Deep Network ◽

Spatial Features ◽

Reduced Parameters

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Download Full-text

A Multi-Scale and Multi-Level Spectral-Spatial Feature Fusion Network for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12010125 ◽

2020 ◽

Vol 12 (1) ◽

pp. 125 ◽

Cited By ~ 5

Author(s):

Mu ◽

Guo ◽

Liu

Keyword(s):

Neural Network ◽

Hyperspectral Image ◽

Feature Fusion ◽

Hyperspectral Images ◽

Spectral Features ◽

Hyperspectral Image Classification ◽

Spatial Feature ◽

Multi Scale ◽

Spatial Features ◽

Multi Level

Extracting spatial and spectral features through deep neural networks has become an effective means of classification of hyperspectral images. However, most networks rarely consider the extraction of multi-scale spatial features and cannot fully integrate spatial and spectral features. In order to solve these problems, this paper proposes a multi-scale and multi-level spectral-spatial feature fusion network (MSSN) for hyperspectral image classification. The network uses the original 3D cube as input data and does not need to use feature engineering. In the MSSN, using different scale neighborhood blocks as the input of the network, the spectral-spatial features of different scales can be effectively extracted. The proposed 3D–2D alternating residual block combines the spectral features extracted by the three-dimensional convolutional neural network (3D-CNN) with the spatial features extracted by the two-dimensional convolutional neural network (2D-CNN). It not only achieves the fusion of spectral features and spatial features but also achieves the fusion of high-level features and low-level features. Experimental results on four hyperspectral datasets show that this method is superior to several state-of-the-art classification methods for hyperspectral images.

Download Full-text

Combining 3D-CNN and Squeeze-and-Excitation Networks for Remote Sensing Sea Ice Image Classification

Mathematical Problems in Engineering ◽

10.1155/2020/8065396 ◽

2020 ◽

Vol 2020 ◽

pp. 1-15

Author(s):

Yanling Han ◽

Cong Wei ◽

Ruyan Zhou ◽

Zhonghua Hong ◽

Yun Zhang ◽

...

Keyword(s):

Remote Sensing ◽

Sea Ice ◽

Image Classification ◽

Classification Accuracy ◽

Spatial Information ◽

Spatial Spectrum ◽

Small Samples ◽

Support Vector ◽

Spectral Features ◽

3D Cnn

Sea ice is one of the most prominent marine disasters in high latitudes. Remote sensing technology provides an effective means for sea ice detection. Remote sensing sea ice images contain rich spectral and spatial information. However, most traditional methods only focus on spectral information or spatial information, and do not excavate the feature of spectral and spatial simultaneously in remote sensing sea ice images classification. At the same time, the complex correlation characteristics among spectra and small sample problem in sea ice classification also limit the improvement of sea ice classification accuracy. For this issue, this paper proposes a new remote sensing sea ice image classification method based on squeeze-and-excitation (SE) network, convolutional neural network (CNN), and support vector machines (SVMs). The proposed method designs 3D-CNN deep network so as to fully exploit the spatial-spectrum features of remote sensing sea ice images and integrates SE-Block into 3D-CNN in-depth network in order to distinguish the contributions of different spectra to sea ice classification. According to the different contributions of spectral features, the weight of each spectral feature is optimized by fusing SE-Block in order to further enhance the sample quality. Finally, information-rich and representative samples are chosen by combining the idea of active learning and input into SVM classifier, and this achieves superior classification accuracy of remote sensing sea ice images with small samples. In order to verify the effectiveness of the proposed method, we conducted experiments on three different data from Baffin Bay, Bohai Bay, and Liaodong Bay. The experimental results show that compared with other classical classification methods, the proposed method comprehensively considers the correlation among spectral features and the small samples problems and deeply excavates the spatial-spectrum characteristics of sea ice and achieves better classification performance, which can be effectively applied to remote sensing sea ice image classification.

Download Full-text

Approaches for Hyperspectral Image Classification Detailed Review

International Journal of Soft Computing and Engineering - Regular Issue ◽

10.35940/ijsce.a3522.0911121 ◽

2021 ◽

Vol 11 (1) ◽

pp. 13-22

Author(s):

Kushalatha M R ◽

◽

Prasantha H S ◽

Beena R. Shetty ◽

◽

...

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Image Classification ◽

Spatial Information ◽

Hyperspectral Image ◽

Support Vector ◽

Electromagnetic Spectrum ◽

Hyperspectral Image Classification ◽

Sensing Applications ◽

Remote Sensing Applications

Hyperspectral Image (HSI) processing is the new advancement in image / signal processing field. The growth over the years is appreciable. The main reason behind the successful growth of the Hyperspectral imaging field is due to the enormous amount of spectral and spatial information that the imagery contains. The spectral band that the HSI which contains is also more in number. When an image is captured through the HSI cameras, it contains around 200-250 images of the same scene. Nowadays HSI is used extensively in the fields of environmental monitoring, Crop-Field monitoring, Classification, Identification, Remote sensing applications, Surveillance etc. The spectral and spatial information content present in Hyperspectral images are with high resolutions.Hyperspectral imaging has shown significant growth and widely used in most of the remote sensing applications due to its presence of information of a scene over hundreds of contiguous bands In. Hyperspectral Image Classification of materials is the critical application of HSI using Hyperspectral sensors. It collects hundreds of spectrum channels, where each channel consists of a sharp point of Electromagnetic Spectrum. The paper mainly focuses on Deep Learning techniques such as Convolutional Neural Network (CNN), Artificial Neural Network (ANN), and Support Vector machines (SVM), K-Nearest Neighbour (KNN) for the accuracy in classification. Finally in the summary the current state-of-the-art scheme, a critical discussion after reviewing the research work by other professionals and organizing it into review-based paper, also implying about the present status on classification accuracy using neural networks is carried out.

Download Full-text

Spectral-Spatial Joint Classification of Hyperspectral Image Based on Broad Learning System

Remote Sensing ◽

10.3390/rs13040583 ◽

2021 ◽

Vol 13 (4) ◽

pp. 583

Author(s):

Guixin Zhao ◽

Xuesong Wang ◽

Yi Kong ◽

Yuhu Cheng

Keyword(s):

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Learning System ◽

Classification Model ◽

Spectral Features ◽

Gaussian Filter ◽

Spatial Features ◽

Joint Classification

At present many researchers pay attention to a combination of spectral features and spatial features to enhance hyperspectral image (HSI) classification accuracy. However, the spatial features in some methods are utilized insufficiently. In order to further improve the performance of HSI classification, the spectral-spatial joint classification of HSI based on the broad learning system (BLS) (SSBLS) method was proposed in this paper; it consists of three parts. Firstly, the Gaussian filter is adopted to smooth each band of the original spectra based on the spatial information to remove the noise. Secondly, the test sample’s labels can be obtained using the optimal BLS classification model trained with the spectral features smoothed by the Gaussian filter. At last, the guided filter is performed to correct the BLS classification results based on the spatial contextual information for improving the classification accuracy. Experiment results on the three real HSI datasets demonstrate that the mean overall accuracies (OAs) of ten experiments are 99.83% on the Indian Pines dataset, 99.96% on the Salinas dataset, and 99.49% on the Pavia University dataset. Compared with other methods, the proposed method in the paper has the best performance.

Download Full-text

Hyperspectral Image Classification Based on Multi-Scale Residual Network with Attention Mechanism

Remote Sensing ◽

10.3390/rs13030335 ◽

2021 ◽

Vol 13 (3) ◽

pp. 335

Author(s):

Yuhao Qing ◽

Wenyi Liu

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Image Classification ◽

Classification Accuracy ◽

Hyperspectral Image ◽

Principal Component ◽

Hyperspectral Image Classification ◽

Deep Network ◽

Multi Scale

In recent years, image classification on hyperspectral imagery utilizing deep learning algorithms has attained good results. Thus, spurred by that finding and to further improve the deep learning classification accuracy, we propose a multi-scale residual convolutional neural network model fused with an efficient channel attention network (MRA-NET) that is appropriate for hyperspectral image classification. The suggested technique comprises a multi-staged architecture, where initially the spectral information of the hyperspectral image is reduced into a two-dimensional tensor, utilizing a principal component analysis (PCA) scheme. Then, the constructed low-dimensional image is input to our proposed ECA-NET deep network, which exploits the advantages of its core components, i.e., multi-scale residual structure and attention mechanisms. We evaluate the performance of the proposed MRA-NET on three public available hyperspectral datasets and demonstrate that, overall, the classification accuracy of our method is 99.82 %, 99.81%, and 99.37, respectively, which is higher compared to the corresponding accuracy of current networks such as 3D convolutional neural network (CNN), three-dimensional residual convolution structure (RES-3D-CNN), and space–spectrum joint deep network (SSRN).

Download Full-text

Comparison of CNN Algorithms on Hyperspectral Image Classification in Agricultural Lands

Sensors ◽

10.3390/s20061734 ◽

2020 ◽

Vol 20 (6) ◽

pp. 1734 ◽

Cited By ~ 4

Author(s):

Tien-Heng Hsieh ◽

Jean-Fu Kiang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Principal Components ◽

Hyperspectral Image ◽

Hyperspectral Images ◽

Agricultural Lands ◽

Hyperspectral Image Classification ◽

Spatial Features ◽

Salinas Valley ◽

Crop Agriculture

Several versions of convolutional neural network (CNN) were developed to classify hyperspectral images (HSIs) of agricultural lands, including 1D-CNN with pixelwise spectral data, 1D-CNN with selected bands, 1D-CNN with spectral-spatial features and 2D-CNN with principal components. The HSI data of a crop agriculture in Salinas Valley and a mixed vegetation agriculture in Indian Pines were used to compare the performance of these CNN algorithms. The highest overall accuracy on these two cases are 99.8% and 98.1%, respectively, achieved by applying 1D-CNN with augmented input vectors, which contain both spectral and spatial features embedded in the HSI data.

Download Full-text

Hyperspectral Image Classification Based on Sparse Superpixel Graph

Remote Sensing ◽

10.3390/rs13183592 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3592

Author(s):

Yifei Zhao ◽

Fengqin Yan

Keyword(s):

Classification Accuracy ◽

Spatial Information ◽

Hyperspectral Image ◽

Hyperspectral Images ◽

Spectral Information ◽

Hyperspectral Image Classification ◽

Computational Burden ◽

Large Size ◽

High Classification Accuracy ◽

Good Classification

Hyperspectral image (HSI) classification is one of the major problems in the field of remote sensing. Particularly, graph-based HSI classification is a promising topic and has received increasing attention in recent years. However, graphs with pixels as nodes generate large size graphs, thus increasing the computational burden. Moreover, satisfactory classification results are often not obtained without considering spatial information in constructing graph. To address these issues, this study proposes an efficient and effective semi-supervised spectral-spatial HSI classification method based on sparse superpixel graph (SSG). In the constructed sparse superpixels graph, each vertex represents a superpixel instead of a pixel, which greatly reduces the size of graph. Meanwhile, both spectral information and spatial structure are considered by using superpixel, local spatial connection and global spectral connection. To verify the effectiveness of the proposed method, three real hyperspectral images, Indian Pines, Pavia University and Salinas, are chosen to test the performance of our proposal. Experimental results show that the proposed method has good classification completion on the three benchmarks. Compared with several competitive superpixel-based HSI classification approaches, the method has the advantages of high classification accuracy (>97.85%) and rapid implementation (<10 s). This clearly favors the application of the proposed method in practice.

Download Full-text