scholarly journals Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution

2020 ◽  
Vol 12 (10) ◽  
pp. 1660 ◽  
Author(s):  
Qiang Li ◽  
Qi Wang ◽  
Xuelong Li
Keyword(s):  
Spatial Information ◽  
Hyperspectral Image ◽  
Feature Fusion ◽  
Super Resolution ◽  
Superior Performance ◽  
Spectral Information ◽  
Great Success ◽  
Convolutional Network ◽  
Training Time ◽  
Image Super Resolution

Deep learning-based hyperspectral image super-resolution (SR) methods have achieved great success recently. However, there are two main problems in the previous works. One is to use the typical three-dimensional convolution analysis, resulting in more parameters of the network. The other is not to pay more attention to the mining of hyperspectral image spatial information, when the spectral information can be extracted. To address these issues, in this paper, we propose a mixed convolutional network (MCNet) for hyperspectral image super-resolution. We design a novel mixed convolutional module (MCM) to extract the potential features by 2D/3D convolution instead of one convolution, which enables the network to more mine spatial features of hyperspectral image. To explore the effective features from 2D unit, we design the local feature fusion to adaptively analyze from all the hierarchical features in 2D units. In 3D unit, we employ spatial and spectral separable 3D convolution to extract spatial and spectral information, which reduces unaffordable memory usage and training time. Extensive evaluations and comparisons on three benchmark datasets demonstrate that the proposed approach achieves superior performance in comparison to existing state-of-the-art methods.

scholarly journals Single Image Super-Resolution Based on Global Dense Feature Fusion Convolutional Network

Sensors ◽  
2019 ◽  
Vol 19 (2) ◽  
pp. 316 ◽  
Author(s):  
Wang Xu ◽  
Renwen Chen ◽  
Bin Huang ◽  
Xiang Zhang ◽  
Chuan Liu
Keyword(s):  
Feature Fusion ◽  
Super Resolution ◽  
Great Success ◽  
Single Image ◽  
Global Features ◽  
Convolutional Network ◽  
Memory Mechanism ◽  
Basic Module ◽  
Image Super Resolution ◽  
Single Image Super Resolution

Deep neural networks (DNNs) have been widely adopted in single image super-resolution (SISR) recently with great success. As a network goes deeper, intermediate features become hierarchical. However, most SISR methods based on DNNs do not make full use of the hierarchical features. The features cannot be read directly by the subsequent layers, therefore, the previous hierarchical information has little influence on the subsequent layer output, and the performance is relatively poor. To address this issue, a novel global dense feature fusion convolutional network (DFFNet) is proposed, which can take full advantage of global intermediate features. Especially, a feature fusion block (FFblock) is introduced as the basic module. Each block can directly read raw global features from previous ones and then learns the feature spatial correlation and channel correlation between features in a holistic way, leading to a continuous global information memory mechanism. Experiments on the benchmark tests show that the proposed method DFFNet achieves favorable performance against the state-of-art methods.

scholarly journals Hyperspectral Image Super-Resolution with 1D–2D Attentional Convolutional Neural Network

2019 ◽  
Vol 11 (23) ◽  
pp. 2859 ◽  
Author(s):  
Jiaojiao Li ◽  
Ruxing Cui ◽  
Bo Li ◽  
Rui Song ◽  
Yunsong Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Spatial Information ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
High Resolution Images ◽  
Image Super Resolution ◽  
Small Change ◽  
Single Image Super Resolution ◽  
Separation Strategy

Hyperspectral image (HSI) super-resolution (SR) is of great application value and has attracted broad attention. The hyperspectral single image super-resolution (HSISR) task is correspondingly difficult in SR due to the unavailability of auxiliary high resolution images. To tackle this challenging task, different from the existing learning-based HSISR algorithms, in this paper we propose a novel framework, i.e., a 1D–2D attentional convolutional neural network, which employs a separation strategy to extract the spatial–spectral information and then fuse them gradually. More specifically, our network consists of two streams: a spatial one and a spectral one. The spectral one is mainly composed of the 1D convolution to encode a small change in the spectrum, while the 2D convolution, cooperating with the attention mechanism, is used in the spatial pathway to encode spatial information. Furthermore, a novel hierarchical side connection strategy is proposed for effectively fusing spectral and spatial information. Compared with the typical 3D convolutional neural network (CNN), the 1D–2D CNN is easier to train with less parameters. More importantly, our proposed framework can not only present a perfect solution for the HSISR problem, but also explore the potential in hyperspectral pansharpening. The experiments over widely used benchmarks on SISR and hyperspectral pansharpening demonstrate that the proposed method could outperform other state-of-the-art methods, both in visual quality and quantity measurements.

scholarly journals Difference Curvature Multidimensional Network for Hyperspectral Image Super-Resolution

2021 ◽  
Vol 13 (17) ◽  
pp. 3455
Author(s):  
Chi Zhang ◽  
Mingjin Zhang ◽  
Yunsong Li ◽  
Xinbo Gao ◽  
Shi Qiu
Keyword(s):  
Hyperspectral Image ◽  
Dimensional Space ◽  
Super Resolution ◽  
Texture Features ◽  
High Dimensional ◽  
Great Success ◽  
Spectral Correlation ◽  
High Dimensional Space ◽  
Image Super Resolution ◽  
Difference Curvature

In recent years, convolutional-neural-network-based methods have been introduced to the field of hyperspectral image super-resolution following their great success in the field of RGB image super-resolution. However, hyperspectral images appear different from RGB images in that they have high dimensionality, implying a redundancy in the high-dimensional space. Existing approaches struggle in learning the spectral correlation and spatial priors, leading to inferior performance. In this paper, we present a difference curvature multidimensional network for hyperspectral image super-resolution that exploits the spectral correlation to help improve the spatial resolution. Specifically, we introduce a multidimensional enhanced convolution (MEC) unit into the network to learn the spectral correlation through a self-attention mechanism. Meanwhile, it reduces the redundancy in the spectral dimension via a bottleneck projection to condense useful spectral features and reduce computations. To remove the unrelated information in high-dimensional space and extract the delicate texture features of a hyperspectral image, we design an additional difference curvature branch (DCB), which works as an edge indicator to fully preserve the texture information and eliminate the unwanted noise. Experiments on three publicly available datasets demonstrate that the proposed method can recover sharper images with minimal spectral distortion compared to state-of-the-art methods. PSNR/SAM is 0.3–0.5 dB/0.2–0.4 better than the second best methods.

scholarly journals Densely Connected Pyramidal Dilated Convolutional Network for Hyperspectral Image Classification

2021 ◽  
Vol 13 (17) ◽  
pp. 3396
Author(s):  
Feng Zhao ◽  
Junjie Zhang ◽  
Zhe Meng ◽  
Hanqiang Liu
Keyword(s):  
Receptive Field ◽  
Spatial Information ◽  
Hyperspectral Image ◽  
Feature Fusion ◽  
Receptive Fields ◽  
Classification Performance ◽  
Convolutional Network ◽  
Dilated Convolution ◽  
Spatial Features ◽  
Good Classification Performance

Recently, with the extensive application of deep learning techniques in the hyperspectral image (HSI) field, particularly convolutional neural network (CNN), the research of HSI classification has stepped into a new stage. To avoid the problem that the receptive field of naive convolution is small, the dilated convolution is introduced into the field of HSI classification. However, the dilated convolution usually generates blind spots in the receptive field, resulting in discontinuous spatial information obtained. In order to solve the above problem, a densely connected pyramidal dilated convolutional network (PDCNet) is proposed in this paper. Firstly, a pyramidal dilated convolutional (PDC) layer integrates different numbers of sub-dilated convolutional layers is proposed, where the dilated factor of the sub-dilated convolution increases exponentially, achieving multi-sacle receptive fields. Secondly, the number of sub-dilated convolutional layers increases in a pyramidal pattern with the depth of the network, thereby capturing more comprehensive hyperspectral information in the receptive field. Furthermore, a feature fusion mechanism combining pixel-by-pixel addition and channel stacking is adopted to extract more abstract spectral–spatial features. Finally, in order to reuse the features of the previous layers more effectively, dense connections are applied in densely pyramidal dilated convolutional (DPDC) blocks. Experiments on three well-known HSI datasets indicate that PDCNet proposed in this paper has good classification performance compared with other popular models.

scholarly journals Hyperspectral Super-Resolution Via Joint Regularization of Low-Rank Tensor Decomposition

2021 ◽  
Vol 13 (20) ◽  
pp. 4116
Author(s):  
Meng Cao ◽  
Wenxing Bao ◽  
Kewen Qu
Keyword(s):  
High Resolution ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
Tensor Decomposition ◽  
Hyperspectral Data ◽  
Hyperspectral Images ◽  
Low Rank ◽  
Superior Performance ◽  
Rank Tensor ◽  
Image Super Resolution

The hyperspectral image super-resolution (HSI-SR) problem aims at reconstructing the high resolution spatial–spectral information of the scene by fusing low-resolution hyperspectral images (LR-HSI) and the corresponding high-resolution multispectral image (HR-MSI). In order to effectively preserve the spatial and spectral structure of hyperspectral images, a new joint regularized low-rank tensor decomposition method (JRLTD) is proposed for HSI-SR. This model alleviates the problem that the traditional HSI-SR method, based on tensor decomposition, fails to adequately take into account the manifold structure of high-dimensional HR-HSI and is sensitive to outliers and noise. The model first operates on the hyperspectral data using the classical Tucker decomposition to transform the hyperspectral data into the form of a three-mode dictionary multiplied by the core tensor, after which the graph regularization and unidirectional total variational (TV) regularization are introduced to constrain the three-mode dictionary. In addition, we impose the l1-norm on core tensor to characterize the sparsity. While effectively preserving the spatial and spectral structures in the fused hyperspectral images, the presence of anomalous noise values in the images is reduced. In this paper, the hyperspectral image super-resolution problem is transformed into a joint regularization optimization problem based on tensor decomposition and solved by a hybrid framework between the alternating direction multiplier method (ADMM) and the proximal alternate optimization (PAO) algorithm. Experimental results conducted on two benchmark datasets and one real dataset show that JRLTD shows superior performance over state-of-the-art hyperspectral super-resolution algorithms.

scholarly journals Hyperspectral Image Super-Resolution by Deep Spatial-Spectral Exploitation

2019 ◽  
Vol 11 (10) ◽  
pp. 1229 ◽  
Author(s):  
Jing Hu ◽  
Minghua Zhao ◽  
Yunsong Li
Keyword(s):  
Spatial Resolution ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
Input Image ◽  
High Spectral Resolution ◽  
Spectral Information ◽  
Spectral Correlation ◽  
Fusion Operation ◽  
Image Super Resolution ◽  
Sr Method

Limited by the existing imagery sensors, hyperspectral images are characterized by high spectral resolution but low spatial resolution. The super-resolution (SR) technique aiming at enhancing the spatial resolution of the input image is a hot topic in computer vision. In this paper, we present a hyperspectral image (HSI) SR method based on a deep information distillation network (IDN) and an intra-fusion operation. Specifically, bands are firstly selected by a certain distance and super-resolved by an IDN. The IDN employs distillation blocks to gradually extract abundant and efficient features for reconstructing the selected bands. Second, the unselected bands are obtained via spectral correlation, yielding a coarse high-resolution (HR) HSI. Finally, the spectral-interpolated coarse HR HSI is intra-fused with the input HSI to achieve a finer HR HSI, making further use of the spatial-spectral information these unselected bands convey. Different from most existing fusion-based HSI SR methods, the proposed intra-fusion operation does not require any auxiliary co-registered image as the input, which makes this method more practical. Moreover, contrary to most single-based HSI SR methods whose performance decreases significantly as the image quality gets worse, the proposal deeply utilizes the spatial-spectral information and the mapping knowledge provided by the IDN, which achieves more robust performance. Experimental data and comparative analysis have demonstrated the effectiveness of this method.

scholarly journals Hyperspectral Image Super-Resolution under the Guidance of Deep Gradient Information

2021 ◽  
Vol 13 (12) ◽  
pp. 2382
Author(s):  
Minghua Zhao ◽  
Jiawei Ning ◽  
Jing Hu ◽  
Tingting Li
Keyword(s):  
Spatial Information ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
Spatial Gradient ◽  
Dense Network ◽  
Discriminative Ability ◽  
Resolution Process ◽  
Benchmark Datasets ◽  
Image Super Resolution ◽  
Hierarchical Features

Hyperspectral image (HSI) super-resolution has gained great attention in remote sensing, due to its effectiveness in enhancing the spatial information of the HSI while preserving the high spectral discriminative ability, without modifying the imagery hardware. In this paper, we proposed a novel HSI super-resolution method via a gradient-guided residual dense network (G-RDN), in which the spatial gradient is exploited to guide the super-resolution process. Specifically, there are three modules in the super-resolving process. Firstly, the spatial mapping between the low-resolution HSI and the desired high-resolution HSI is learned via a residual dense network. The residual dense network is used to fully exploit the hierarchical features learned from all the convolutional layers. Meanwhile, the gradient detail is extracted via a residual network (ResNet), which is further utilized to guide the super-resolution process. Finally, an empirical weight is set between the fully obtained global hierarchical features and the gradient details. Experimental results and the data analysis on three benchmark datasets with different scaling factors demonstrated that our proposed G-RDN achieved favorable performance.

Sign in / Sign up

Export Citation Format

Share Document