scholarly journals Hyperspectral Image Super-Resolution Based on Spatial Group Sparsity Regularization Unmixing

2020 ◽  
Vol 10 (16) ◽  
pp. 5583 ◽  
Author(s):  
Jun Li ◽  
Yuanxi Peng ◽  
Tian Jiang ◽  
Longlong Zhang ◽  
Jian Long
Keyword(s):  
Spatial Resolution ◽  
Spatial Information ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
Superior Performance ◽  
High Spectral Resolution ◽  
Data Sets ◽  
Group Sparsity ◽  
Sparsity Regularization ◽  
High Spatial Resolution Image

A hyperspectral image (HSI) contains many narrow spectral channels, thus containing efficient information in the spectral domain. However, high spectral resolution usually leads to lower spatial resolution as a result of the limitations of sensors. Hyperspectral super-resolution aims to fuse a low spatial resolution HSI with a conventional high spatial resolution image, producing an HSI with high resolution in both the spectral and spatial dimensions. In this paper, we propose a spatial group sparsity regularization unmixing-based method for hyperspectral super-resolution. The hyperspectral image (HSI) is pre-clustered using an improved Simple Linear Iterative Clustering (SLIC) superpixel algorithm to make full use of the spatial information. A robust sparse hyperspectral unmixing method is then used to unmix the input images. Then, the endmembers extracted from the HSI and the abundances extracted from the conventional image are fused. This ensures that the method makes full use of the spatial structure and the spectra of the images. The proposed method is compared with several related methods on public HSI data sets. The results demonstrate that the proposed method has superior performance when compared to the existing state-of-the-art.

scholarly journals Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution

2020 ◽  
Vol 12 (10) ◽  
pp. 1660 ◽  
Author(s):  
Qiang Li ◽  
Qi Wang ◽  
Xuelong Li
Keyword(s):  
Spatial Information ◽  
Hyperspectral Image ◽  
Feature Fusion ◽  
Super Resolution ◽  
Superior Performance ◽  
Spectral Information ◽  
Great Success ◽  
Convolutional Network ◽  
Training Time ◽  
Image Super Resolution

Deep learning-based hyperspectral image super-resolution (SR) methods have achieved great success recently. However, there are two main problems in the previous works. One is to use the typical three-dimensional convolution analysis, resulting in more parameters of the network. The other is not to pay more attention to the mining of hyperspectral image spatial information, when the spectral information can be extracted. To address these issues, in this paper, we propose a mixed convolutional network (MCNet) for hyperspectral image super-resolution. We design a novel mixed convolutional module (MCM) to extract the potential features by 2D/3D convolution instead of one convolution, which enables the network to more mine spatial features of hyperspectral image. To explore the effective features from 2D unit, we design the local feature fusion to adaptively analyze from all the hierarchical features in 2D units. In 3D unit, we employ spatial and spectral separable 3D convolution to extract spatial and spectral information, which reduces unaffordable memory usage and training time. Extensive evaluations and comparisons on three benchmark datasets demonstrate that the proposed approach achieves superior performance in comparison to existing state-of-the-art methods.

scholarly journals Self-Dictionary Regression for Hyperspectral Image Super-Resolution

2018 ◽  
Vol 10 (10) ◽  
pp. 1574 ◽  
Author(s):  
Dongsheng Gao ◽  
Zhentao Hu ◽  
Renzhen Ye
Keyword(s):  
High Resolution ◽  
Spatial Resolution ◽  
Spectral Resolution ◽  
Cross Correlation ◽  
Hyperspectral Image ◽  
High Spatial Resolution ◽  
Super Resolution ◽  
High Spectral Resolution ◽  
Low Resolution ◽  
Image Super Resolution

Due to sensor limitations, hyperspectral images (HSIs) are acquired by hyperspectral sensors with high-spectral-resolution but low-spatial-resolution. It is difficult for sensors to acquire images with high-spatial-resolution and high-spectral-resolution simultaneously. Hyperspectral image super-resolution tries to enhance the spatial resolution of HSI by software techniques. In recent years, various methods have been proposed to fuse HSI and multispectral image (MSI) from an unmixing or a spectral dictionary perspective. However, these methods extract the spectral information from each image individually, and therefore ignore the cross-correlation between the observed HSI and MSI. It is difficult to achieve high-spatial-resolution while preserving the spatial-spectral consistency between low-resolution HSI and high-resolution HSI. In this paper, a self-dictionary regression based method is proposed to utilize cross-correlation between the observed HSI and MSI. Both the observed low-resolution HSI and MSI are simultaneously considered to estimate the endmember dictionary and the abundance code. To preserve the spectral consistency, the endmember dictionary is extracted by performing a common sparse basis selection on the concatenation of observed HSI and MSI. Then, a consistent constraint is exploited to ensure the spatial consistency between the abundance code of low-resolution HSI and the abundance code of high-resolution HSI. Extensive experiments on three datasets demonstrate that the proposed method outperforms the state-of-the-art methods.

scholarly journals Hyperspectral Image Super-Resolution by Deep Spatial-Spectral Exploitation

2019 ◽  
Vol 11 (10) ◽  
pp. 1229 ◽  
Author(s):  
Jing Hu ◽  
Minghua Zhao ◽  
Yunsong Li
Keyword(s):  
Spatial Resolution ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
Input Image ◽  
High Spectral Resolution ◽  
Spectral Information ◽  
Spectral Correlation ◽  
Fusion Operation ◽  
Image Super Resolution ◽  
Sr Method

Limited by the existing imagery sensors, hyperspectral images are characterized by high spectral resolution but low spatial resolution. The super-resolution (SR) technique aiming at enhancing the spatial resolution of the input image is a hot topic in computer vision. In this paper, we present a hyperspectral image (HSI) SR method based on a deep information distillation network (IDN) and an intra-fusion operation. Specifically, bands are firstly selected by a certain distance and super-resolved by an IDN. The IDN employs distillation blocks to gradually extract abundant and efficient features for reconstructing the selected bands. Second, the unselected bands are obtained via spectral correlation, yielding a coarse high-resolution (HR) HSI. Finally, the spectral-interpolated coarse HR HSI is intra-fused with the input HSI to achieve a finer HR HSI, making further use of the spatial-spectral information these unselected bands convey. Different from most existing fusion-based HSI SR methods, the proposed intra-fusion operation does not require any auxiliary co-registered image as the input, which makes this method more practical. Moreover, contrary to most single-based HSI SR methods whose performance decreases significantly as the image quality gets worse, the proposal deeply utilizes the spatial-spectral information and the mapping knowledge provided by the IDN, which achieves more robust performance. Experimental data and comparative analysis have demonstrated the effectiveness of this method.

scholarly journals Hyperspectral Image Super-Resolution Inspired by Deep Laplacian Pyramid Network

2018 ◽  
Vol 10 (12) ◽  
pp. 1939 ◽  
Author(s):  
Zhi He ◽  
Lin Liu
Keyword(s):  
Spatial Resolution ◽  
Hyperspectral Image ◽  
Spectral Characteristics ◽  
Super Resolution ◽  
High Spectral Resolution ◽  
Spectral Information ◽  
Laplacian Pyramid ◽  
Multiple Observations ◽  
Learned Dictionary ◽  
Coarse To Fine

Existing hyperspectral sensors usually produce high-spectral-resolution but low-spatial-resolution images, and super-resolution has yielded impressive results in improving the resolution of the hyperspectral images (HSIs). However, most of the super-resolution methods require multiple observations of the same scene and improve the spatial resolution without fully considering the spectral information. In this paper, we propose an HSI super-resolution method inspired by the deep Laplacian pyramid network (LPN). First, the spatial resolution is enhanced by an LPN, which can exploit the knowledge from natural images without using any auxiliary observations. The LPN progressively reconstructs the high-spatial-resolution images in a coarse-to-fine fashion by using multiple pyramid levels. Second, spectral characteristics between the low- and high-resolution HSIs are studied by the non-negative dictionary learning (NDL), which is proposed to learn the common dictionary with non-negative constraints. The super-resolution results can finally be obtained by multiplying the learned dictionary and its corresponding sparse codes. Experimental results on three hyperspectral datasets demonstrate the feasibility of the proposed method in enhancing the spatial resolution of the HSI with preserving the spectral information simultaneously.

scholarly journals Domain Transfer Learning for Hyperspectral Image Super-Resolution

2019 ◽  
Vol 11 (6) ◽  
pp. 694 ◽  
Author(s):  
Xiaoyan Li ◽  
Lefei Zhang ◽  
Jane You
Keyword(s):  
High Resolution ◽  
Transfer Learning ◽  
Spatial Resolution ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
High Spectral Resolution ◽  
Low Resolution ◽  
Image Super Resolution ◽  
Domain Transfer ◽  
Domain Transfer Learning

A Hyperspectral Image (HSI) contains a great number of spectral bands for each pixel; however, the spatial resolution of HSI is low. Hyperspectral image super-resolution is effective to enhance the spatial resolution while preserving the high-spectral-resolution by software techniques. Recently, the existing methods have been presented to fuse HSI and Multispectral Images (MSI) by assuming that the MSI of the same scene is required with the observed HSI, which limits the super-resolution reconstruction quality. In this paper, a new framework based on domain transfer learning for HSI super-resolution is proposed to enhance the spatial resolution of HSI by learning the knowledge from the general purpose optical images (natural scene images) and exploiting the cross-correlation between the observed low-resolution HSI and high-resolution MSI. First, the relationship between low- and high-resolution images is learned by a single convolutional super-resolution network and then is transferred to HSI by the idea of transfer learning. Second, the obtained Pre-high-resolution HSI (pre-HSI), the observed low-resolution HSI, and high-resolution MSI are simultaneously considered to estimate the endmember matrix and the abundance code for learning the spectral characteristic. Experimental results on ground-based and remote sensing datasets demonstrate that the proposed method achieves comparable performance and outperforms the existing HSI super-resolution methods.

scholarly journals Hyperspectral and Multispectral Remote Sensing Image Fusion Based on Endmember Spatial Information

2020 ◽  
Vol 12 (6) ◽  
pp. 1009
Author(s):  
Xiaoxiao Feng ◽  
Luxiao He ◽  
Qimin Cheng ◽  
Xiaoyi Long ◽  
Yuxin Yuan
Keyword(s):  
Image Fusion ◽  
Spatial Resolution ◽  
Spectral Resolution ◽  
Spatial Information ◽  
Spectral Unmixing ◽  
High Spectral Resolution ◽  
Remote Sensing Image Fusion ◽  
Fusion Methods ◽  
Invariant Regions ◽  
The Difference

Hyperspectral (HS) images usually have high spectral resolution and low spatial resolution (LSR). However, multispectral (MS) images have high spatial resolution (HSR) and low spectral resolution. HS–MS image fusion technology can combine both advantages, which is beneficial for accurate feature classification. Nevertheless, heterogeneous sensors always have temporal differences between LSR-HS and HSR-MS images in the real cases, which means that the classical fusion methods cannot get effective results. For this problem, we present a fusion method via spectral unmixing and image mask. Considering the difference between the two images, we firstly extracted the endmembers and their corresponding positions from the invariant regions of LSR-HS images. Then we can get the endmembers of HSR-MS images based on the theory that HSR-MS images and LSR-HS images are the spectral and spatial degradation from HSR-HS images, respectively. The fusion image is obtained by two result matrices. Series experimental results on simulated and real datasets substantiated the effectiveness of our method both quantitatively and visually.

scholarly journals A Robust Structured Tracker Using Local Deep Features

Electronics ◽  
2020 ◽  
Vol 9 (5) ◽  
pp. 846 ◽  
Author(s):  
Mohammadreza Javanmardi ◽  
Amir Hossein Farzaneh ◽  
Xiaojun Qi
Keyword(s):  
Optimization Model ◽  
Spatial Information ◽  
State Of The Art ◽  
Semantic Representation ◽  
Spatial Layout ◽  
Image Sequences ◽  
Superior Performance ◽  
Group Sparsity ◽  
Layout Structure ◽  
Close Form

Deep features extracted from convolutional neural networks have been recently utilized in visual tracking to obtain a generic and semantic representation of target candidates. In this paper, we propose a robust structured tracker using local deep features (STLDF). This tracker exploits the deep features of local patches inside target candidates and sparsely represents them by a set of templates in the particle filter framework. The proposed STLDF utilizes a new optimization model, which employs a group-sparsity regularization term to adopt local and spatial information of the target candidates and attain the spatial layout structure among them. To solve the optimization model, we propose an efficient and fast numerical algorithm that consists of two subproblems with the close-form solutions. Different evaluations in terms of success and precision on the benchmarks of challenging image sequences (e.g., OTB50 and OTB100) demonstrate the superior performance of the STLDF against several state-of-the-art trackers.

scholarly journals Sparse Unmixing for Hyperspectral Image with Nonlocal Low-Rank Prior

2019 ◽  
Vol 11 (24) ◽  
pp. 2897 ◽  
Author(s):  
Yuhui Zheng ◽  
Feiyang Wu ◽  
Hiuk Jae Shim ◽  
Le Sun
Keyword(s):  
Mixed Model ◽  
Linear Mixed Model ◽  
Spatial Information ◽  
Hyperspectral Image ◽  
Simulated Data ◽  
Hyperspectral Data ◽  
Low Rank ◽  
Data Sets ◽  
Self Similarity ◽  
Model Framework

Hyperspectral unmixing is a key preprocessing technique for hyperspectral image analysis. To further improve the unmixing performance, in this paper, a nonlocal low-rank prior associated with spatial smoothness and spectral collaborative sparsity are integrated together for unmixing the hyperspectral data. The proposed method is based on a fact that hyperspectral images have self-similarity in nonlocal sense and smoothness in local sense. To explore the spatial self-similarity, nonlocal cubic patches are grouped together to compose a low-rank matrix. Then, based on the linear mixed model framework, the nuclear norm is constrained to the abundance matrix of these similar patches to enforce low-rank property. In addition, the local spatial information and spectral characteristic are also taken into account by introducing TV regularization and collaborative sparse terms, respectively. Finally, the results of the experiments on two simulated data sets and two real data sets show that the proposed algorithm produces better performance than other state-of-the-art algorithms.

scholarly journals Hyperspectral Image Super-Resolution with 1D–2D Attentional Convolutional Neural Network

2019 ◽  
Vol 11 (23) ◽  
pp. 2859 ◽  
Author(s):  
Jiaojiao Li ◽  
Ruxing Cui ◽  
Bo Li ◽  
Rui Song ◽  
Yunsong Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Spatial Information ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
High Resolution Images ◽  
Image Super Resolution ◽  
Small Change ◽  
Single Image Super Resolution ◽  
Separation Strategy

Hyperspectral image (HSI) super-resolution (SR) is of great application value and has attracted broad attention. The hyperspectral single image super-resolution (HSISR) task is correspondingly difficult in SR due to the unavailability of auxiliary high resolution images. To tackle this challenging task, different from the existing learning-based HSISR algorithms, in this paper we propose a novel framework, i.e., a 1D–2D attentional convolutional neural network, which employs a separation strategy to extract the spatial–spectral information and then fuse them gradually. More specifically, our network consists of two streams: a spatial one and a spectral one. The spectral one is mainly composed of the 1D convolution to encode a small change in the spectrum, while the 2D convolution, cooperating with the attention mechanism, is used in the spatial pathway to encode spatial information. Furthermore, a novel hierarchical side connection strategy is proposed for effectively fusing spectral and spatial information. Compared with the typical 3D convolutional neural network (CNN), the 1D–2D CNN is easier to train with less parameters. More importantly, our proposed framework can not only present a perfect solution for the HSISR problem, but also explore the potential in hyperspectral pansharpening. The experiments over widely used benchmarks on SISR and hyperspectral pansharpening demonstrate that the proposed method could outperform other state-of-the-art methods, both in visual quality and quantity measurements.

Sign in / Sign up

Export Citation Format

Share Document