Hyperspectral Image Super-Resolution Based on Spatial Correlation-Regularized Unmixing Convolutional Neural Network

Super-resolution (SR) technology has emerged as an effective tool for image analysis and interpretation. However, single hyperspectral (HS) image SR remains challenging, due to the high spectral dimensionality and lack of available high-resolution information of auxiliary sources. To fully exploit the spectral and spatial characteristics, in this paper, a novel single HS image SR approach is proposed based on a spatial correlation-regularized unmixing convolutional neural network (CNN). The proposed approach takes advantage of a CNN to explore the collaborative spatial and spectral information of an HS image and infer the high-resolution abundance maps, thereby reconstructing the anticipated high-resolution HS image via the linear spectral mixture model. Moreover, a dual-branch architecture network and spatial spread transform function are employed to characterize the spatial correlation between the high- and low-resolution HS images, aiming at promoting the fidelity of the super-resolved image. Experiments on three public remote sensing HS images demonstrate the feasibility and superiority in terms of spectral fidelity, compared with some state-of-the-art HS image super-resolution methods.

Fusion of China ZY-1 02D Hyperspectral Data and Multispectral Data: Which Methods Should Be Used?

ZY-1 02D is China’s first civil hyperspectral (HS) operational satellite, developed independently and successfully launched in 2019. It can collect HS data with a spatial resolution of 30 m, 166 spectral bands, a spectral range of 400~2500 nm, and a swath width of 60 km. Its competitive advantages over other on-orbit or planned satellites are its high spectral resolution and large swath width. Unfortunately, the relatively low spatial resolution may limit its applications. As a result, fusing ZY-1 02D HS data with high-spatial-resolution multispectral (MS) data is required to improve spatial resolution while maintaining spectral fidelity. This paper conducted a comprehensive evaluation study on the fusion of ZY-1 02D HS data with ZY-1 02D MS data (10-m spatial resolution), based on visual interpretation and quantitative metrics. Datasets from Hebei, China, were used in this experiment, and the performances of six common data fusion methods, namely Gram-Schmidt (GS), High Pass Filter (HPF), Nearest-Neighbor Diffusion (NND), Modified Intensity-Hue-Saturation (IHS), Wavelet Transform (Wavelet), and Color Normalized Sharping (Brovey), were compared. The experimental results show that: (1) HPF and GS methods are better suited for the fusion of ZY-1 02D HS Data and MS Data, (2) IHS and Brovey methods can well improve the spatial resolution of ZY-1 02D HS data but introduce spectral distortion, and (3) Wavelet and NND results have high spectral fidelity but poor spatial detail representation. The findings of this study could serve as a good reference for the practical application of ZY-1 02D HS data fusion.

Bond-selective imaging by optically sensing the mid-infrared photothermal effect

Mid-infrared (IR) spectroscopic imaging using inherent vibrational contrast has been broadly used as a powerful analytical tool for sample identification and characterization. However, the low spatial resolution and large water absorption associated with the long IR wavelengths hinder its applications to study subcellular features in living systems. Recently developed mid-infrared photothermal (MIP) microscopy overcomes these limitations by probing the IR absorption–induced photothermal effect using a visible light. MIP microscopy yields submicrometer spatial resolution with high spectral fidelity and reduced water background. In this review, we categorize different photothermal contrast mechanisms and discuss instrumentations for scanning and widefield MIP microscope configurations. We highlight a broad range of applications from life science to materials. We further provide future perspective and potential venues in MIP microscopy field.

MicroPulse DIAL (MPD) – a Diode-Laser-Based Lidar Architecture for Quantitative Atmospheric Profiling

Continuous water vapor and temperature profiles are critically needed for improved understanding of the lower atmosphere and potential advances in weather forecasting skill. Ground-based, national-scale profiling networks are part of a suite of instruments to provide such observations; however, the technological method must be cost-effective and quantitative. We have been developing an active remote sensing technology based on a diode-laser-based lidar architecture to address this observational need. Narrowband, high spectral fidelity diode lasers enable accurate and calibration-free measurements requiring a minimal set of assumptions based on direct absorption (Beer-Lambert law) and a ratio of two signals. These well-proven quantitative methods are known as differential absorption lidar (DIAL) and the high spectral resolution lidar (HSRL). This diode-laser-based architecture, characterized by less powerful laser transmitters than those historically used for atmospheric studies, can be made eye-safe and robust. Nevertheless, it also requires solar background suppression techniques such as narrow field-of-view receivers with an ultra-narrow bandpass to observe individual photons backscattered from the atmosphere. We will discuss this diode-laser-based lidar architecture's latest generation and analyze how it addresses a national-scale profiling network's need to provide continuous thermodynamic observations.

Assessment of Suitable Image Fusion Method for CARTOSAT-2E Satellite Urban Imagery for Automatic Feature Extraction

The purpose of this study is to investigate the best suitable pan sharpening method for CARTOSAT-2E satellite launched by ISRO (Indian Space Research Organisation). This satellite provides high resolution images that are being used for many urban applications such as mapping, feature extraction, facility management etc. The synthesized image using pan sharpening method enables users to take the combined advantage of the best available spatial and spectral resolutions. In this paper, various pan sharpening methods based on component substitution (CS) and Multi Resolution Analysis (MRA) are applied on the CARTOSAT-2E images and the resultant images are tested for their qualitative and quantitative performance. Qualitative analysis is carried out based on image blur and spectral distortion and quantitative evaluation is performed using image metrics by comparing the synthesized image with the original image. The results show that the High-Pass Filter (HPF) method offers the good spectral fidelity. However, due to its inherent stair-casing effect in the resultant image; modified-IHS followed by PRACS method is found to be preferable for automatic urban feature extraction from CARTOSAT-2E images.

Coupled Convolutional Neural Network-Based Detail Injection Method for Hyperspectral and Multispectral Image Fusion

In this paper, a detail-injection method based on a coupled convolutional neural network (CNN) is proposed for hyperspectral (HS) and multispectral (MS) image fusion with the goal of enhancing the spatial resolution of HS images. Owing to the excellent performance in spectral fidelity of the detail-injection model and the image spatial–spectral feature exploration ability of CNN, the proposed method utilizes a couple of CNN networks as the feature extraction method and learns details from the HS and MS images individually. By appending an additional convolutional layer, both the extracted features of two images are concatenated to predict the missing details of the anticipated HS image. Experiments on simulated and real HS and MS data show that compared with some state-of-the-art HS and MS image fusion methods, our proposed method achieves better fusion results, provides excellent spectrum preservation ability, and is easy to implement.

Small Multispectral UAV Sensor and Its Image Fusion Capability in Cultural Heritage Applications

For many decades the multispectral images of the earth’s surface and its objects were taken from multispectral sensors placed on satellites. In recent years, the technological evolution produced similar sensors (much smaller in size and weight) which can be placed on Unmanned Aerial Vehicles (UAVs), thereby allowing the collection of higher spatial resolution multispectral images. In this paper, Parrot’s small Multispectral (MS) camera Sequoia+ is used, and its images are evaluated at two archaeological sites, on the Byzantine wall (ground application) of Thessaloniki city (Greece) and on a mosaic floor (aerial application) at the archaeological site of Dion (Greece). The camera receives RGB and MS images simultaneously, a fact which does not allow image fusion to be performed, as in the standard utilization procedure of Panchromatic (PAN) and MS image of satellite passive systems. In this direction, that is, utilizing the image fusion processes of satellite PAN and MS images, this paper demonstrates that with proper digital processing the images (RGB and MS) of small MS cameras can lead to a fused image with a high spatial resolution, which retains a large percentage of the spectral information of the original MS image. The high percentage of spectral fidelity of the fused images makes it possible to perform high-precision digital measurements in archaeological sites such as the accurate digital separation of the objects, area measurements and retrieval of information not so visible with common RGB sensors via the MS and RGB data of small MS sensors.

An Improved Pulse-Coupled Neural Network Model for Pansharpening

Pulse-coupled neural network (PCNN) and its modified models are suitable for dealing with multi-focus and medical image fusion tasks. Unfortunately, PCNNs are difficult to directly apply to multispectral image fusion, especially when the spectral fidelity is considered. A key problem is that most fusion methods using PCNNs usually focus on the selection mechanism either in the space domain or in the transform domain, rather than a details injection mechanism, which is of utmost importance in multispectral image fusion. Thus, a novel pansharpening PCNN model for multispectral image fusion is proposed. The new model is designed to acquire the spectral fidelity in terms of human visual perception for the fusion tasks. The experimental results, examined by different kinds of datasets, show the suitability of the proposed model for pansharpening.