RSCNN: A CNN-Based Method to Enhance Low-Light Remote-Sensing Images

Image enhancement (IE) technology can help enhance the brightness of remote-sensing images to obtain better interpretation and visualization effects. Convolutional neural networks (CNN), such as the Low-light CNN (LLCNN) and Super-resolution CNN (SRCNN), have achieved great success in image enhancement, image super resolution, and other image-processing applications. Therefore, we adopt CNN to propose a new neural network architecture with end-to-end strategy for low-light remote-sensing IE, named remote-sensing CNN (RSCNN). In RSCNN, an upsampling operator is adopted to help learn more multi-scaled features. With respect to the lack of labeled training data in remote-sensing image datasets for IE, we use real natural image patches to train firstly and then perform fine-tuning operations with simulated remote-sensing image pairs. Reasonably designed experiments are carried out, and the results quantitatively show the superiority of RSCNN in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) over conventional techniques for low-light remote-sensing IE. Furthermore, the results of our method have obvious qualitative advantages in denoising and maintaining the authenticity of colors and textures.

Download Full-text

Hybrid Collaborative Representation for Remote-Sensing Image Scene Classification

Remote Sensing ◽

10.3390/rs10121934 ◽

2018 ◽

Vol 10 (12) ◽

pp. 1934 ◽

Cited By ~ 11

Author(s):

Bao-Di Liu ◽

Wen-Yang Xie ◽

Jie Meng ◽

Ye Li ◽

Yanjiang Wang

Keyword(s):

Remote Sensing ◽

Test Sample ◽

Remote Sensing Image ◽

Image Features ◽

Superior Performance ◽

Collaborative Representation ◽

Great Success ◽

Specific Class ◽

Remote Sensing Images ◽

Training Samples

In recent years, the collaborative representation-based classification (CRC) method has achieved great success in visual recognition by directly utilizing training images as dictionary bases. However, it describes a test sample with all training samples to extract shared attributes and does not consider the representation of the test sample with the training samples in a specific class to extract the class-specific attributes. For remote-sensing images, both the shared attributes and class-specific attributes are important for classification. In this paper, we propose a hybrid collaborative representation-based classification approach. The proposed method is capable of improving the performance of classifying remote-sensing images by embedding the class-specific collaborative representation to conventional collaborative representation-based classification. Moreover, we extend the proposed method to arbitrary kernel space to explore the nonlinear characteristics hidden in remote-sensing image features to further enhance classification performance. Extensive experiments on several benchmark remote-sensing image datasets were conducted and clearly demonstrate the superior performance of our proposed algorithm to state-of-the-art approaches.

Download Full-text

Superpixel-Guided Layer-Wise Embedding CNN for Remote Sensing Image Classification

Remote Sensing ◽

10.3390/rs11020174 ◽

2019 ◽

Vol 11 (2) ◽

pp. 174 ◽

Cited By ~ 4

Author(s):

Han Liu ◽

Jun Li ◽

Lin He ◽

Yu Wang

Keyword(s):

Remote Sensing ◽

Image Classification ◽

Remote Sensing Data ◽

Sampling Strategy ◽

Remote Sensing Image ◽

Fine Tuning ◽

Spatial Dependency ◽

Remote Sensing Images ◽

Training Set ◽

Remote Sensing Image Classification

Irregular spatial dependency is one of the major characteristics of remote sensing images, which brings about challenges for classification tasks. Deep supervised models such as convolutional neural networks (CNNs) have shown great capacity for remote sensing image classification. However, they generally require a huge labeled training set for the fine tuning of a deep neural network. To handle the irregular spatial dependency of remote sensing images and mitigate the conflict between limited labeled samples and training demand, we design a superpixel-guided layer-wise embedding CNN (SLE-CNN) for remote sensing image classification, which can efficiently exploit the information from both labeled and unlabeled samples. With the superpixel-guided sampling strategy for unlabeled samples, we can achieve an automatic determination of the neighborhood covering for a spatial dependency system and thus adapting to real scenes of remote sensing images. In the designed network, two types of loss costs are combined for the training of CNN, i.e., supervised cross entropy and unsupervised reconstruction cost on both labeled and unlabeled samples, respectively. Our experimental results are conducted with three types of remote sensing data, including hyperspectral, multispectral, and synthetic aperture radar (SAR) images. The designed SLE-CNN achieves excellent classification performance in all cases with a limited labeled training set, suggesting its good potential for remote sensing image classification.

Download Full-text

Transferred Multi-Perception Attention Networks for Remote Sensing Image Super-Resolution

Remote Sensing ◽

10.3390/rs11232857 ◽

2019 ◽

Vol 11 (23) ◽

pp. 2857 ◽

Cited By ~ 3

Author(s):

Xiaoyu Dong ◽

Zhihong Xi ◽

Xu Sun ◽

Lianru Gao

Keyword(s):

Remote Sensing ◽

Learning Strategy ◽

State Of The Art ◽

Spatial Scales ◽

Super Resolution ◽

Natural Image ◽

Training Procedure ◽

Remote Sensing Images ◽

Objective Criterion ◽

Image Super Resolution

Image super-resolution (SR) reconstruction plays a key role in coping with the increasing demand on remote sensing imaging applications with high spatial resolution requirements. Though many SR methods have been proposed over the last few years, further research is needed to improve SR processes with regard to the complex spatial distribution of the remote sensing images and the diverse spatial scales of ground objects. In this paper, a novel multi-perception attention network (MPSR) is developed with performance exceeding those of many existing state-of-the-art models. By incorporating the proposed enhanced residual block (ERB) and residual channel attention group (RCAG), MPSR can super-resolve low-resolution remote sensing images via multi-perception learning and multi-level information adaptive weighted fusion. Moreover, a pre-train and transfer learning strategy is introduced, which improved the SR performance and stabilized the training procedure. Experimental comparisons are conducted using 13 state-of-the-art methods over a remote sensing dataset and benchmark natural image sets. The proposed model proved its excellence in both objective criterion and subjective perspective.

Download Full-text

When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on a Multitask Learning Framework

Remote Sensing ◽

10.3390/rs12203276 ◽

2020 ◽

Vol 12 (20) ◽

pp. 3276 ◽

Cited By ~ 2

Author(s):

Zhicheng Zhao ◽

Ze Luo ◽

Jian Li ◽

Can Chen ◽

Yingchao Piao

Keyword(s):

Remote Sensing ◽

Supervised Learning ◽

Remote Sensing Image ◽

Multitask Learning ◽

Natural Image ◽

Scene Classification ◽

Remote Sensing Images ◽

Learning Framework ◽

Continuous Progress ◽

Image Datasets

In recent years, the development of convolutional neural networks (CNNs) has promoted continuous progress in scene classification of remote sensing images. Compared with natural image datasets, however, the acquisition of remote sensing scene images is more difficult, and consequently the scale of remote sensing image datasets is generally small. In addition, many problems related to small objects and complex backgrounds arise in remote sensing image scenes, presenting great challenges for CNN-based recognition methods. In this article, to improve the feature extraction ability and generalization ability of such models and to enable better use of the information contained in the original remote sensing images, we introduce a multitask learning framework which combines the tasks of self-supervised learning and scene classification. Unlike previous multitask methods, we adopt a new mixup loss strategy to combine the two tasks with dynamic weight. The proposed multitask learning framework empowers a deep neural network to learn more discriminative features without increasing the amounts of parameters. Comprehensive experiments were conducted on four representative remote sensing scene classification datasets. We achieved state-of-the-art performance, with average accuracies of 94.21%, 96.89%, 99.11%, and 98.98% on the NWPU, AID, UC Merced, and WHU-RS19 datasets, respectively. The experimental results and visualizations show that our proposed method can learn more discriminative features and simultaneously encode orientation information while effectively improving the accuracy of remote sensing scene classification.

Download Full-text

Deep Residual Squeeze and Excitation Network for Remote Sensing Image Super-Resolution

Remote Sensing ◽

10.3390/rs11151817 ◽

2019 ◽

Vol 11 (15) ◽

pp. 1817 ◽

Cited By ~ 8

Author(s):

Jun Gu ◽

Xian Sun ◽

Yue Zhang ◽

Kun Fu ◽

Lei Wang

Keyword(s):

Remote Sensing ◽

State Of The Art ◽

Super Resolution ◽

Remote Sensing Image ◽

Single Image ◽

Remote Sensing Images ◽

Deep Convolutional Neural Networks ◽

Current Block ◽

Image Super Resolution ◽

Single Image Super Resolution

Recently, deep convolutional neural networks (DCNN) have obtained promising results in single image super-resolution (SISR) of remote sensing images. Due to the high complexity of remote sensing image distribution, most of the existing methods are not good enough for remote sensing image super-resolution. Enhancing the representation ability of the network is one of the critical factors to improve remote sensing image super-resolution performance. To address this problem, we propose a new SISR algorithm called a Deep Residual Squeeze and Excitation Network (DRSEN). Specifically, we propose a residual squeeze and excitation block (RSEB) as a building block in DRSEN. The RSEB fuses the input and its internal features of current block, and models the interdependencies and relationships between channels to enhance the representation power. At the same time, we improve the up-sampling module and the global residual pathway in the network to reduce the parameters of the network. Experiments on two public remote sensing datasets (UC Merced and NWPU-RESISC45) show that our DRSEN achieves better accuracy and visual improvements against most state-of-the-art methods. The DRSEN is beneficial for the progress in the remote sensing images super-resolution field.

Download Full-text

Deep Memory Connected Neural Network for Optical Remote Sensing Image Restoration

Remote Sensing ◽

10.3390/rs10121893 ◽

2018 ◽

Vol 10 (12) ◽

pp. 1893 ◽

Cited By ~ 6

Author(s):

Wenjia Xu ◽

Guangluan Xu ◽

Yang Wang ◽

Xian Sun ◽

Daoyu Lin ◽

...

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Image Restoration ◽

Spatial Resolution ◽

Remote Sensing Image ◽

Global Memory ◽

Great Success ◽

Optical Remote Sensing ◽

Remote Sensing Images ◽

Feature Maps

The spatial resolution and clarity of remote sensing images are crucial for many applications such as target detection and image classification. In the last several decades, tremendous image restoration tasks have shown great success in ordinary images. However, since remote sensing images are more complex and more blurry than ordinary images, most of the existing methods are not good enough for remote sensing image restoration. To address such problem, we propose a novel method named deep memory connected network (DMCN) based on the convolutional neural network to reconstruct high-quality images. We build local and global memory connections to combine image detail with global information. To further reduce parameters and ease time consumption, we propose Downsampling Units, shrinking the spatial size of feature maps. We verify its capability on two representative applications, Gaussian image denoising and single image super-resolution (SR). DMCN is tested on three remote sensing datasets with various spatial resolution. Experimental results indicate that our method yields promising improvements and better visual performance over the current state-of-the-art. The PSNR and SSIM improvements over the second best method are up to 0.3 dB.

Download Full-text

Target Detection Method for Low-Resolution Remote Sensing Image Based on ESRGAN and ReDet

Photonics ◽

10.3390/photonics8100431 ◽

2021 ◽

Vol 8 (10) ◽

pp. 431

Author(s):

Yuwu Wang ◽

Guobing Sun ◽

Shengwei Guo

Keyword(s):

Remote Sensing ◽

Target Detection ◽

Detection Method ◽

Recognition Rate ◽

Super Resolution ◽

Remote Sensing Image ◽

Remote Sensing Images ◽

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network

With the widespread use of remote sensing images, low-resolution target detection in remote sensing images has become a hot research topic in the field of computer vision. In this paper, we propose a Target Detection on Super-Resolution Reconstruction (TDoSR) method to solve the problem of low target recognition rates in low-resolution remote sensing images under foggy conditions. The TDoSR method uses the Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) to perform defogging and super-resolution reconstruction of foggy low-resolution remote sensing images. In the target detection part, the Rotation Equivariant Detector (ReDet) algorithm, which has a higher recognition rate at this stage, is used to identify and classify various types of targets. While a large number of experiments have been carried out on the remote sensing image dataset DOTA-v1.5, the results of this paper suggest that the proposed method achieves good results in the target detection of low-resolution foggy remote sensing images. The principal result of this paper demonstrates that the recognition rate of the TDoSR method increases by roughly 20% when compared with low-resolution foggy remote sensing images.

Download Full-text

Saliency-Guided Remote Sensing Image Super-Resolution

Remote Sensing ◽

10.3390/rs13245144 ◽

2021 ◽

Vol 13 (24) ◽

pp. 5144

Author(s):

Baodi Liu ◽

Lifei Zhao ◽

Jiaoyue Li ◽

Hengle Zhao ◽

Weifeng Liu ◽

...

Keyword(s):

Remote Sensing ◽

Saliency Detection ◽

Super Resolution ◽

Remote Sensing Image ◽

Complex Structures ◽

Remote Sensing Images ◽

Saliency Maps ◽

Image Super Resolution ◽

The One ◽

Visual Results

Deep learning has recently attracted extensive attention and developed significantly in remote sensing image super-resolution. Although remote sensing images are composed of various scenes, most existing methods consider each part equally. These methods ignore the salient objects (e.g., buildings, airplanes, and vehicles) that have more complex structures and require more attention in recovery processing. This paper proposes a saliency-guided remote sensing image super-resolution (SG-GAN) method to alleviate the above issue while maintaining the merits of GAN-based methods for the generation of perceptual-pleasant details. More specifically, we exploit the salient maps of images to guide the recovery in two aspects: On the one hand, the saliency detection network in SG-GAN learns more high-resolution saliency maps to provide additional structure priors. On the other hand, the well-designed saliency loss imposes a second-order restriction on the super-resolution process, which helps SG-GAN concentrate more on the salient objects of remote sensing images. Experimental results show that SG-GAN achieves competitive PSNR and SSIM compared with the advanced super-resolution methods. Visual results demonstrate our superiority in restoring structures while generating remote sensing super-resolution images.

Download Full-text

Distribution Consistency Loss for Large-Scale Remote Sensing Image Retrieval

Remote Sensing ◽

10.3390/rs12010175 ◽

2020 ◽

Vol 12 (1) ◽

pp. 175 ◽

Cited By ~ 3

Author(s):

Lili Fan ◽

Hongwei Zhao ◽

Haoyu Zhao

Keyword(s):

Remote Sensing ◽

Image Retrieval ◽

Large Scale ◽

Metric Learning ◽

Remote Sensing Image ◽

Positive Sample ◽

Fine Tuning ◽

Remote Sensing Images ◽

Hard Samples ◽

Image Datasets

Remote sensing images are featured by massiveness, diversity and complexity. These features put forward higher requirements for the speed and accuracy of remote sensing image retrieval. The extraction method plays a key role in retrieving remote sensing images. Deep metric learning (DML) captures the semantic similarity information between data points by learning embedding in vector space. However, due to the uneven distribution of sample data in remote sensing image datasets, the pair-based loss currently used in DML is not suitable. To improve this, we propose a novel distribution consistency loss to solve this problem. First, we define a new way to mine samples by selecting five in-class hard samples and five inter-class hard samples to form an informative set. This method can make the network extract more useful information in a short time. Secondly, in order to avoid inaccurate feature extraction due to sample imbalance, we assign dynamic weight to the positive samples according to the ratio of the number of hard samples and easy samples in the class, and name the loss caused by the positive sample as the sample balance loss. We combine the sample balance of the positive samples with the ranking consistency of the negative samples to form our distribution consistency loss. Finally, we built an end-to-end fine-tuning network suitable for remote sensing image retrieval. We display comprehensive experimental results drawing on three remote sensing image datasets that are publicly available and show that our method achieves the state-of-the-art performance.

Download Full-text

Single Remote Sensing Image Super-Resolution with an Adaptive Joint Constraint Model

Sensors ◽

10.3390/s20051276 ◽

2020 ◽

Vol 20 (5) ◽

pp. 1276

Author(s):

Lingli Fu ◽

Chao Ren ◽

Xiaohai He ◽

Xiaohong Wu ◽

Zhengyong Wang

Keyword(s):

Remote Sensing ◽

Sparse Representation ◽

Super Resolution ◽

Objective Evaluation ◽

Remote Sensing Image ◽

Iteration Algorithm ◽

Remote Sensing Images ◽

Constraint Model ◽

Local Gradient ◽

Sr Method

Remote sensing images have been widely used in many applications. However, the resolution of the obtained remote sensing images may not meet the increasing demands for some applications. In general, the sparse representation-based super-resolution (SR) method is one of the most popular methods to solve this issue. However, traditional sparse representation SR methods do not fully exploit the complementary constraints of images. Therefore, they cannot accurately reconstruct the unknown HR images. To address this issue, we propose a novel adaptive joint constraint (AJC) based on sparse representation for the single remote sensing image SR. First, we construct a nonlocal constraint by using the nonlocal self-similarity. Second, we propose a local structure filter according to the local gradient of the image and then construct a local constraint. Next, the nonlocal and local constraints are introduced into the sparse representation-based SR framework. Finally, the parameters of the joint constraint model are selected adaptively according to the level of image noise. We utilize the alternate iteration algorithm to tackle the minimization problem in AJC. Experimental results show that the proposed method achieves good SR performance in preserving image details and significantly improves the objective evaluation indices.

Download Full-text