scholarly journals A GENERATIVE ADVERSARIAL NETWORK APPROACH FOR SUPER-RESOLUTION OF SENTINEL-2 SATELLITE IMAGES

2020 ◽  
Vol XLIII-B1-2020 ◽  
pp. 9-14
Author(s):  
F. Pineda ◽  
V. Ayma ◽  
C. Beltran
Keyword(s):  
High Resolution ◽  
Satellite Images ◽  
Resolution Enhancement ◽  
Super Resolution ◽  
Structural Similarity ◽  
Generative Adversarial Network ◽  
Acceptable Result ◽  
Adversarial Network ◽  
High Resolution Satellite Images ◽  
Sentinel 2

Abstract. High-resolution satellite images have always been in high demand due to the greater detail and precision they offer, as well as the wide scope of the fields in which they could be applied; however, satellites in operation offering very high-resolution (VHR) images has experienced an important increase, but they remain as a smaller proportion against existing lower resolution (HR) satellites. Recent models of convolutional neural networks (CNN) are very suitable for applications with image processing, like resolution enhancement of images; but in order to obtain an acceptable result, it is important, not only to define the kind of CNN architecture but the reference set of images to train the model. Our work proposes an alternative to improve the spatial resolution of HR images obtained by Sentinel-2 satellite by using the VHR images from PeruSat1, a peruvian satellite, which serve as the reference for the super-resolution approach implementation based on a Generative Adversarial Network (GAN) model, as an alternative for obtaining VHR images. The VHR PeruSat-1 image dataset is used for the training process of the network. The results obtained were analyzed considering the Peak Signal to Noise Ratios (PSNR) and the Structural Similarity (SSIM). Finally, some visual outcomes, over a given testing dataset, are presented so the performance of the model could be analyzed as well.

scholarly journals Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2164
Author(s):  
Md. Shahinur Alam ◽  
Ki-Chul Kwon ◽  
Munkh-Uchral Erdenebat ◽  
Mohammed Y. Abbass ◽  
Md. Ashraful Alam ◽  
...  
Keyword(s):  
High Resolution ◽  
Resolution Enhancement ◽  
Three Dimensional ◽  
Super Resolution ◽  
Input Image ◽  
Low Resolution ◽  
Generative Adversarial Network ◽  
Microscopic Object ◽  
Integral Imaging ◽  
Adversarial Network

The integral imaging microscopy system provides a three-dimensional visualization of a microscopic object. However, it has a low-resolution problem due to the fundamental limitation of the F-number (the aperture stops) by using micro lens array (MLA) and a poor illumination environment. In this paper, a generative adversarial network (GAN)-based super-resolution algorithm is proposed to enhance the resolution where the directional view image is directly fed as input. In a GAN network, the generator regresses the high-resolution output from the low-resolution input image, whereas the discriminator distinguishes between the original and generated image. In the generator part, we use consecutive residual blocks with the content loss to retrieve the photo-realistic original image. It can restore the edges and enhance the resolution by ×2, ×4, and even ×8 times without seriously hampering the image quality. The model is tested with a variety of low-resolution microscopic sample images and successfully generates high-resolution directional view images with better illumination. The quantitative analysis shows that the proposed model performs better for microscopic images than the existing algorithms.

scholarly journals Towards Streamlined Single-Image Super-Resolution: Demonstration with 10 m Sentinel-2 Colour and 10–60 m Multi-Spectral VNIR and SWIR Bands

2021 ◽  
Vol 13 (13) ◽  
pp. 2614
Author(s):  
Yu Tao ◽  
Siting Xiong ◽  
Rui Song ◽  
Jan-Peter Muller
Keyword(s):  
Resolution Enhancement ◽  
Super Resolution ◽  
Urban Landscapes ◽  
Imaging Data ◽  
Generative Adversarial Network ◽  
Line Spread Function ◽  
Adversarial Network ◽  
Spread Function ◽  
Ice Floes ◽  
Sentinel 2

Higher spatial resolution imaging data are considered desirable in many Earth observation applications. In this work, we propose and demonstrate the TARSGAN (learning Terrestrial image deblurring using Adaptive weighted dense Residual Super-resolution Generative Adversarial Network) system for Super-resolution Restoration (SRR) of 10 m/pixel Sentinel-2 “true” colour images as well as all the other multispectral bands. In parallel, the ELF (automated image Edge detection and measurements of edge spread function, Line spread function, and Full width at half maximum) system is proposed to achieve automated and precise assessments of the effective resolutions of the input and SRR images. Subsequent ELF measurements of the TARSGAN SRR results suggest an averaged effective resolution enhancement factor of about 2.91 times (equivalent to ~3.44 m/pixel for the 10 m/pixel bands) given a nominal SRR upscaling factor of 4 times. Several examples are provided for different types of scenes from urban landscapes to agricultural scenes and sea-ice floes.

scholarly journals Single Image Super-Resolution: Depthwise Separable Convolution Super-Resolution Generative Adversarial Network

2020 ◽  
Vol 10 (1) ◽  
pp. 375 ◽  
Author(s):  
Zetao Jiang ◽  
Yongsong Huang ◽  
Lirui Hu
Keyword(s):  
Similarity Index ◽  
Super Resolution ◽  
Structural Similarity ◽  
Image Features ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Visual Evaluation ◽  
Adversarial Network ◽  
Image Super Resolution ◽  
Single Image Super Resolution

The super-resolution generative adversarial network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied by unpleasant artifacts. To further enhance the visual quality, we propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The method is based on depthwise separable convolution super-resolution generative adversarial network (DSCSRGAN). A new depthwise separable convolution dense block (DSC Dense Block) was designed for the generator network, which improved the ability to represent and extract image features, while greatly reducing the total amount of parameters. For the discriminator network, the batch normalization (BN) layer was discarded, and the problem of artifacts was reduced. A frequency energy similarity loss function was designed to constrain the generator network to generate better super-resolution images. Experiments on several different datasets showed that the peak signal-to-noise ratio (PSNR) was improved by more than 3 dB, structural similarity index (SSIM) was increased by 16%, and the total parameter was reduced to 42.8% compared with the original model. Combining various objective indicators and subjective visual evaluation, the algorithm was shown to generate richer image details, clearer texture, and lower complexity.

scholarly journals Fast and accurate learned multiresolution dynamical downscaling for precipitation

2021 ◽  
Author(s):  
Jiali Wang ◽  
Zhengchun Liu ◽  
Ian Foster ◽  
Won Chang ◽  
Rajkumar Kettimuthu ◽  
...  
Keyword(s):  
Neural Network ◽  
High Resolution ◽  
Dynamical Downscaling ◽  
Computational Cost ◽  
Super Resolution ◽  
Ground Truth ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Spatial And Temporal Distributions ◽  
Data Variability

Abstract. This study develops a neural network-based approach for emulating high-resolution modeled precipitation data with comparable statistical properties but at greatly reduced computational cost. The key idea is to use combination of low- and high- resolution simulations to train a neural network to map from the former to the latter. Specifically, we define two types of CNNs, one that stacks variables directly and one that encodes each variable before stacking, and we train each CNN type both with a conventional loss function, such as mean square error (MSE), and with a conditional generative adversarial network (CGAN), for a total of four CNN variants.We compare the four new CNN-derived high-resolution precipitation results with precipitation generated from original high resolution simulations, a bilinear interpolater and the state-of-the-art CNN-based super-resolution (SR) technique. Results show that the SR technique produces results similar to those of the bilinear interpolator with smoother spatial and temporal distributions and smaller data variabilities and extremes than the high resolution simulations. While the new CNNs trained by MSE generate better results over some regions than the interpolator and SR technique do, their predictions are still not as close as ground truth. The CNNs trained by CGAN generate more realistic and physically reasonable results, better capturing not only data variability in time and space but also extremes such as intense and long-lasting storms. The new proposed CNN-based downscaling approach can downscale precipitation from 50 km to 12 km in 14 min for 30 years once the network is trained (training takes 4 hours using 1 GPU), while the conventional dynamical downscaling would take 1 months using 600 CPU cores to generate simulations at the resolution of 12 km over contiguous United States.

scholarly journals Application of a Modified Generative Adversarial Network in the Superresolution Reconstruction of Ancient Murals

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Jianfang Cao ◽  
Zibang Zhang ◽  
Aidi Zhao
Keyword(s):  
High Resolution ◽  
Signal To Noise Ratio ◽  
Reconstruction Algorithm ◽  
Structural Similarity ◽  
Image Feature ◽  
Generative Adversarial Network ◽  
Resolution Image ◽  
High Resolution Image ◽  
Adversarial Network ◽  
Set Up

Considering the problems of low resolution and rough details in existing mural images, this paper proposes a superresolution reconstruction algorithm for enhancing artistic mural images, thereby optimizing mural images. The algorithm takes a generative adversarial network (GAN) as the framework. First, a convolutional neural network (CNN) is used to extract image feature information, and then, the features are mapped to the high-resolution image space of the same size as the original image. Finally, the reconstructed high-resolution image is output to complete the design of the generative network. Then, a CNN with deep and residual modules is used for image feature extraction to determine whether the output of the generative network is an authentic, high-resolution mural image. In detail, the depth of the network increases, the residual module is introduced, the batch standardization of the network convolution layer is deleted, and the subpixel convolution is used to realize upsampling. Additionally, a combination of multiple loss functions and staged construction of the network model is adopted to further optimize the mural image. A mural dataset is set up by the current team. Compared with several existing image superresolution algorithms, the peak signal-to-noise ratio (PSNR) of the proposed algorithm increases by an average of 1.2–3.3 dB and the structural similarity (SSIM) increases by 0.04 = 0.13; it is also superior to other algorithms in terms of subjective scoring. The proposed method in this study is effective in the superresolution reconstruction of mural images, which contributes to the further optimization of ancient mural images.

scholarly journals Upsampling Real-Time, Low-Resolution CCTV Videos Using Generative Adversarial Networks

Electronics ◽  
2020 ◽  
Vol 9 (8) ◽  
pp. 1312
Author(s):  
Debapriya Hazra ◽  
Yung-Cheol Byun
Keyword(s):  
High Resolution ◽  
Real Time ◽  
Super Resolution ◽  
Poor Quality ◽  
Generative Adversarial Networks ◽  
Temporal Consistency ◽  
Low Resolution ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Benchmark Datasets

Video super-resolution has become an emerging topic in the field of machine learning. The generative adversarial network is a framework that is widely used to develop solutions for low-resolution videos. Video surveillance using closed-circuit television (CCTV) is significant in every field, all over the world. A common problem with CCTV videos is sudden video loss or poor quality. In this paper, we propose a generative adversarial network that implements spatio-temporal generators and discriminators to enhance real-time low-resolution CCTV videos to high-resolution. The proposed model considers both foreground and background motion of a CCTV video and effectively models the spatial and temporal consistency from low-resolution video frames to generate high-resolution videos. Quantitative and qualitative experiments on benchmark datasets, including Kinetics-700, UCF101, HMDB51 and IITH_Helmet2, showed that our model outperforms the existing GAN models for video super-resolution.

scholarly journals HiCSR: a Hi-C super-resolution framework for producing highly realistic contact maps

2020 ◽  
Author(s):  
Michael C. Dimmick ◽  
Leo J. Lee ◽  
Brendan J. Frey
Keyword(s):  
High Resolution ◽  
Super Resolution ◽  
Cell Types ◽  
Supplementary Information ◽  
Low Resolution ◽  
Generative Adversarial Network ◽  
High Resolution Data ◽  
Contact Maps ◽  
Adversarial Network ◽  
And Function

AbstractMotivationHi-C data has enabled the genome-wide study of chromatin folding and architecture, and has led to important discoveries in the structure and function of chromatin conformation. Here, high resolution data plays a particularly important role as many chromatin substructures such as Topologically Associating Domains (TADs) and chromatin loops cannot be adequately studied with low resolution contact maps. However, the high sequencing costs associated with the generation of high resolution Hi-C data has become an experimental barrier. Data driven machine learning models, which allow low resolution Hi-C data to be computationally enhanced, offer a promising avenue to address this challenge.ResultsBy carefully examining the properties of Hi-C maps and integrating various recent advances in deep learning, we developed a Hi-C Super-Resolution (HiCSR) framework capable of accurately recovering the fine details, textures, and substructures found in high resolution contact maps. This was achieved using a novel loss function tailored to the Hi-C enhancement problem which optimizes for an adversarial loss from a Generative Adversarial Network (GAN), a feature reconstruction loss derived from the latent representation of a denoising autoencoder, and a pixel-wise loss. Not only can the resulting framework generate enhanced Hi-C maps more visually similar to the original high resolution maps, it also excels on a suite of reproducibility metrics produced by members of the ENCODE Consortium compared to existing approaches, including HiCPlus, HiCNN, hicGAN and DeepHiC. Finally, we demonstrate that HiCSR is capable of enhancing Hi-C data across sequencing depth, cell types, and species, recovering biologically significant contact domain boundaries.AvailabilityWe make our implementation available for download at: https://github.com/PSI-Lab/[email protected] informationAvailable Online

scholarly journals Super-Resolution Restoration of Spaceborne Ultra-High-Resolution Images Using the UCL OpTiGAN System

2021 ◽  
Vol 13 (12) ◽  
pp. 2269
Author(s):  
Yu Tao ◽  
Jan-Peter Muller
Keyword(s):  
High Resolution ◽  
Satellite Images ◽  
Processing System ◽  
Super Resolution ◽  
Processing Scheme ◽  
Qualitative And Quantitative ◽  
Video Frames ◽  
High Resolution Satellite Images ◽  
High Resolution Images ◽  
Image Super Resolution

We introduce a robust and light-weight multi-image super-resolution restoration (SRR) method and processing system, called OpTiGAN, using a combination of a multi-image maximum a posteriori approach and a deep learning approach. We show the advantages of using a combined two-stage SRR processing scheme for significantly reducing inference artefacts and improving effective resolution in comparison to other SRR techniques. We demonstrate the optimality of OpTiGAN for SRR of ultra-high-resolution satellite images and video frames from 31 cm/pixel WorldView-3, 75 cm/pixel Deimos-2 and 70 cm/pixel SkySat. Detailed qualitative and quantitative assessments are provided for the SRR results on a CEOS-WGCV-IVOS geo-calibration and validation site at Baotou, China, which features artificial permanent optical targets. Our measurements have shown a 3.69 times enhancement of effective resolution from 31 cm/pixel WorldView-3 imagery to 9 cm/pixel SRR.

scholarly journals DeepHiC: A Generative Adversarial Network for Enhancing Hi-C Data Resolution

2019 ◽  
Author(s):  
Hao Hong ◽  
Shuai Jiang ◽  
Hao Li ◽  
Cheng Quan ◽  
Chenghui Zhao ◽  
...  
Keyword(s):  
High Resolution ◽  
Embryonic Development ◽  
Resolution Enhancement ◽  
Sequencing Data ◽  
Generative Adversarial Network ◽  
Chromatin Loops ◽  
Link Type ◽  
Adversarial Network ◽  
Data Resolution ◽  
Low Coverage

AbstractHi-C is commonly used to study three-dimensional genome organization. However, due to the high sequencing cost and technical constraints, the resolution of most Hi-C datasets is coarse, resulting in a loss of information and biological interpretability. Here we develop DeepHiC, a generative adversarial network, to predict high-resolution Hi-C contact maps from low-coverage sequencing data. We demonstrated that DeepHiC is capable of reproducing high-resolution Hi-C data from as few as 1% downsampled reads. Empowered by adversarial training, our method can restore fine-grained details similar to those in high-resolution Hi-C matrices, boosting accuracy in chromatin loops identification and TADs detection, and outperforms the state-of-the-art methods in accuracy of prediction. Finally, application of DeepHiC to Hi-C data on mouse embryonic development can facilitate chromatin loop detection with higher accuracy. We develop a web-based tool (DeepHiC, http://sysomics.com/deephic) that allows researchers to enhance their own Hi-C data with just a few clicks.Author summaryWe developed a novel method, DeepHiC, for enhancing Hi-C data resolution from low-coverage sequencing data using generative adversarial network. DeepHiC is capable of reproducing highresolution (10-kb) Hi-C data with high quality even using 1/100 downsampled reads. Our method outperforms the previous methods in Hi-C data resolution enhancement, boosting accuracy in chromatin loops identification and TADs detection. Application of DeepHiC on mouse embryonic development data shows that enhancements afforded by DeepHiC facilitates the chromatin loops identification of these data achieving higher accuracy. We also developed a user-friendly web server (http://sysomics.com/deephic) that allows researchers to enhance their own low-resolution Hi-C data (40kb-1 Mb) with just few clicks.

scholarly journals Jitter Detection and Image Restoration Based on Generative Adversarial Networks in Satellite Images

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4693
Author(s):  
Zilin Wang ◽  
Zhaoxiang Zhang ◽  
Limin Dong ◽  
Guodong Xu
Keyword(s):  
Remote Sensing ◽  
Satellite Images ◽  
Satellite Image ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Satellite Attitude ◽  
Adversarial Networks ◽  
Proposed Model ◽  
High Resolution Satellite Images

High-resolution satellite images (HRSIs) obtained from onboard satellite linear array cameras suffer from geometric disturbance in the presence of attitude jitter. Therefore, detection and compensation of satellite attitude jitter are crucial to reduce the geopositioning error and to improve the geometric accuracy of HRSIs. In this work, a generative adversarial network (GAN) architecture is proposed to automatically learn and correct the deformed scene features from a single remote sensing image. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs, and another CNN is used to generate so-called fake inputs. To explore the usefulness and effectiveness of a GAN for jitter detection, the proposed GANs are trained on part of the PatternNet dataset and tested on three popular remote sensing datasets, along with a deformed Yaogan-26 satellite image. Several experiments show that the proposed model provides competitive results. The proposed GAN reveals the enormous potential of GAN-based methods for the analysis of attitude jitter from remote sensing images.

Sign in / Sign up

Export Citation Format

Share Document