scholarly journals Serial GANs: A Feature-Preserving Heterogeneous Remote Sensing Image Transformation Model

2021 ◽  
Vol 13 (19) ◽  
pp. 3968
Author(s):  
Daning Tan ◽  
Yu Liu ◽  
Gang Li ◽  
Libo Yao ◽  
Shun Sun ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Remote Sensing Image ◽  
Transformation Model ◽  
Spectral Distortion ◽  
Image Transformation ◽  
Sar Images ◽  
Optical Images ◽  
Image Translation ◽  
Feature Preserving ◽  
Optical Transformation

In recent years, the interpretation of SAR images has been significantly improved with the development of deep learning technology, and using conditional generative adversarial nets (CGANs) for SAR-to-optical transformation, also known as image translation, has become popular. Most of the existing image translation methods based on conditional generative adversarial nets are modified based on CycleGAN and pix2pix, focusing on style transformation in practice. In addition, SAR images and optical images are characterized by heterogeneous features and large spectral differences, leading to problems such as incomplete image details and spectral distortion in the heterogeneous transformation of SAR images in urban or semiurban areas and with complex terrain. Aiming to solve the problems of SAR-to-optical transformation, Serial GANs, a feature-preserving heterogeneous remote sensing image transformation model, is proposed in this paper for the first time. This model uses the Serial Despeckling GAN and Colorization GAN to complete the SAR-to-optical transformation. Despeckling GAN transforms the SAR images into optical gray images, retaining the texture details and semantic information. Colorization GAN transforms the optical gray images obtained in the first step into optical color images and keeps the structural features unchanged. The model proposed in this paper provides a new idea for heterogeneous image transformation. Through decoupling network design, structural detail information and spectral information are relatively independent in the process of heterogeneous transformation, thereby enhancing the detail information of the generated optical images and reducing its spectral distortion. Using SEN-2 satellite images as the reference, this paper compares the degree of similarity between the images generated by different models and the reference, and the results revealed that the proposed model has obvious advantages in feature reconstruction and the economical volume of the parameters. It also showed that Serial GANs have great potential in decoupling image transformation.

scholarly journals A Robust False Matching Points Detection Method for Remote Sensing Image Registration

2015 ◽  
Vol XL-7/W3 ◽  
pp. 699-702
Author(s):  
X. J. Shan ◽  
P. Tang
Keyword(s):  
Remote Sensing ◽  
Image Registration ◽  
Detection Method ◽  
Graph Transformation ◽  
Remote Sensing Image ◽  
Transformation Model ◽  
Superior Performance ◽  
Geometric Distortion ◽  
Remote Sensing Images ◽  
Accuracy Result

Given the influences of illumination, imaging angle, and geometric distortion, among others, false matching points still occur in all image registration algorithms. Therefore, false matching points detection is an important step in remote sensing image registration. Random Sample Consensus (RANSAC) is typically used to detect false matching points. However, RANSAC method cannot detect all false matching points in some remote sensing images. Therefore, a robust false matching points detection method based on Knearest- neighbour (K-NN) graph (KGD) is proposed in this method to obtain robust and high accuracy result. The KGD method starts with the construction of the K-NN graph in one image. K-NN graph can be first generated for each matching points and its K nearest matching points. Local transformation model for each matching point is then obtained by using its K nearest matching points. The error of each matching point is computed by using its transformation model. Last, L matching points with largest error are identified false matching points and removed. This process is iterative until all errors are smaller than the given threshold. In addition, KGD method can be used in combination with other methods, such as RANSAC. Several remote sensing images with different resolutions and terrains are used in the experiment. We evaluate the performance of KGD method, RANSAC + KGD method, RANSAC, and Graph Transformation Matching (GTM). The experimental results demonstrate the superior performance of the KGD and RANSAC + KGD methods.

scholarly journals MOUNTAINOUS REMOTE SENSING IMAGES REGISTRATION BASED ON IMPROVED OPTICAL FLOW ESTIMATION

2019 ◽  
Vol IV-2/W5 ◽  
pp. 479-484 ◽  
Author(s):  
R. Feng ◽  
X. Li ◽  
H. Shen
Keyword(s):  
Remote Sensing ◽  
Optical Flow ◽  
Remote Sensing Image ◽  
Transformation Model ◽  
Geometric Distortion ◽  
Estimation Methods ◽  
Remote Sensing Images ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Feature Based

<p><strong>Abstract.</strong> Mountainous remote sensing images registration is more complicated than in other areas as geometric distortion caused by topographic relief, which could not be precisely achieved via constructing local mapping functions in the feature-based framework. Optical flow algorithm estimating motion of consecutive frames in computer vision pixel by pixel is introduced for mountainous remote sensing images registration. However, it is sensitive to land cover changes that are inevitable for remote sensing image, resulting in incorrect displacement. To address this problem, we proposed an improved optical flow estimation concentrated on post-processing, namely displacement modification. First of all, the Laplacian of Gaussian (LoG) algorithm is employed to detect the abnormal value in color map of displacement. Then, the abnormal displacement is recalculated in the interpolation surface constructed by the rest accurate displacements. Following the successful coordinate transformation and resampling, the registration outcome is generated. Experiments demonstrated that the proposed method is insensitive in changeable region of mountainous remote sensing image, generating precise registration, outperforming the other local transformation model estimation methods in both visual judgment and quantitative evaluation.</p>

scholarly journals Do Game Data Generalize Well for Remote Sensing Image Segmentation?

2020 ◽  
Vol 12 (2) ◽  
pp. 275 ◽  
Author(s):  
Zhengxia Zou ◽  
Tianyang Shi ◽  
Wenyuan Li ◽  
Zhou Zhang ◽  
Zhenwei Shi
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Video Game ◽  
Real World ◽  
Remote Sensing Data ◽  
Ground Truth ◽  
Remote Sensing Image ◽  
Learning Model ◽  
Image Translation ◽  
Deep Learning Model

Despite the recent progress in deep learning and remote sensing image interpretation, the adaption of a deep learning model between different sources of remote sensing data still remains a challenge. This paper investigates an interesting question: do synthetic data generalize well for remote sensing image applications? To answer this question, we take the building segmentation as an example by training a deep learning model on the city map of a well-known video game “Grand Theft Auto V” and then adapting the model to real-world remote sensing images. We propose a generative adversarial training based segmentation framework to improve the adaptability of the segmentation model. Our model consists of a CycleGAN model and a ResNet based segmentation network, where the former one is a well-known image-to-image translation framework which learns a mapping of the image from the game domain to the remote sensing domain; and the latter one learns to predict pixel-wise building masks based on the transformed data. All models in our method can be trained in an end-to-end fashion. The segmentation model can be trained without using any additional ground truth reference of the real-world images. Experimental results on a public building segmentation dataset suggest the effectiveness of our adaptation method. Our method shows superiority over other state-of-the-art semantic segmentation methods, for example, Deeplab-v3 and UNet. Another advantage of our method is that by introducing semantic information to the image-to-image translation framework, the image style conversion can be further improved.

Simple and efficient remote sensing image transformation for lossless compression

2011 ◽  
Author(s):  
Farshid Sepehrband ◽  
Pedram Ghamisi ◽  
Mohammad Mortazavi ◽  
Jeiran Choupan
Keyword(s):  
Remote Sensing ◽  
Lossless Compression ◽  
Remote Sensing Image ◽  
Image Transformation

scholarly journals Comparative Analysis of Edge Information and Polarization on SAR-to-Optical Translation Based on Conditional Generative Adversarial Networks

2021 ◽  
Vol 13 (1) ◽  
pp. 128
Author(s):  
Qian Zhang ◽  
Xiangnan Liu ◽  
Meiling Liu ◽  
Xinyu Zou ◽  
Lihong Zhu ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Remote Sensing Data ◽  
Weather Conditions ◽  
Structural Similarity ◽  
Spectral Information ◽  
Vegetation Changes ◽  
Sar Images ◽  
Edge Information ◽  
Adversarial Networks ◽  
Optical Images

To accurately describe dynamic vegetation changes, high temporal and spectral resolution data are urgently required. Optical images contain rich spectral information but are limited by poor weather conditions and cloud contamination. Conversely, synthetic-aperture radar (SAR) is effective under all weather conditions but contains insufficient spectral information to recognize certain vegetation changes. Conditional adversarial networks (cGANs) can be adopted to transform SAR images (Sentinel-1) into optical images (Landsat8), which exploits the advantages of both optical and SAR images. As the features of SAR and optical remote sensing data play a decisive role in the translation process, this study explores the quantitative impact of edge information and polarization (VV, VH, VV&VH) on the peak signal-to-noise ratio, structural similarity index measure, correlation coefficient (r), and root mean squared error. The addition of edge information improves the structural similarity between generated and real images. Moreover, using the VH and VV&VH polarization modes as the input provides the cGANs with more effective information and results in better image quality. The optimal polarization mode with the addition of edge information is VV&VH, whereas that without edge information is VV. Near-infrared and short-wave infrared bands in the generated image exhibit higher accuracy (r > 0.8) than visible light bands. The conclusions of this study could serve as an important reference for selecting cGANs input features, and as a potential reference for the applications of cGANs to the SAR-to-optical translation of other multi-source remote sensing data.

scholarly journals Dialectical GAN for SAR Image Translation: From Sentinel-1 to TerraSAR-X

2018 ◽  
Author(s):  
Dongyang Ao ◽  
Corneliu Octavian Dumitru ◽  
Gottfried Schwarz ◽  
Mihai Datcu
Keyword(s):  
Electromagnetic Spectrum ◽  
Sar Image ◽  
Sar Images ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Optical Images ◽  
Image Translation ◽  
Gram Matrices ◽  
Processing Resources

Contrary to optical images, Synthetic Aperture Radar (SAR) images are in different electromagnetic spectrum where the human visual system is not accustomed to. Thus, with more and more SAR applications, the demand for enhanced high-quality SAR images has increased considerably. However, high-quality SAR images entail high costs due to the limitations of current SAR devices and their image processing resources. To improve the quality of SAR images and to reduce the costs of their generation, we propose a Dialectical Generative Adversarial Network (Dialectical GAN) to generate high-quality SAR images. This method is based on the analysis of hierarchical SAR information and the &ldquo;dialectical&rdquo; structure of GAN frameworks.&nbsp; As a demonstration, a typical example will be shown where a low-resolution SAR image (e.g., a Sentinel-1 image) with large ground coverage is translated into a high-resolution SAR image (e.g., a TerraSAR-X image). Three traditional algorithms are compared, and a new algorithm is proposed based on a network framework by combining conditional WGAN-GP (Wasserstein Generative Adversarial Network - Gradient Penalty) loss functions and Spatial Gram matrices under the rule of dialectics. Experimental results show that the SAR image translation works very well when we compare the results of our proposed method with the selected traditional methods.

scholarly journals SAR TO OPTICAL IMAGE SYNTHESIS FOR CLOUD REMOVAL WITH GENERATIVE ADVERSARIAL NETWORKS

2018 ◽  
Vol IV-1 ◽  
pp. 5-11 ◽  
Author(s):  
J. D. Bermudez ◽  
P. N. Happ ◽  
D. A. B. Oliveira ◽  
R. Q. Feitosa
Keyword(s):  
Mapping Function ◽  
Image Synthesis ◽  
Generative Adversarial Networks ◽  
Atmospheric Conditions ◽  
Sar Images ◽  
Active Sensors ◽  
Adversarial Networks ◽  
Optical Images ◽  
Image Translation ◽  
Cloud Removal

<p><strong>Abstract.</strong> Optical imagery is often affected by the presence of clouds. Aiming to reduce their effects, different reconstruction techniques have been proposed in the last years. A common alternative is to extract data from active sensors, like Synthetic Aperture Radar (SAR), because they are almost independent on the atmospheric conditions and solar illumination. On the other hand, SAR images are more complex to interpret than optical images requiring particular handling. Recently, Conditional Generative Adversarial Networks (cGANs) have been widely used in different image generation tasks presenting state-of-the-art results. One application of cGANs is learning a nonlinear mapping function from two images of different domains. In this work, we combine the fact that SAR images are hardly affected by clouds with the ability of cGANS for image translation in order to map optical images from SAR ones so as to recover regions that are covered by clouds. Experimental results indicate that the proposed solution achieves better classification accuracy than SAR based classification.</p>

Sign in / Sign up

Export Citation Format

Share Document