scholarly journals GENERATING ARTIFICIAL NEAR INFRARED SPECTRAL BAND FROM RGB IMAGE USING CONDITIONAL GENERATIVE ADVERSARIAL NETWORK

2020 ◽  
Vol V-3-2020 ◽  
pp. 279-285
Author(s):  
X. Yuan ◽  
J. Tian ◽  
P. Reinartz
Keyword(s):  
Near Infrared ◽  
Spectral Band ◽  
Depth Estimation ◽  
Absolute Error ◽  
Physical Models ◽  
Generative Adversarial Networks ◽  
Surface Model ◽  
Generative Adversarial Network ◽  
Sensing Applications ◽  
Rgb Image

Abstract. Near infrared bands (NIR) provide rich information for many remote sensing applications. In addition to deriving useful indices to delineate water and vegetation, near infrared channels could also be used to facilitate image pre-processing. However, synthesizing bands from RGB spectrum is not an easy task. The inter-correlations between bands are not clearly identified in physical models. Generative adversarial networks (GAN) have been used in many tasks such as generating photorealistic images, monocular depth estimation and Digital Surface Model (DSM) refinement etc. Conditional GAN is different in that it observes some data as a condition. In this paper, we explore a cGAN network structure to generate a NIR spectral band that is conditioned on the input RGB image. We test different discriminators and loss functions, and evaluate results using various metrics. The best simulated NIR channel has a mean absolute error of around 5 percent in Sentinel-2 dataset. In addition, the simulated NIR image can correctly distinguish between various classes of landcover.

scholarly journals Single-Image Depth Inference Using Generative Adversarial Networks

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1708 ◽  
Author(s):  
Daniel Stanley Tan ◽  
Chih-Yuan Yao ◽  
Conrado Ruiz ◽  
Kai-Lung Hua
Keyword(s):  
Smart Cities ◽  
Depth Map ◽  
Depth Estimation ◽  
Input Image ◽  
Generative Adversarial Networks ◽  
Depth Information ◽  
Single Image ◽  
Neural Network Models ◽  
Generative Adversarial Network ◽  
Depth Sensors

Depth has been a valuable piece of information for perception tasks such as robot grasping, obstacle avoidance, and navigation, which are essential tasks for developing smart homes and smart cities. However, not all applications have the luxury of using depth sensors or multiple cameras to obtain depth information. In this paper, we tackle the problem of estimating the per-pixel depths from a single image. Inspired by the recent works on generative neural network models, we formulate the task of depth estimation as a generative task where we synthesize an image of the depth map from a single Red, Green, and Blue (RGB) input image. We propose a novel generative adversarial network that has an encoder-decoder type generator with residual transposed convolution blocks trained with an adversarial loss. Quantitative and qualitative experimental results demonstrate the effectiveness of our approach over several depth estimation works.

scholarly journals Super-Resolution of Remote Sensing Images via a Dense Residual Generative Adversarial Network

2019 ◽  
Vol 11 (21) ◽  
pp. 2578 ◽  
Author(s):  
Wen Ma ◽  
Zongxu Pan ◽  
Feng Yuan ◽  
Bin Lei
Keyword(s):  
Remote Sensing ◽  
Network Architecture ◽  
Super Resolution ◽  
Objective Evaluation ◽  
Generative Adversarial Networks ◽  
Remote Sensing Images ◽  
Generative Adversarial Network ◽  
Memory Mechanism ◽  
Adversarial Network ◽  
Sensing Applications

Single image super-resolution (SISR) has been widely studied in recent years as a crucial technique for remote sensing applications. In this paper, a dense residual generative adversarial network (DRGAN)-based SISR method is proposed to promote the resolution of remote sensing images. Different from previous super-resolution (SR) approaches based on generative adversarial networks (GANs), the novelty of our method mainly lies in the following factors. First, we made a breakthrough in terms of network architecture to improve performance. We designed a dense residual network as the generative network in GAN, which can make full use of the hierarchical features from low-resolution (LR) images. We also introduced a contiguous memory mechanism into the network to take advantage of the dense residual block. Second, we modified the loss function and altered the model of the discriminative network according to the Wasserstein GAN with a gradient penalty (WGAN-GP) for stable training. Extensive experiments were performed using the NWPU-RESISC45 dataset, and the results demonstrated that the proposed method outperforms state-of-the-art methods in terms of both objective evaluation and subjective perspective.

scholarly journals The Influence of Shadow Effects on the Spectral Characteristics of Glacial Meltwater

2020 ◽  
Vol 13 (1) ◽  
pp. 36
Author(s):  
Kornelia Anna Wójcik-Długoborska ◽  
Robert Józef Bialik
Keyword(s):  
Spectral Properties ◽  
Satellite Images ◽  
Near Infrared ◽  
Spectral Band ◽  
Spectral Characteristics ◽  
High Sensitivity ◽  
Surface Model ◽  
Landsat 8 ◽  
Pixel Resolution ◽  
Ice Phenomena

The phenomenon of shadows due to glaciers is investigated in Antarctica. The observed shadow effect disrupts analyses conducted by remote sensing and is a challenge in the assessment of sediment meltwater plumes in polar marine environments. A DJI Inspire 2 drone equipped with a Zenmuse x5s camera was used to generate a digital surface model (DSM) of 6 King George Island glaciers: Ecology, Dera, Zalewski, Ladies, Krak, and Vieville. On this basis, shaded areas of coves near glaciers were traced. For the first time, spectral characteristics of shaded meltwater were observed with the simultaneous use of a Sequoia+ spectral camera mounted on a Parrot Bluegrass drone and in Landsat 8 satellite images. In total, 44 drone flights were made, and 399 satellite images were analyzed. Among them, four drone spectral images and four satellite images were selected, meeting the condition of a visible shadow. For homogeneous waters (deep, low turbidity, without ice phenomena), the spectral properties tend to change during the approach to an obstacle casting a shadow especially during low shortwave downward radiation. In this case, in the shade, the amount of radiation reflected in the green spectral band decreases by 50% far from the obstacle and by 43% near the obstacle, while in near infrared (NIR), it decreases by 42% and 21%, respectively. With highly turbid, shallow water and ice phenomena, this tendency does not occur. It was found that the green spectral band had the highest contrast in the amount of reflected radiation between nonshaded and shaded areas, but due to its high sensitivity, the analysis could have been overestimated. The spectral properties of shaded meltwater differ depending on the distance from the glacier front, which is related to the saturation of the water with sediment particles. We discovered that the pixel aggregation of uniform areas caused the loss of detailed information, while pixel aggregation of nonuniform, shallow areas with ice phenomena caused changes and the loss of original information. During the aggregation of the original pixel resolution (15 cm) up to 30 m, the smallest error occurred in the area with a homogeneous water surface, while the greatest error (over 100%) was identified in the places where the water was strongly cloudy or there were ice phenomena.

scholarly journals Whole Body Positron Emission Tomography Attenuation Correction Map Synthesizing using 3D Deep Generative Adversarial Networks

2020 ◽  
Author(s):  
Ramiro Rodriguez Colmeiro ◽  
Claudio Verrastro ◽  
Daniel Minsky ◽  
Thomas Grosges
Keyword(s):  
Positron Emission Tomography ◽  
Structural Information ◽  
Absolute Error ◽  
Emission Tomography ◽  
Whole Body ◽  
Generative Adversarial Networks ◽  
Multiple Sources ◽  
Generative Adversarial Network ◽  
Attenuation Map ◽  
Positron Emission

Abstract Background: The correction of attenuation effects in Positron Emission Tomography (PET) imaging is fundamental to obtain a correct radiotracer distribution. However direct measurement of this attenuation map is not error-free and normally results in additional ionization radiation dose to the patient. Here, we propose to obtain the whole body attenuation map using a 3D U-Net generative adversarial network. The network is trained to learn the mapping from non attenuation corrected 18-F-fluorodeoxyglucose PET images to a synthetic Computerized Tomography (sCT) and also to label the input voxel tissue. The sCT image is further refined using an adversarial training scheme to recover higher frequency details and lost structures using context information. This work is trained and tested on public available datasets, containing several PET images from different scanners with different radiotracer administration and reconstruction modalities. The network is trained with 108 samples and validated on 10 samples.Results: The sCT generation was tested on 133 samples from 8 distinct datasets. The resulting mean absolute error of the network is 103 ± 18 HU and a peak signal to noise ratio of 18.6 ± 1.5 dB. The generated images show good correlation with the unknown structural information.Conclusions: The proposed deep learning topology is capable of generating whole body attenuation maps from uncorrected PET image data. Moreover, the method accuracy holds in the presence of data form multiple sources and modalities and is trained on publicly available datasets.

scholarly journals Generation of the NIR Spectral Band for Satellite Images with Convolutional Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5646 ◽  
Author(s):  
Svetlana Illarionova ◽  
Dmitrii Shadrin ◽  
Alexey Trekin ◽  
Vladimir Ignatiev ◽  
Ivan Oseledets
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Near Infrared ◽  
Spectral Band ◽  
Model Performance ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
High Resolution Satellite Imagery ◽  
Segmentation Task ◽  
The Impact

The near-infrared (NIR) spectral range (from 780 to 2500 nm) of the multispectral remote sensing imagery provides vital information for landcover classification, especially concerning vegetation assessment. Despite the usefulness of NIR, it does not always accomplish common RGB. Modern achievements in image processing via deep neural networks make it possible to generate artificial spectral information, for example, to solve the image colorization problem. In this research, we aim to investigate whether this approach can produce not only visually similar images but also an artificial spectral band that can improve the performance of computer vision algorithms for solving remote sensing tasks. We study the use of a generative adversarial network (GAN) approach in the task of the NIR band generation using only RGB channels of high-resolution satellite imagery. We evaluate the impact of a generated channel on the model performance to solve the forest segmentation task. Our results show an increase in model accuracy when using generated NIR compared to the baseline model, which uses only RGB (0.947 and 0.914 F1-scores, respectively). The presented study shows the advantages of generating the extra band such as the opportunity to reduce the required amount of labeled data.

ORGANIC (1).pdf

2017 ◽  
Author(s):  
Benjamin Sanchez-Lengeling ◽  
Carlos Outeiral ◽  
Gabriel L. Guimaraes ◽  
Alan Aspuru-Guzik
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
Chemical Species ◽  
Material Design ◽  
Organic Photovoltaic ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

scholarly journals Restoring Raindrops Using Attentive Generative Adversarial Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 7034
Author(s):  
Hee-Deok Yang
Keyword(s):  
Weather Conditions ◽  
Recurrent Network ◽  
Generative Adversarial Networks ◽  
Navigation Systems ◽  
Vision Systems ◽  
Generative Adversarial Network ◽  
Network Layers ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Outdoor Vision

Artificial intelligence technologies and vision systems are used in various devices, such as automotive navigation systems, object-tracking systems, and intelligent closed-circuit televisions. In particular, outdoor vision systems have been applied across numerous fields of analysis. Despite their widespread use, current systems work well under good weather conditions. They cannot account for inclement conditions, such as rain, fog, mist, and snow. Images captured under inclement conditions degrade the performance of vision systems. Vision systems need to detect, recognize, and remove noise because of rain, snow, and mist to boost the performance of the algorithms employed in image processing. Several studies have targeted the removal of noise resulting from inclement conditions. We focused on eliminating the effects of raindrops on images captured with outdoor vision systems in which the camera was exposed to rain. An attentive generative adversarial network (ATTGAN) was used to remove raindrops from the images. This network was composed of two parts: an attentive-recurrent network and a contextual autoencoder. The ATTGAN generated an attention map to detect rain droplets. A de-rained image was generated by increasing the number of attentive-recurrent network layers. We increased the number of visual attentive-recurrent network layers in order to prevent gradient sparsity so that the entire generation was more stable against the network without preventing the network from converging. The experimental results confirmed that the extended ATTGAN could effectively remove various types of raindrops from images.

scholarly journals Prediction and analysis of multiple protein lysine modified sites based on conditional wasserstein generative adversarial networks

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Yingxi Yang ◽  
Hui Wang ◽  
Wen Li ◽  
Xiaobo Wang ◽  
Shizhao Wei ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Sequence Data ◽  
Rapid Development ◽  
Pearson Correlation ◽  
Structural Features ◽  
Generative Adversarial Networks ◽  
Post Translational Modification ◽  
Generative Adversarial Network ◽  
Data Imbalance ◽  
Adversarial Network

Abstract Background Protein post-translational modification (PTM) is a key issue to investigate the mechanism of protein’s function. With the rapid development of proteomics technology, a large amount of protein sequence data has been generated, which highlights the importance of the in-depth study and analysis of PTMs in proteins. Method We proposed a new multi-classification machine learning pipeline MultiLyGAN to identity seven types of lysine modified sites. Using eight different sequential and five structural construction methods, 1497 valid features were remained after the filtering by Pearson correlation coefficient. To solve the data imbalance problem, Conditional Generative Adversarial Network (CGAN) and Conditional Wasserstein Generative Adversarial Network (CWGAN), two influential deep generative methods were leveraged and compared to generate new samples for the types with fewer samples. Finally, random forest algorithm was utilized to predict seven categories. Results In the tenfold cross-validation, accuracy (Acc) and Matthews correlation coefficient (MCC) were 0.8589 and 0.8376, respectively. In the independent test, Acc and MCC were 0.8549 and 0.8330, respectively. The results indicated that CWGAN better solved the existing data imbalance and stabilized the training error. Alternatively, an accumulated feature importance analysis reported that CKSAAP, PWM and structural features were the three most important feature-encoding schemes. MultiLyGAN can be found at https://github.com/Lab-Xu/MultiLyGAN. Conclusions The CWGAN greatly improved the predictive performance in all experiments. Features derived from CKSAAP, PWM and structure schemes are the most informative and had the greatest contribution to the prediction of PTM.

scholarly journals Stochastic Restoration of Heavily Compressed Musical Audio Using Generative Adversarial Networks

Electronics ◽  
2021 ◽  
Vol 10 (11) ◽  
pp. 1349
Author(s):  
Stefan Lattner ◽  
Javier Nistal
Keyword(s):  
Data Storage ◽  
Audio Signal ◽  
Human Perception ◽  
Generative Adversarial Networks ◽  
Audio Signals ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Extensive Evaluation ◽  
Listening Tests ◽  
Musical Audio

Lossy audio codecs compress (and decompress) digital audio streams by removing information that tends to be inaudible in human perception. Under high compression rates, such codecs may introduce a variety of impairments in the audio signal. Many works have tackled the problem of audio enhancement and compression artifact removal using deep-learning techniques. However, only a few works tackle the restoration of heavily compressed audio signals in the musical domain. In such a scenario, there is no unique solution for the restoration of the original signal. Therefore, in this study, we test a stochastic generator of a Generative Adversarial Network (GAN) architecture for this task. Such a stochastic generator, conditioned on highly compressed musical audio signals, could one day generate outputs indistinguishable from high-quality releases. Therefore, the present study may yield insights into more efficient musical data storage and transmission. We train stochastic and deterministic generators on MP3-compressed audio signals with 16, 32, and 64 kbit/s. We perform an extensive evaluation of the different experiments utilizing objective metrics and listening tests. We find that the models can improve the quality of the audio signals over the MP3 versions for 16 and 32 kbit/s and that the stochastic generators are capable of generating outputs that are closer to the original signals than those of the deterministic generators.

Sign in / Sign up

Export Citation Format

Share Document