scholarly journals SEGMENTATION OF SINGLE STANDING DEAD TREES IN HIGH-RESOLUTION AERIAL IMAGERY WITH GENERATIVE ADVERSARIAL NETWORK-BASED SHAPE PRIORS

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 717-723
Author(s):  
P. Polewski ◽  
J. Shelton ◽  
W. Yao ◽  
M. Heurich
Keyword(s):  
Near Infrared ◽  
Spectral Response ◽  
Mathematical Formulation ◽  
Dimensional Manifold ◽  
Shape Priors ◽  
Tree Crown ◽  
Infrared Band ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Adversarial Network

Abstract. The use of multispectral imagery for monitoring biodiversity in ecosystems is becoming widespread. A key parameter of forest ecosystems is the distribution of dead wood. This work addresses the segmentation of individual dead tree crowns in nadir-view aerial infrared imagery. While dead vegetation produces a distinct spectral response in the near infrared band, separating adjacent trees within large swaths of dead stands remains a challenge. We tackle this problem by casting the segmentation task within the active contour framework, a mathematical formulation combining learned models of the object’s shape and appearance as prior information. We explore the use of a deep convolutional generative adversarial network (DCGAN) in the role of the shape model, replacing the original linear mixture-of-eigenshapes formulation. Also, we rely on probabilities obtained from a deep fully convolutional network (FCN) as the appearance prior. Experiments conducted on manually labeled reference polygons show that the DCGAN is able to learn a low-dimensional manifold of tree crown shapes, outperforming the eigenshape model with respect to the similarity of the reproduced and referenced shapes on about 45 % of the test samples. The DCGAN is successful mostly for less convex shapes, whereas the baseline remains superior for more regular tree crown polygons.

SGAN4AbSum: A Semantic-Enhanced Generative Adversarial Network for Abstractive Text Summarization

2021 ◽  
Author(s):  
Tham Vo
Keyword(s):  
Ground Truth ◽  
Text Summarization ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Training Strategy ◽  
Adversarial Network ◽  
Deep Recurrent Neural Network ◽  
Benchmark Datasets ◽  
Latent Representations ◽  
Abstractive Summarization

Abstract In abstractive summarization task, most of proposed models adopt the deep recurrent neural network (RNN)-based encoder-decoder architecture to learn and generate meaningful summary for a given input document. However, most of recent RNN-based models always suffer the challenges related to the involvement of much capturing high-frequency/reparative phrases in long documents during the training process which leads to the outcome of trivial and generic summaries are generated. Moreover, the lack of thorough analysis on the sequential and long-range dependency relationships between words within different contexts while learning the textual representation also make the generated summaries unnatural and incoherent. To deal with these challenges, in this paper we proposed a novel semantic-enhanced generative adversarial network (GAN)-based approach for abstractive text summarization task, called as: SGAN4AbSum. We use an adversarial training strategy for our text summarization model in which train the generator and discriminator to simultaneously handle the summary generation and distinguishing the generated summary with the ground-truth one. The input of generator is the jointed rich-semantic and global structural latent representations of training documents which are achieved by applying a combined BERT and graph convolutional network (GCN) textual embedding mechanism. Extensive experiments in benchmark datasets demonstrate the effectiveness of our proposed SGAN4AbSum which achieve the competitive ROUGE-based scores in comparing with state-of-the-art abstractive text summarization baselines.

scholarly journals Insulator Segmentation for Power Line Inspection Based on Modified Conditional Generative Adversarial Network

2019 ◽  
Vol 2019 ◽  
pp. 1-8
Author(s):  
Zishu Gao ◽  
Guodong Yang ◽  
En Li ◽  
Tianyu Shen ◽  
Zhe Wang ◽  
...  
Keyword(s):  
Transmission Lines ◽  
Power Supply ◽  
Power Transmission ◽  
State Of The Art ◽  
Power Line ◽  
Power Transmission Lines ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Adversarial Network ◽  
Feature Information

There are a large number of insulators on the transmission line, and insulator damage will have a major impact on power supply security. Image-based segmentation of the insulators in the power transmission lines is a premise and also a critical task for power line inspection. In this paper, a modified conditional generative adversarial network for insulator pixel-level segmentation is proposed. The generator is reconstructed by encoder-decoder layers with asymmetric convolution kernel which can simplify the network complexity and extract more kinds of feature information. The discriminator is composed of a fully convolutional network based on patchGAN and learns the loss to train the generator. It is verified in experiments that the proposed method has better performances on mIoU and computational efficiency than Pix2pix, SegNet, and other state-of-the-art networks.

scholarly journals Design of Painting Art Style Rendering System Based on Convolutional Neural Network

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Xingyu Xie ◽  
Bin Lv
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Data Set ◽  
Adversarial Network ◽  
Paired Samples ◽  
Network Generator ◽  
The Stability ◽  
Art Style

Convolutional Neural Network- (CNN-) based GAN models mainly suffer from problems such as data set limitation and rendering efficiency in the segmentation and rendering of painting art. In order to solve these problems, this paper uses the improved cycle generative adversarial network (CycleGAN) to render the current image style. This method replaces the deep residual network (ResNet) of the original network generator with a dense connected convolutional network (DenseNet) and uses the perceptual loss function for adversarial training. The painting art style rendering system built in this paper is based on perceptual adversarial network (PAN) for the improved CycleGAN that suppresses the limitation of the network model on paired samples. The proposed method also improves the quality of the image generated by the artistic style of painting and further improves the stability and speeds up the network convergence speed. Experiments were conducted on the painting art style rendering system based on the proposed model. Experimental results have shown that the image style rendering method based on the perceptual adversarial error to improve the CycleGAN + PAN model can achieve better results. The PSNR value of the generated image is increased by 6.27% on average, and the SSIM values are all increased by about 10%. Therefore, the improved CycleGAN + PAN image painting art style rendering method produces better painting art style images, which has strong application value.

scholarly journals Asymmetric Encoder-Decoder Structured FCN Based LiDAR to Color Image Generation

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4818 ◽  
Author(s):  
Hyun-Koo Kim ◽  
Kook-Yeol Yoo ◽  
Ju H. Park ◽  
Ho-Youl Jung
Keyword(s):  
Color Image ◽  
Ground Truth ◽  
Sensor Data ◽  
Image Generation ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Adversarial Network ◽  
Ground Truth Image ◽  
And Performance ◽  
Reflection Intensity

In this paper, we propose a method of generating a color image from light detection and ranging (LiDAR) 3D reflection intensity. The proposed method is composed of two steps: projection of LiDAR 3D reflection intensity into 2D intensity, and color image generation from the projected intensity by using a fully convolutional network (FCN). The color image should be generated from a very sparse projected intensity image. For this reason, the FCN is designed to have an asymmetric network structure, i.e., the layer depth of the decoder in the FCN is deeper than that of the encoder. The well-known KITTI dataset for various scenarios is used for the proposed FCN training and performance evaluation. Performance of the asymmetric network structures are empirically analyzed for various depth combinations for the encoder and decoder. Through simulations, it is shown that the proposed method generates fairly good visual quality of images while maintaining almost the same color as the ground truth image. Moreover, the proposed FCN has much higher performance than conventional interpolation methods and generative adversarial network based Pix2Pix. One interesting result is that the proposed FCN produces shadow-free and daylight color images. This result is caused by the fact that the LiDAR sensor data is produced by the light reflection and is, therefore, not affected by sunlight and shadow.

scholarly journals Conditional Generative Adversarial Networks (cGANs) for Near Real-Time Precipitation Estimation from Multispectral GOES-16 Satellite Imageries—PERSIANN-cGAN

2019 ◽  
Vol 11 (19) ◽  
pp. 2193 ◽  
Author(s):  
Negin Hayatbini ◽  
Bailey Kong ◽  
Kuo-lin Hsu ◽  
Phu Nguyen ◽  
Soroosh Sorooshian ◽  
...  
Keyword(s):  
Mean Squared Error ◽  
Probability Of Detection ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Critical Success ◽  
Adversarial Network ◽  
Precipitation Estimation ◽  
Precipitation Estimates ◽  
Global Precipitation

In this paper, we present a state-of-the-art precipitation estimation framework which leverages advances in satellite remote sensing as well as Deep Learning (DL). The framework takes advantage of the improvements in spatial, spectral and temporal resolutions of the Advanced Baseline Imager (ABI) onboard the GOES-16 platform along with elevation information to improve the precipitation estimates. The procedure begins by first deriving a Rain/No Rain (R/NR) binary mask through classification of the pixels and then applying regression to estimate the amount of rainfall for rainy pixels. A Fully Convolutional Network is used as a regressor to predict precipitation estimates. The network is trained using the non-saturating conditional Generative Adversarial Network (cGAN) and Mean Squared Error (MSE) loss terms to generate results that better learn the complex distribution of precipitation in the observed data. Common verification metrics such as Probability Of Detection (POD), False Alarm Ratio (FAR), Critical Success Index (CSI), Bias, Correlation and MSE are used to evaluate the accuracy of both R/NR classification and real-valued precipitation estimates. Statistics and visualizations of the evaluation measures show improvements in the precipitation retrieval accuracy in the proposed framework compared to the baseline models trained using conventional MSE loss terms. This framework is proposed as an augmentation for PERSIANN-CCS (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network- Cloud Classification System) algorithm for estimating global precipitation.

scholarly journals Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network

2021 ◽  
Vol 13 (8) ◽  
pp. 1512
Author(s):  
Quan Xiong ◽  
Liping Di ◽  
Quanlong Feng ◽  
Diyou Liu ◽  
Wei Liu ◽  
...  
Keyword(s):  
Land Surface ◽  
Near Infrared ◽  
Similarity Index ◽  
Data Availability ◽  
Optical Data ◽  
Generative Adversarial Network ◽  
Spectral Angle Mapper ◽  
Adversarial Network ◽  
Target Date ◽  
Sentinel 2

Sentinel-2 images have been widely used in studying land surface phenomena and processes, but they inevitably suffer from cloud contamination. To solve this critical optical data availability issue, it is ideal to fuse Sentinel-1 and Sentinel-2 images to create fused, cloud-free Sentinel-2-like images for facilitating land surface applications. In this paper, we propose a new data fusion model, the Multi-channels Conditional Generative Adversarial Network (MCcGAN), based on the conditional generative adversarial network, which is able to convert images from Domain A to Domain B. With the model, we were able to generate fused, cloud-free Sentinel-2-like images for a target date by using a pair of reference Sentinel-1/Sentinel-2 images and target-date Sentinel-1 images as inputs. In order to demonstrate the superiority of our method, we also compared it with other state-of-the-art methods using the same data. To make the evaluation more objective and reliable, we calculated the root-mean-square-error (RSME), R2, Kling–Gupta efficiency (KGE), structural similarity index (SSIM), spectral angle mapper (SAM), and peak signal-to-noise ratio (PSNR) of the simulated Sentinel-2 images generated by different methods. The results show that the simulated Sentinel-2 images generated by the MCcGAN have a higher quality and accuracy than those produced via the previous methods.

scholarly journals Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

2021 ◽  
Vol 14 (1) ◽  
pp. 144
Author(s):  
Luiz E. Christovam ◽  
Milton H. Shimabukuro ◽  
Maria de Lourdes B. T. Galo ◽  
Eija Honkavaara
Keyword(s):  
Time Series ◽  
Loss Function ◽  
Visual Analysis ◽  
Spectral Response ◽  
Semantic Segmentation ◽  
Generative Models ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Crop Type

Clouds are one of the major limitations to crop monitoring using optical satellite images. Despite all efforts to provide decision-makers with high-quality agricultural statistics, there is still a lack of techniques to optimally process satellite image time series in the presence of clouds. In this regard, in this article it was proposed to add a Multi-Layer Perceptron loss function to the pix2pix conditional Generative Adversarial Network (cGAN) objective function. The aim was to enforce the generative model to learn how to deliver synthetic pixels whose values were proxies for the spectral response improving further crop type mapping. Furthermore, it was evaluated the generalization capacity of the generative models in producing pixels with plausible values for images not used in the training. To assess the performance of the proposed approach it was compared real images with synthetic images generated with the proposed approach as well as with the original pix2pix cGAN. The comparative analysis was performed through visual analysis, pixel values analysis, semantic segmentation and similarity metrics. In general, the proposed approach provided slightly better synthetic pixels than the original pix2pix cGAN, removing more noise than the original pix2pix algorithm as well as providing better crop type semantic segmentation; the semantic segmentation of the synthetic image generated with the proposed approach achieved an F1-score of 44.2%, while the real image achieved 44.7%. Regarding the generalization, the models trained utilizing different regions of the same image provided better pixels than models trained using other images in the time series. Besides this, the experiments also showed that the models trained using a pair of images selected every three months along the time series also provided acceptable results on images that do not have cloud-free areas.

scholarly journals HeartNet: Self Multi-Head Attention Mechanism via Convolutional Network with Adversarial Data Synthesis for ECG-based Arrhythmia Classification

2021 ◽  
Author(s):  
Taki Hasan Rafi ◽  
Young Woong-Ko
Keyword(s):  
Deep Learning ◽  
Attention Mechanism ◽  
Classification Systems ◽  
Insufficient Data ◽  
Learning Method ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Data Synthesis ◽  
Adversarial Network ◽  
Main Challenge

Cardiovascular disease is now one of the leading causes of morbidity and mortality in humans. Electrocardiogram (ECG) is a reliable tool for monitoring the health of the cardiovascular system. Currently, there has been a lot of focus on accurately categorizing heartbeats. There is a high demand on automatic ECG classification systems to assist medical professionals. In this paper we proposed a new deep learning method called HeartNet for developing an automatic ECG classifier. The proposed deep learning method is compressed by multi-head attention mechanism on top of CNN model. The main challenge of insufficient data label is solved by adversarial data synthesis adopting generative adversarial network (GAN) with generating additional training samples. It drastically improves the overall performance of the proposed method by 5-10% on each insufficient data label category. We evaluated our proposed method utilizing MIT-BIH dataset. Our proposed method has shown 99.67 ± 0.11 accuracy and 89.24 ± 1.71 MCC trained with adversarial data synthesized dataset. However, we have also utilized two individual datasets such as Atrial Fibrillation Detection Database and PTB Diagnostic Database to see the performance of our proposed model on ECG classification. The effectiveness and robustness of proposed method are validated by extensive experiments, comparison and analysis. Later on, we also highlighted some limitations of this work.

Sign in / Sign up

Export Citation Format

Share Document