scholarly journals A Generative Adversarial Network-Based Image Denoiser Controlling Heterogeneous Losses

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1191
Author(s):  
Sung In Cho ◽  
Jae Hyeon Park ◽  
Suk-Ju Kang
Keyword(s):  
Visual Information ◽  
Mean Squared Error ◽  
Structural Information ◽  
Similarity Index ◽  
Generative Adversarial Network ◽  
Denoising Method ◽  
Adversarial Network ◽  
Feature Similarity Index ◽  
Structural Loss ◽  
Information Fidelity

We propose a novel generative adversarial network (GAN)-based image denoising method that utilizes heterogeneous losses. In order to improve the restoration quality of the structural information of the generator, the heterogeneous losses, including the structural loss in addition to the conventional mean squared error (MSE)-based loss, are used to train the generator. To maximize the improvements brought on by the heterogeneous losses, the strength of the structural loss is adaptively adjusted by the discriminator for each input patch. In addition, a depth wise separable convolution-based module that utilizes the dilated convolution and symmetric skip connection is used for the proposed GAN so as to reduce the computational complexity while providing improved denoising quality compared to the convolutional neural network (CNN) denoiser. The experiments showed that the proposed method improved visual information fidelity and feature similarity index values by up to 0.027 and 0.008, respectively, compared to the existing CNN denoiser.

scholarly journals High-fidelity fast volumetric brain MRI using a hybrid denoising generative adversarial network (HDnGAN)

2021 ◽  
Author(s):  
Ziyu Li ◽  
Qiyuan Tian ◽  
Chanon Ngamsombat ◽  
Samuel Cartmell ◽  
John Conklin ◽  
...  
Keyword(s):  
Parallel Imaging ◽  
Mean Squared Error ◽  
Brain Mri ◽  
Similarity Index ◽  
Image Synthesis ◽  
Image Sharpness ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Significant Difference ◽  
Multiple Sclerosis Patients

Purpose: To improve the signal-to-noise ratio (SNR) of highly accelerated volumetric MRI while preserve realistic textures using a generative adversarial network (GAN). Methods: A hybrid GAN for denoising entitled "HDnGAN" with a 3D generator and a 2D discriminator was proposed to denoise 3D T2-weighted fluid-attenuated inversion recovery (FLAIR) images acquired in 2.75 minutes (R=3×2) using wave-controlled aliasing in parallel imaging (Wave-CAIPI). HDnGAN was trained on data from 25 multiple sclerosis patients by minimizing a combined mean squared error and adversarial loss with adjustable weight λ. Results were evaluated on eight separate patients by comparing to standard T2-SPACE FLAIR images acquired in 7.25 minutes (R=2×2) using mean absolute error (MAE), peak SNR (PSNR), structural similarity index (SSIM), and VGG perceptual loss, and by two neuroradiologists using a five-point score regarding gray-white matter contrast, sharpness, SNR, lesion conspicuity, and overall quality. Results: HDnGAN (λ=0) produced the lowest MAE, highest PSNR and SSIM. HDnGAN (λ=10-3) produced the lowest VGG loss. In the reader study, HDnGAN (λ=10-3) significantly improved the gray-white contrast and SNR of Wave-CAIPI images, and outperformed BM4D and HDnGAN (λ=0) regarding image sharpness. The overall quality score from HDnGAN (λ=10-3) was significantly higher than those from Wave-CAIPI, BM4D, and HDnGAN (λ=0), with no significant difference compared to standard images. Conclusion: HDnGAN concurrently benefits from improved image synthesis performance of 3D convolution and increased training samples for training the 2D discriminator on limited data. HDnGAN generates images with high SNR and realistic textures, similar to those acquired in longer times and preferred by neuroradiologists.

scholarly journals Single Image Super-Resolution: Depthwise Separable Convolution Super-Resolution Generative Adversarial Network

2020 ◽  
Vol 10 (1) ◽  
pp. 375 ◽  
Author(s):  
Zetao Jiang ◽  
Yongsong Huang ◽  
Lirui Hu
Keyword(s):  
Similarity Index ◽  
Super Resolution ◽  
Structural Similarity ◽  
Image Features ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Visual Evaluation ◽  
Adversarial Network ◽  
Image Super Resolution ◽  
Single Image Super Resolution

The super-resolution generative adversarial network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied by unpleasant artifacts. To further enhance the visual quality, we propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The method is based on depthwise separable convolution super-resolution generative adversarial network (DSCSRGAN). A new depthwise separable convolution dense block (DSC Dense Block) was designed for the generator network, which improved the ability to represent and extract image features, while greatly reducing the total amount of parameters. For the discriminator network, the batch normalization (BN) layer was discarded, and the problem of artifacts was reduced. A frequency energy similarity loss function was designed to constrain the generator network to generate better super-resolution images. Experiments on several different datasets showed that the peak signal-to-noise ratio (PSNR) was improved by more than 3 dB, structural similarity index (SSIM) was increased by 16%, and the total parameter was reduced to 42.8% compared with the original model. Combining various objective indicators and subjective visual evaluation, the algorithm was shown to generate richer image details, clearer texture, and lower complexity.

Development and investigation of a deep learning based method for TEC map completion

2020 ◽  
Author(s):  
Mingwu Jin ◽  
Yang Pan ◽  
Shunrong Zhang ◽  
Yue Deng
Keyword(s):  
Solar Activity ◽  
Mean Squared Error ◽  
Image Inpainting ◽  
Science Research ◽  
High Solar Activity ◽  
Electron Content ◽  
International Gnss Service ◽  
Generative Adversarial Network ◽  
Total Electron ◽  
Adversarial Network

<p>Because of the limited coverage of receiver stations, current measurements of Total Electron Content (TEC) by ground-based GNSS receivers are not complete with large portions of data gaps. The processing to obtain complete TEC maps for space science research is time consuming and needs the collaboration of five International GNSS Service (IGS) Ionosphere Associate Analysis Centers (IAACs) to use different data processing and filling algorithms and to consolidate their results into final IGS completed TEC maps. In this work, we developed a Deep Convolutional Generative Adversarial Network (DCGAN) and Poisson blending model (DCGAN-PB) to learn IGS completion process for automatic completion of TEC maps. Using 10-fold cross validation of 20-year IGS TEC data, DCGAN-PB achieves the average root mean squared error (RMSE) about 4 absolute TEC units (TECu) of the high solar activity years and around 2 TECu for low solar activity years, which is about 50% reduction of RMSE for recovered TEC values compared to two conventional single-image inpainting methods. The developed DCGAN-PB model can lead to an efficient automatic completion tool for TEC maps.</p>

scholarly journals Conditional Generative Adversarial Networks (cGANs) for Near Real-Time Precipitation Estimation from Multispectral GOES-16 Satellite Imageries—PERSIANN-cGAN

2019 ◽  
Vol 11 (19) ◽  
pp. 2193 ◽  
Author(s):  
Negin Hayatbini ◽  
Bailey Kong ◽  
Kuo-lin Hsu ◽  
Phu Nguyen ◽  
Soroosh Sorooshian ◽  
...  
Keyword(s):  
Mean Squared Error ◽  
Probability Of Detection ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Critical Success ◽  
Adversarial Network ◽  
Precipitation Estimation ◽  
Precipitation Estimates ◽  
Global Precipitation

In this paper, we present a state-of-the-art precipitation estimation framework which leverages advances in satellite remote sensing as well as Deep Learning (DL). The framework takes advantage of the improvements in spatial, spectral and temporal resolutions of the Advanced Baseline Imager (ABI) onboard the GOES-16 platform along with elevation information to improve the precipitation estimates. The procedure begins by first deriving a Rain/No Rain (R/NR) binary mask through classification of the pixels and then applying regression to estimate the amount of rainfall for rainy pixels. A Fully Convolutional Network is used as a regressor to predict precipitation estimates. The network is trained using the non-saturating conditional Generative Adversarial Network (cGAN) and Mean Squared Error (MSE) loss terms to generate results that better learn the complex distribution of precipitation in the observed data. Common verification metrics such as Probability Of Detection (POD), False Alarm Ratio (FAR), Critical Success Index (CSI), Bias, Correlation and MSE are used to evaluate the accuracy of both R/NR classification and real-valued precipitation estimates. Statistics and visualizations of the evaluation measures show improvements in the precipitation retrieval accuracy in the proposed framework compared to the baseline models trained using conventional MSE loss terms. This framework is proposed as an augmentation for PERSIANN-CCS (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network- Cloud Classification System) algorithm for estimating global precipitation.

scholarly journals A Reference-Guided Double Pipeline Face Image Completion Network

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1969
Author(s):  
Hongrui Liu ◽  
Shuoshi Li ◽  
Hongquan Wang ◽  
Xinshan Zhu
Keyword(s):  
Signal To Noise Ratio ◽  
Receptive Fields ◽  
Similarity Index ◽  
Face Image ◽  
Reference Image ◽  
Image Completion ◽  
Generative Adversarial Network ◽  
Identity Information ◽  
Adversarial Network ◽  
Reference Images

The existing face image completion approaches cannot be utilized to rationally complete damaged face images where their identity information is completely lost due to being obscured by center masks. Hence, in this paper, a reference-guided double-pipeline face image completion network (RG-DP-FICN) is designed within the framework of the generative adversarial network (GAN) completing the identity information of damaged images utilizing reference images with the same identity as damaged images. To reasonably integrate the identity information of reference images into completed images, the reference image is decoupled into identity features (e.g., the contour of eyes, eyebrows, nose) and pose features (e.g., the orientation of face and the positions of the facial features), and then the resulting identity features are fused with posture features of damaged images. Specifically, a lightweight identity predictor is used to extract the pose features; an identity extraction module is designed to compress and globally extract the identity features of the reference images, and an identity transfer module is proposed to effectively fuse identity and pose features by performing identity rendering on different receptive fields. Furthermore, quantitative and qualitative evaluations are conducted on a public dataset CelebA-HQ. Compared to the state-of-the-art methods, the evaluation metrics peak signal-to-noise ratio (PSNR), structure similarity index (SSIM) and L1 loss are improved by 2.22 dB, 0.033 and 0.79%, respectively. The results indicate that RG-DP-FICN can generate completed images with reasonable identity, with superior completion effect compared to existing completion approaches.

scholarly journals Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network

2021 ◽  
Vol 13 (8) ◽  
pp. 1512
Author(s):  
Quan Xiong ◽  
Liping Di ◽  
Quanlong Feng ◽  
Diyou Liu ◽  
Wei Liu ◽  
...  
Keyword(s):  
Land Surface ◽  
Near Infrared ◽  
Similarity Index ◽  
Data Availability ◽  
Optical Data ◽  
Generative Adversarial Network ◽  
Spectral Angle Mapper ◽  
Adversarial Network ◽  
Target Date ◽  
Sentinel 2

Sentinel-2 images have been widely used in studying land surface phenomena and processes, but they inevitably suffer from cloud contamination. To solve this critical optical data availability issue, it is ideal to fuse Sentinel-1 and Sentinel-2 images to create fused, cloud-free Sentinel-2-like images for facilitating land surface applications. In this paper, we propose a new data fusion model, the Multi-channels Conditional Generative Adversarial Network (MCcGAN), based on the conditional generative adversarial network, which is able to convert images from Domain A to Domain B. With the model, we were able to generate fused, cloud-free Sentinel-2-like images for a target date by using a pair of reference Sentinel-1/Sentinel-2 images and target-date Sentinel-1 images as inputs. In order to demonstrate the superiority of our method, we also compared it with other state-of-the-art methods using the same data. To make the evaluation more objective and reliable, we calculated the root-mean-square-error (RSME), R2, Kling–Gupta efficiency (KGE), structural similarity index (SSIM), spectral angle mapper (SAM), and peak signal-to-noise ratio (PSNR) of the simulated Sentinel-2 images generated by different methods. The results show that the simulated Sentinel-2 images generated by the MCcGAN have a higher quality and accuracy than those produced via the previous methods.

scholarly journals Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution

2021 ◽  
Vol 18 (12) ◽  
pp. 6216
Author(s):  
Mooseop Kim ◽  
YunKyung Park ◽  
KyeongDeok Moon ◽  
Chi Yoon Jeong
Keyword(s):  
Visual Information ◽  
Experimental Results ◽  
Auditory Sensitivity ◽  
Sensory Substitution ◽  
Sensitivity Analyses ◽  
Auditory Signal ◽  
Behavioral Experiments ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Proposed Model

Visual-auditory sensory substitution has demonstrated great potential to help visually impaired and blind groups to recognize objects and to perform basic navigational tasks. However, the high latency between visual information acquisition and auditory transduction may contribute to the lack of the successful adoption of such aid technologies in the blind community; thus far, substitution methods have remained only laboratory-scale research or pilot demonstrations. This high latency for data conversion leads to challenges in perceiving fast-moving objects or rapid environmental changes. To reduce this latency, prior analysis of auditory sensitivity is necessary. However, existing auditory sensitivity analyses are subjective because they were conducted using human behavioral analysis. Therefore, in this study, we propose a cross-modal generative adversarial network-based evaluation method to find an optimal auditory sensitivity to reduce transmission latency in visual-auditory sensory substitution, which is related to the perception of visual information. We further conducted a human-based assessment to evaluate the effectiveness of the proposed model-based analysis in human behavioral experiments. We conducted experiments with three participant groups, including sighted users (SU), congenitally blind (CB) and late-blind (LB) individuals. Experimental results from the proposed model showed that the temporal length of the auditory signal for sensory substitution could be reduced by 50%. This result indicates the possibility of improving the performance of the conventional vOICe method by up to two times. We confirmed that our experimental results are consistent with human assessment through behavioral experiments. Analyzing auditory sensitivity with deep learning models has the potential to improve the efficiency of sensory substitution.

scholarly journals Facial Inpainting Using Generative Adversarial Network with Feature Reconstruction and Landmark Loss to Preserve Spatial Consistency in Unaligned Face Images

2020 ◽  
Vol 13 (6) ◽  
pp. 219-228
Author(s):  
Avin Maulana ◽  
◽  
Chastine Fatichah ◽  
Nanik Suciati ◽  
◽  
...  
Keyword(s):  
Signal To Noise Ratio ◽  
Similarity Index ◽  
Perceptual Quality ◽  
Average Score ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Face Images ◽  
Spatial Consistency ◽  
Improvement Method ◽  
Feature Reconstruction

Facial inpainting is a process to reconstruct some missing or damaged pixels in the facial image. The reconstructed pixels should still be realistic, so the observer could not differentiate between the reconstructed pixels and the original one. However, there are a few problems that may arise when the inpainting algorithm has been done. There was an inconsistency between adjacent pixels when done on an unaligned face image, which caused a failure to reconstruct. We propose an improvement method in facial inpainting using Generative Adversarial Network (GAN) with additional loss using pre-trained network VGG-Net and face landmark. The feature reconstruction loss will help to preserve deep-feature on an image, while the landmark will increase the result’s perceptual quality. The training process has been done using a curriculum learning scenario. Qualitative results show that our inpainting method can reconstruct the missing area on unaligned face images. From the quantitative results, our proposed method achieves the average score of 21.528 and 0.665, while the maximum score of 29.922 and 0.908 on PSNR (Peak Signal to Noise Ratio) and SSIM (Structure Similarity Index Measure) metrics, respectively.

scholarly journals Synthesizing VERDICT maps from standard diffusion mp-MRI data using GANs

2021 ◽  
Author(s):  
Eleni Chiou ◽  
Vanya Valindria ◽  
Francesco Giganti ◽  
Shonit Punwani ◽  
Iasonas Kokkinos ◽  
...  
Keyword(s):  
Prostate Cancer ◽  
Mean Squared Error ◽  
Quantitative Estimation ◽  
Ground Truth ◽  
Quantitative Information ◽  
Strongly Correlated ◽  
Generative Adversarial Network ◽  
Mean Values ◽  
The Real ◽  
Adversarial Network

AbstractPurposeVERDICT maps have shown promising results in clinical settings discriminating normal from malignant tissue and identifying specific Gleason grades non-invasively. However, the quantitative estimation of VERDICT maps requires a specific diffusion-weighed imaging (DWI) acquisition. In this study we investigate the feasibility of synthesizing VERDICT maps from DWI data from multi-parametric (mp)-MRI which is widely used in clinical practice for prostate cancer diagnosis.MethodsWe use data from 67 patients who underwent both mp-MRI and VERDICT MRI. We compute the ground truth VERDICT maps from VERDICT MRI and we propose a generative adversarial network (GAN)-based approach to synthesize VERDICT maps from mp-MRI DWI data. We use correlation analysis and mean squared error to quantitatively evaluate the quality of the synthetic VERDICT maps compared to the real ones.ResultsQuantitative results show that the mean values of tumour areas in the synthetic and the real VERDICT maps were strongly correlated while qualitative results indicate that our method can generate realistic VERDICT maps from DWI from mp-MRI acquisitions.ConclusionRealistic VERDICT maps can be generated using DWI from standard mp-MRI. The synthetic maps preserve important quantitative information enabling the exploitation of VERDICT MRI for precise prostate cancer characterization with a single mp-MRI acquisition.

scholarly journals Low-Light Image Enhancement Based on Generative Adversarial Network

2021 ◽  
Vol 12 ◽  
Author(s):  
Nandhini Abirami R. ◽  
Durai Raj Vincent P. M.
Keyword(s):  
Image Enhancement ◽  
Visual Information ◽  
Low Light ◽  
Generative Adversarial Network ◽  
Light Image ◽  
Adversarial Network ◽  
Benchmark Datasets ◽  
Dim Light ◽  
Effective Network

Image enhancement is considered to be one of the complex tasks in image processing. When the images are captured under dim light, the quality of the images degrades due to low visibility degenerating the vision-based algorithms’ performance that is built for very good quality images with better visibility. After the emergence of a deep neural network number of methods has been put forward to improve images captured under low light. But, the results shown by existing low-light enhancement methods are not satisfactory because of the lack of effective network structures. A low-light image enhancement technique (LIMET) with a fine-tuned conditional generative adversarial network is presented in this paper. The proposed approach employs two discriminators to acquire a semantic meaning that imposes the obtained results to be realistic and natural. Finally, the proposed approach is evaluated with benchmark datasets. The experimental results highlight that the presented approach attains state-of-the-performance when compared to existing methods. The models’ performance is assessed using Visual Information Fidelitysse, which assesses the generated image’s quality over the degraded input. VIF obtained for different datasets using the proposed approach are 0.709123 for LIME dataset, 0.849982 for DICM dataset, 0.619342 for MEF dataset.

Sign in / Sign up

Export Citation Format

Share Document