scholarly journals A Reference-Guided Double Pipeline Face Image Completion Network

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1969
Author(s):  
Hongrui Liu ◽  
Shuoshi Li ◽  
Hongquan Wang ◽  
Xinshan Zhu
Keyword(s):  
Signal To Noise Ratio ◽  
Receptive Fields ◽  
Similarity Index ◽  
Face Image ◽  
Reference Image ◽  
Image Completion ◽  
Generative Adversarial Network ◽  
Identity Information ◽  
Adversarial Network ◽  
Reference Images

The existing face image completion approaches cannot be utilized to rationally complete damaged face images where their identity information is completely lost due to being obscured by center masks. Hence, in this paper, a reference-guided double-pipeline face image completion network (RG-DP-FICN) is designed within the framework of the generative adversarial network (GAN) completing the identity information of damaged images utilizing reference images with the same identity as damaged images. To reasonably integrate the identity information of reference images into completed images, the reference image is decoupled into identity features (e.g., the contour of eyes, eyebrows, nose) and pose features (e.g., the orientation of face and the positions of the facial features), and then the resulting identity features are fused with posture features of damaged images. Specifically, a lightweight identity predictor is used to extract the pose features; an identity extraction module is designed to compress and globally extract the identity features of the reference images, and an identity transfer module is proposed to effectively fuse identity and pose features by performing identity rendering on different receptive fields. Furthermore, quantitative and qualitative evaluations are conducted on a public dataset CelebA-HQ. Compared to the state-of-the-art methods, the evaluation metrics peak signal-to-noise ratio (PSNR), structure similarity index (SSIM) and L1 loss are improved by 2.22 dB, 0.033 and 0.79%, respectively. The results indicate that RG-DP-FICN can generate completed images with reasonable identity, with superior completion effect compared to existing completion approaches.

scholarly journals Facial Inpainting Using Generative Adversarial Network with Feature Reconstruction and Landmark Loss to Preserve Spatial Consistency in Unaligned Face Images

2020 ◽  
Vol 13 (6) ◽  
pp. 219-228
Author(s):  
Avin Maulana ◽  
◽  
Chastine Fatichah ◽  
Nanik Suciati ◽  
◽  
...  
Keyword(s):  
Signal To Noise Ratio ◽  
Similarity Index ◽  
Perceptual Quality ◽  
Average Score ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Face Images ◽  
Spatial Consistency ◽  
Improvement Method ◽  
Feature Reconstruction

Facial inpainting is a process to reconstruct some missing or damaged pixels in the facial image. The reconstructed pixels should still be realistic, so the observer could not differentiate between the reconstructed pixels and the original one. However, there are a few problems that may arise when the inpainting algorithm has been done. There was an inconsistency between adjacent pixels when done on an unaligned face image, which caused a failure to reconstruct. We propose an improvement method in facial inpainting using Generative Adversarial Network (GAN) with additional loss using pre-trained network VGG-Net and face landmark. The feature reconstruction loss will help to preserve deep-feature on an image, while the landmark will increase the result’s perceptual quality. The training process has been done using a curriculum learning scenario. Qualitative results show that our inpainting method can reconstruct the missing area on unaligned face images. From the quantitative results, our proposed method achieves the average score of 21.528 and 0.665, while the maximum score of 29.922 and 0.908 on PSNR (Peak Signal to Noise Ratio) and SSIM (Structure Similarity Index Measure) metrics, respectively.

scholarly journals An Enhanced pix2pix Dehazing Network with Guided Filter Layer

2020 ◽  
Vol 10 (17) ◽  
pp. 5898
Author(s):  
Qirong Bu ◽  
Jie Luo ◽  
Kuan Ma ◽  
Hongwei Feng ◽  
Jun Feng
Keyword(s):  
Signal To Noise Ratio ◽  
Similarity Index ◽  
Structural Similarity ◽  
Comparison Method ◽  
Guided Filter ◽  
Generative Adversarial Network ◽  
Test Dataset ◽  
Adversarial Network ◽  
Filter Layer

In this paper, we propose an enhanced pix2pix dehazing network, which generates clear images without relying on a physical scattering model. This network is a generative adversarial network (GAN) which combines multiple guided filter layers. First, the input of hazy images is smoothed to obtain high-frequency features according to different smoothing kernels of the guided filter layer. Then, these features are embedded in higher dimensions of the network and connected with the output of the generator’s encoder. Finally, Visual Geometry Group (VGG) features are introduced to serve as a loss function to improve the quality of the texture information restoration and generate better hazy-free images. We conduct experiments on NYU-Depth, I-HAZE and O-HAZE datasets. The enhanced pix2pix dehazing network we propose produces increases of 1.22 dB in the Peak Signal-to-Noise Ratio (PSNR) and 0.01 in the Structural Similarity Index Metric (SSIM) compared with a second successful comparison method using the indoor test dataset. Extensive experiments demonstrate that the proposed method has good performance for image dehazing.

scholarly journals RoboCoDraw: Robotic Avatar Drawing with GAN-Based Style Transfer and Time-Efficient Path Optimization

2020 ◽  
Vol 34 (06) ◽  
pp. 10402-10409
Author(s):  
Tianying Wang ◽  
Wei Qi Toh ◽  
Hao Zhang ◽  
Xiuchao Sui ◽  
Shaohua Li ◽  
...  
Keyword(s):  
Face Image ◽  
Path Optimization ◽  
Robotic Arm ◽  
Human Face ◽  
Generative Adversarial Network ◽  
Style Transfer ◽  
Adversarial Network ◽  
Face Images ◽  
Drawing System ◽  
Collaborative Robot

Robotic drawing has become increasingly popular as an entertainment and interactive tool. In this paper we present RoboCoDraw, a real-time collaborative robot-based drawing system that draws stylized human face sketches interactively in front of human users, by using the Generative Adversarial Network (GAN)-based style transfer and a Random-Key Genetic Algorithm (RKGA)-based path optimization. The proposed RoboCoDraw system takes a real human face image as input, converts it to a stylized avatar, then draws it with a robotic arm. A core component in this system is the AvatarGAN proposed by us, which generates a cartoon avatar face image from a real human face. AvatarGAN is trained with unpaired face and avatar images only and can generate avatar images of much better likeness with human face images in comparison with the vanilla CycleGAN. After the avatar image is generated, it is fed to a line extraction algorithm and converted to sketches. An RKGA-based path optimization algorithm is applied to find a time-efficient robotic drawing path to be executed by the robotic arm. We demonstrate the capability of RoboCoDraw on various face images using a lightweight, safe collaborative robot UR5.

scholarly journals Shuffle block SRGAN for face image super-resolution reconstruction

2020 ◽  
Vol 53 (7-8) ◽  
pp. 1429-1439
Author(s):  
Ziwei Zhang ◽  
Yangjing Shi ◽  
Xiaoshi Zhou ◽  
Hongfei Kan ◽  
Juan Wen
Keyword(s):  
Face Recognition ◽  
Learning Strategy ◽  
Signal To Noise Ratio ◽  
Similarity Index ◽  
Super Resolution ◽  
Face Image ◽  
Low Resolution ◽  
Index Method ◽  
Face Features ◽  
Image Super Resolution

When low-resolution face images are used for face recognition, the model accuracy is substantially decreased. How to recover high-resolution face features from low-resolution images precisely and efficiently is an essential subtask in face recognition. In this study, we introduce shuffle block SRGAN, a new image super-resolution network inspired by the SRGAN structure. By replacing the residual blocks with shuffle blocks, we can achieve efficient super-resolution reconstruction. Furthermore, by considering the generated image quality in the loss function, we can obtain more realistic super-resolution images. We train and test SB-SRGAN in three public face image datasets and use transfer learning strategy during the training process. The experimental results show that shuffle block SRGAN can achieve desirable image super-resolution performance with respect to visual effect as well as the peak signal-to-noise ratio and structure similarity index method metrics, compared with the performance attained by the other chosen deep-leaning models.

scholarly journals No-Reference Image Quality Assessment Based on Multi-Task Generative Adversarial Network

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 146893-146902 ◽  
Author(s):  
Yao Ma ◽  
Xibiao Cai ◽  
Fuming Sun ◽  
Shijie Hao
Keyword(s):  
Image Quality ◽  
Quality Assessment ◽  
Image Quality Assessment ◽  
Reference Image ◽  
Generative Adversarial Network ◽  
Adversarial Network

scholarly journals Single Image Super-Resolution: Depthwise Separable Convolution Super-Resolution Generative Adversarial Network

2020 ◽  
Vol 10 (1) ◽  
pp. 375 ◽  
Author(s):  
Zetao Jiang ◽  
Yongsong Huang ◽  
Lirui Hu
Keyword(s):  
Similarity Index ◽  
Super Resolution ◽  
Structural Similarity ◽  
Image Features ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Visual Evaluation ◽  
Adversarial Network ◽  
Image Super Resolution ◽  
Single Image Super Resolution

The super-resolution generative adversarial network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied by unpleasant artifacts. To further enhance the visual quality, we propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The method is based on depthwise separable convolution super-resolution generative adversarial network (DSCSRGAN). A new depthwise separable convolution dense block (DSC Dense Block) was designed for the generator network, which improved the ability to represent and extract image features, while greatly reducing the total amount of parameters. For the discriminator network, the batch normalization (BN) layer was discarded, and the problem of artifacts was reduced. A frequency energy similarity loss function was designed to constrain the generator network to generate better super-resolution images. Experiments on several different datasets showed that the peak signal-to-noise ratio (PSNR) was improved by more than 3 dB, structural similarity index (SSIM) was increased by 16%, and the total parameter was reduced to 42.8% compared with the original model. Combining various objective indicators and subjective visual evaluation, the algorithm was shown to generate richer image details, clearer texture, and lower complexity.

scholarly journals Single Image Defogging Algorithm Based on Conditional Generative Adversarial Network

2020 ◽  
Vol 2020 ◽  
pp. 1-8
Author(s):  
Rui-Qiang Ma ◽  
Xing-Run Shen ◽  
Shan-Jun Zhang
Keyword(s):  
Signal To Noise Ratio ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Visual Effect ◽  
Low Contrast ◽  
Attention Model ◽  
Adversarial Network ◽  
Dark Channel ◽  
Network Generator ◽  
Bright Area

Outside the house, images taken using a phone in foggy weather are not suitable for automation due to low contrast. Usually, it is revised in the dark channel prior (DCP) method (K. He et al. 2009), but the non-sky bright area exists due to mistakes in the removal. In this paper, we propose an algorithm, defog-based generative adversarial network (DbGAN). We use generative adversarial network (GAN) for training and embed target map (TM) in the anti-network generator, only the part of bright area layer of image, in local attention model image training and testing in deep learning, and the effective processing of the wrong removal part is achieved, thus better restoring the defog image. Then, the DCP method obtains a good defog visual effect, and the evaluation index peak signal-to-noise ratio (PSNR) is used to make a judgment; the simulation result is consistent with the visual effect. We proved the DbGAN is a practical import of target map in the GAN. The algorithm is used defogging in the highlighted area is well realized, which makes up for the shortcomings of the DCP algorithm.

scholarly journals High-fidelity fast volumetric brain MRI using a hybrid denoising generative adversarial network (HDnGAN)

2021 ◽  
Author(s):  
Ziyu Li ◽  
Qiyuan Tian ◽  
Chanon Ngamsombat ◽  
Samuel Cartmell ◽  
John Conklin ◽  
...  
Keyword(s):  
Parallel Imaging ◽  
Mean Squared Error ◽  
Brain Mri ◽  
Similarity Index ◽  
Image Synthesis ◽  
Image Sharpness ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Significant Difference ◽  
Multiple Sclerosis Patients

Purpose: To improve the signal-to-noise ratio (SNR) of highly accelerated volumetric MRI while preserve realistic textures using a generative adversarial network (GAN). Methods: A hybrid GAN for denoising entitled "HDnGAN" with a 3D generator and a 2D discriminator was proposed to denoise 3D T2-weighted fluid-attenuated inversion recovery (FLAIR) images acquired in 2.75 minutes (R=3×2) using wave-controlled aliasing in parallel imaging (Wave-CAIPI). HDnGAN was trained on data from 25 multiple sclerosis patients by minimizing a combined mean squared error and adversarial loss with adjustable weight λ. Results were evaluated on eight separate patients by comparing to standard T2-SPACE FLAIR images acquired in 7.25 minutes (R=2×2) using mean absolute error (MAE), peak SNR (PSNR), structural similarity index (SSIM), and VGG perceptual loss, and by two neuroradiologists using a five-point score regarding gray-white matter contrast, sharpness, SNR, lesion conspicuity, and overall quality. Results: HDnGAN (λ=0) produced the lowest MAE, highest PSNR and SSIM. HDnGAN (λ=10-3) produced the lowest VGG loss. In the reader study, HDnGAN (λ=10-3) significantly improved the gray-white contrast and SNR of Wave-CAIPI images, and outperformed BM4D and HDnGAN (λ=0) regarding image sharpness. The overall quality score from HDnGAN (λ=10-3) was significantly higher than those from Wave-CAIPI, BM4D, and HDnGAN (λ=0), with no significant difference compared to standard images. Conclusion: HDnGAN concurrently benefits from improved image synthesis performance of 3D convolution and increased training samples for training the 2D discriminator on limited data. HDnGAN generates images with high SNR and realistic textures, similar to those acquired in longer times and preferred by neuroradiologists.

scholarly journals Complete Face Recovering: An Approach towards Recognizing a Person by a Single Partial Face Image without the Target Photo in Gallery

2020 ◽  
Author(s):  
Yiu-ming Cheung ◽  
Mengke Li
Keyword(s):  
Promising Result ◽  
Face Image ◽  
Attractive Potential ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Discriminative Feature ◽  
New Variant ◽  
Potential Applications ◽  
Benchmark Datasets

Complete face recovering (CFR) is to recover the complete face image of a given partial face image of a target person whose photo may not be included in the gallery set. The CFR has several attractive potential applications but is challenging. As far as we know, the CFR problem has yet to be explored in the literature. This paper therefore proposes an identity-preserved CFR approach (IP-CFR) to addressing the CFR. First, a denoising auto-encoder based network is applied to acquire the discriminative feature. Then, we propose an identity-preserved loss function to keep the personal identity information. Furthermore, the acquired features are fed into a new variant of the generative adversarial network (GAN) to restore the complete face image. In addition, a two-pathway discriminator is leveraged to enhance the quality of the recovered image. Experimental results on the benchmark datasets show the promising result of the proposed approach.

scholarly journals Complete Face Recovering: An Approach towards Recognizing a Person by a Single Partial Face Image without the Target Photo in Gallery

2020 ◽  
Author(s):  
Yiu-ming Cheung ◽  
Mengke Li
Keyword(s):  
Promising Result ◽  
Face Image ◽  
Attractive Potential ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Discriminative Feature ◽  
New Variant ◽  
Potential Applications ◽  
Benchmark Datasets

Complete face recovering (CFR) is to recover the complete face image of a given partial face image of a target person whose photo may not be included in the gallery set. The CFR has several attractive potential applications but is challenging. As far as we know, the CFR problem has yet to be explored in the literature. This paper therefore proposes an identity-preserved CFR approach (IP-CFR) to addressing the CFR. First, a denoising auto-encoder based network is applied to acquire the discriminative feature. Then, we propose an identity-preserved loss function to keep the personal identity information. Furthermore, the acquired features are fed into a new variant of the generative adversarial network (GAN) to restore the complete face image. In addition, a two-pathway discriminator is leveraged to enhance the quality of the recovered image. Experimental results on the benchmark datasets show the promising result of the proposed approach.

Sign in / Sign up

Export Citation Format

Share Document