A DenseUnet generative adversarial network for near-infrared face image colorization

Robotic drawing has become increasingly popular as an entertainment and interactive tool. In this paper we present RoboCoDraw, a real-time collaborative robot-based drawing system that draws stylized human face sketches interactively in front of human users, by using the Generative Adversarial Network (GAN)-based style transfer and a Random-Key Genetic Algorithm (RKGA)-based path optimization. The proposed RoboCoDraw system takes a real human face image as input, converts it to a stylized avatar, then draws it with a robotic arm. A core component in this system is the AvatarGAN proposed by us, which generates a cartoon avatar face image from a real human face. AvatarGAN is trained with unpaired face and avatar images only and can generate avatar images of much better likeness with human face images in comparison with the vanilla CycleGAN. After the avatar image is generated, it is fed to a line extraction algorithm and converted to sketches. An RKGA-based path optimization algorithm is applied to find a time-efficient robotic drawing path to be executed by the robotic arm. We demonstrate the capability of RoboCoDraw on various face images using a lightweight, safe collaborative robot UR5.

Download Full-text

Complete Face Recovering: An Approach towards Recognizing a Person by a Single Partial Face Image without the Target Photo in Gallery

10.36227/techrxiv.12333176.v1 ◽

2020 ◽

Author(s):

Yiu-ming Cheung ◽

Mengke Li

Keyword(s):

Promising Result ◽

Face Image ◽

Attractive Potential ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Discriminative Feature ◽

New Variant ◽

Potential Applications ◽

Benchmark Datasets

Complete face recovering (CFR) is to recover the complete face image of a given partial face image of a target person whose photo may not be included in the gallery set. The CFR has several attractive potential applications but is challenging. As far as we know, the CFR problem has yet to be explored in the literature. This paper therefore proposes an identity-preserved CFR approach (IP-CFR) to addressing the CFR. First, a denoising auto-encoder based network is applied to acquire the discriminative feature. Then, we propose an identity-preserved loss function to keep the personal identity information. Furthermore, the acquired features are fed into a new variant of the generative adversarial network (GAN) to restore the complete face image. In addition, a two-pathway discriminator is leveraged to enhance the quality of the recovered image. Experimental results on the benchmark datasets show the promising result of the proposed approach.

Download Full-text

Complete Face Recovering: An Approach towards Recognizing a Person by a Single Partial Face Image without the Target Photo in Gallery

10.36227/techrxiv.12333176 ◽

2020 ◽

Author(s):

Yiu-ming Cheung ◽

Mengke Li

Keyword(s):

Promising Result ◽

Face Image ◽

Attractive Potential ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Discriminative Feature ◽

New Variant ◽

Potential Applications ◽

Benchmark Datasets

Complete face recovering (CFR) is to recover the complete face image of a given partial face image of a target person whose photo may not be included in the gallery set. The CFR has several attractive potential applications but is challenging. As far as we know, the CFR problem has yet to be explored in the literature. This paper therefore proposes an identity-preserved CFR approach (IP-CFR) to addressing the CFR. First, a denoising auto-encoder based network is applied to acquire the discriminative feature. Then, we propose an identity-preserved loss function to keep the personal identity information. Furthermore, the acquired features are fed into a new variant of the generative adversarial network (GAN) to restore the complete face image. In addition, a two-pathway discriminator is leveraged to enhance the quality of the recovered image. Experimental results on the benchmark datasets show the promising result of the proposed approach.

Download Full-text

Robust Image Colorization Using Self Attention Based Progressive Generative Adversarial Network

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) ◽

10.1109/cvprw.2019.00272 ◽

2019 ◽

Cited By ~ 2

Author(s):

Manoj Sharma ◽

Megh Makwana ◽

Avinash Upadhyay ◽

Ajay Pratap Singh ◽

Anuj Badhwar ◽

...

Keyword(s):

Generative Adversarial Network ◽

Image Colorization ◽

Adversarial Network ◽

Robust Image

Download Full-text

A Reference-Guided Double Pipeline Face Image Completion Network

Electronics ◽

10.3390/electronics9111969 ◽

2020 ◽

Vol 9 (11) ◽

pp. 1969

Author(s):

Hongrui Liu ◽

Shuoshi Li ◽

Hongquan Wang ◽

Xinshan Zhu

Keyword(s):

Signal To Noise Ratio ◽

Receptive Fields ◽

Similarity Index ◽

Face Image ◽

Reference Image ◽

Image Completion ◽

Generative Adversarial Network ◽

Identity Information ◽

Adversarial Network ◽

Reference Images

The existing face image completion approaches cannot be utilized to rationally complete damaged face images where their identity information is completely lost due to being obscured by center masks. Hence, in this paper, a reference-guided double-pipeline face image completion network (RG-DP-FICN) is designed within the framework of the generative adversarial network (GAN) completing the identity information of damaged images utilizing reference images with the same identity as damaged images. To reasonably integrate the identity information of reference images into completed images, the reference image is decoupled into identity features (e.g., the contour of eyes, eyebrows, nose) and pose features (e.g., the orientation of face and the positions of the facial features), and then the resulting identity features are fused with posture features of damaged images. Specifically, a lightweight identity predictor is used to extract the pose features; an identity extraction module is designed to compress and globally extract the identity features of the reference images, and an identity transfer module is proposed to effectively fuse identity and pose features by performing identity rendering on different receptive fields. Furthermore, quantitative and qualitative evaluations are conducted on a public dataset CelebA-HQ. Compared to the state-of-the-art methods, the evaluation metrics peak signal-to-noise ratio (PSNR), structure similarity index (SSIM) and L1 loss are improved by 2.22 dB, 0.033 and 0.79%, respectively. The results indicate that RG-DP-FICN can generate completed images with reasonable identity, with superior completion effect compared to existing completion approaches.

Download Full-text

Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network

Remote Sensing ◽

10.3390/rs13081512 ◽

2021 ◽

Vol 13 (8) ◽

pp. 1512

Author(s):

Quan Xiong ◽

Liping Di ◽

Quanlong Feng ◽

Diyou Liu ◽

Wei Liu ◽

...

Keyword(s):

Land Surface ◽

Near Infrared ◽

Similarity Index ◽

Data Availability ◽

Optical Data ◽

Generative Adversarial Network ◽

Spectral Angle Mapper ◽

Adversarial Network ◽

Target Date ◽

Sentinel 2

Sentinel-2 images have been widely used in studying land surface phenomena and processes, but they inevitably suffer from cloud contamination. To solve this critical optical data availability issue, it is ideal to fuse Sentinel-1 and Sentinel-2 images to create fused, cloud-free Sentinel-2-like images for facilitating land surface applications. In this paper, we propose a new data fusion model, the Multi-channels Conditional Generative Adversarial Network (MCcGAN), based on the conditional generative adversarial network, which is able to convert images from Domain A to Domain B. With the model, we were able to generate fused, cloud-free Sentinel-2-like images for a target date by using a pair of reference Sentinel-1/Sentinel-2 images and target-date Sentinel-1 images as inputs. In order to demonstrate the superiority of our method, we also compared it with other state-of-the-art methods using the same data. To make the evaluation more objective and reliable, we calculated the root-mean-square-error (RSME), R2, Kling–Gupta efficiency (KGE), structural similarity index (SSIM), spectral angle mapper (SAM), and peak signal-to-noise ratio (PSNR) of the simulated Sentinel-2 images generated by different methods. The results show that the simulated Sentinel-2 images generated by the MCcGAN have a higher quality and accuracy than those produced via the previous methods.

Download Full-text

Realistic Face Image Generation Based on Generative Adversarial Network

2019 16th International Computer Conference on Wavelet Active Media Technology and Information Processing ◽

10.1109/iccwamtip47768.2019.9067742 ◽

2019 ◽

Author(s):

TING ZHANG ◽

WEN-HONG TIAN ◽

TING-YING ZHENG ◽

ZU-NING LI ◽

XUE-MEI DU ◽

...

Keyword(s):

Face Image ◽

Image Generation ◽

Generative Adversarial Network ◽

Adversarial Network

Download Full-text

Low Resolution Face Recognition Using Generative Adversarial Network (GAN)

10.21203/rs.3.rs-36971/v2 ◽

2020 ◽

Author(s):

Howard Martin ◽

Suharjito

Keyword(s):

Face Recognition ◽

Super Resolution ◽

Image Data ◽

Face Image ◽

Image Resolution ◽

Low Resolution ◽

Generative Adversarial Network ◽

Resolution Image ◽

Surveillance Camera ◽

Adversarial Network

Abstract Face recognition has a lot of use on smartphone authentication, finding people, etc. Nowadays, face recognition with a constrained environment has achieved very good performance on accuracy. However, the accuracy of existing face recognition methods will gradually decrease when using a dataset with an unconstrained environment. Face image with an unconstrained environment is usually taken from a surveillance camera. In general, surveillance cameras will be placed on the corner of a room or even on the street. So, the image resolution will be low. Low-resolution image will cause the face very hard to be recognized and the accuracy will eventually decrease. That is the main reason why increasing the accuracy of the Low-Resolution Face Recognition (LRFR) problem is still challenging. This research aimed to solve the Low-Resolution Face Recognition (LRFR) problem. The datasets are YouTube Faces Database (YTF) and Labelled Faces in The Wild (LFW). In this research, face image resolution would be decreased using bicubic linear and became the low-resolution image data. Then super resolution methods as the preprocessing step would increase the image resolution. Super resolution methods used in this research are Super resolution GAN (SRGAN) [1] and Enhanced Super resolution GAN (ESRGAN) [2]. These methods would be compared to reach a better accuracy on solving LRFR problem. After increased the image resolution, the image would be recognized using FaceNet. This research concluded that using super resolution as the preprocessing step for LRFR problem has achieved a higher accuracy compared to [3]. The highest accuracy achieved by using ESRGAN as the preprocessing and FaceNet for face recognition with accuracy of 98.96 % and Validation rate 96.757 %.

Download Full-text