scholarly journals Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement

2021 ◽  
Author(s):  
Huiwen Luo ◽  
Koki Nagano ◽  
Han-Wei Kung ◽  
Mclean Goldwhite
Keyword(s):  
Cutting Edge ◽  
Superior Performance ◽  
3D Face Reconstruction ◽  
High Quality ◽  
Ground Truth Data ◽  
Lighting Condition ◽  
3D Face ◽  
Lighting Conditions ◽  
Reconstruction Methods ◽  
Non Linear

We introduce a highly robust GAN-based framework for digitizing a normalized 3D avatar of a person from a single unconstrained photo. While the input image can be of a smiling person or taken in extreme lighting conditions, our method can reliably produce a high-quality textured model of a person's face in neutral expression and skin textures under diffuse lighting condition. Cutting-edge 3D face reconstruction methods use non-linear morphable face models combined with GAN-based decoders to capture the likeness and details of a person but fail to produce neutral head models with unshaded albedo textures which is critical for creating relightable and animation-friendly avatars for integration in virtual environments. The key challenges for existing methods to work is the lack of training and ground truth data containing normalized 3D faces. We propose a two-stage approach to address this problem. First, we adopt a highly robust normalized 3D face generator by embedding a non-linear morphable face model into a StyleGAN2 network. This allows us to generate detailed but normalized facial assets. This inference is then followed by a perceptual refinement step that uses the generated assets as regularization to cope with the limited available training samples of normalized faces. We further introduce a Normalized Face Dataset, which consists of a combination photogrammetry scans, carefully selected photographs, and generated fake people with neutral expressions in diffuse lighting conditions. While our prepared dataset contains two orders of magnitude less subjects than cutting edge GAN-based 3D facial reconstruction methods, we show that it is possible to produce high-quality normalized face models for very challenging unconstrained input images, and demonstrate superior performance to the current state-of-the-art.

IMAGE BASED 3D FACE RECONSTRUCTION: A SURVEY

2009 ◽  
Vol 09 (02) ◽  
pp. 217-250 ◽  
Author(s):  
GEORGIOS STYLIANOU ◽  
ANDREAS LANITIS
Keyword(s):  
3D Reconstruction ◽  
Feature Detection ◽  
Low Cost ◽  
Reconstruction Algorithms ◽  
3D Face Reconstruction ◽  
Reconstruction Process ◽  
3D Face ◽  
Advantages And Disadvantages ◽  
Face Reconstruction ◽  
Reconstruction Methods

The use of 3D data in face image processing applications has received considerable attention during the last few years. A major issue for the implementation of 3D face processing systems is the accurate and real time acquisition of 3D faces using low cost equipment. In this paper we provide a survey of 3D reconstruction methods used for generating the 3D appearance of a face using either a single or multiple 2D images captured with ordinary equipment such as digital cameras and camcorders. In this context we discuss various issues pertaining to the general problem of 3D face reconstruction such as the existence of suitable 3D face databases, correspondence of 3D faces, feature detection, deformable 3D models and typical assumptions used during the reconstruction process. Different approaches to the problem of 3D reconstruction are presented and for each category the most important advantages and disadvantages are outlined. In particular we describe example-based methods, stereo methods, video-based methods and silhouette-based methods. The issue of performance evaluation of 3D face reconstruction algorithms, the state of the art and future trends are also discussed.

NONITERATIVE 3D FACE RECONSTRUCTION BASED ON PHOTOMETRIC STEREO

2008 ◽  
Vol 22 (03) ◽  
pp. 389-410 ◽  
Author(s):  
SANG-WOONG LEE ◽  
PATRICK S. P. WANG ◽  
SVETLANA N. YANUSHKEVICH ◽  
SEONG-WHAN LEE
Keyword(s):  
Photometric Stereo ◽  
Face Database ◽  
3D Face Reconstruction ◽  
3D Face ◽  
3D Shape ◽  
Face Reconstruction ◽  
Lighting Conditions ◽  
3D Shape Reconstruction ◽  
3D Shapes ◽  
2D Images

3D face reconstruction is a popular area within the computer vision domain. 3D face reconstruction should ideally be achieved easily and cost-effectively, without requiring specialized equipment to estimate 3D shapes. As a result of this, many techniques for retrieving 3D shapes from 2D images have been proposed. In this paper, a novel method for 3D face reconstruction based on photometric stereo, which estimates the surface normal from shading information in multiple images, hence recovering the 3D shape of a face, is proposed. In order to overcome the problems of previous approaches related to prior-knowledge regarding lighting conditions and iterative algorithms, the exemplar is synthesized with known lighting conditions from at least three images, under arbitrary lighting conditions and using an illumination reference. Experiments in 3D face reconstruction were made by verifying the proposed approach using the illumination subset of the Max-Planck Institute face database and Yale face database B. Experimental results demonstrate that the proposed method is effective for 3D shape reconstruction of faces from 2D images.

scholarly journals Joint 3D facial shape reconstruction and texture completion from a single image

2021 ◽  
Vol 8 (2) ◽  
pp. 239-256
Author(s):  
Xiaoxing Zeng ◽  
Zhelun Wu ◽  
Xiaojiang Peng ◽  
Yu Qiao
Keyword(s):  
Shape Reconstruction ◽  
Input Image ◽  
Deep Convolutional Neural Networks ◽  
3D Face Reconstruction ◽  
3D Face ◽  
3D Texture ◽  
Reconstruction Methods ◽  
Facial Shape ◽  
In The Wild ◽  
Geometric Cues

AbstractRecent years have witnessed significant progress in image-based 3D face reconstruction using deep convolutional neural networks. However, current reconstruction methods often perform improperly in self-occluded regions and can lead to inaccurate correspondences between a 2D input image and a 3D face template, hindering use in real applications. To address these problems, we propose a deep shape reconstruction and texture completion network, SRTC-Net, which jointly reconstructs 3D facial geometry and completes texture with correspondences from a single input face image. In SRTC-Net, we leverage the geometric cues from completed 3D texture to reconstruct detailed structures of 3D shapes. The SRTC-Net pipeline has three stages. The first introduces a correspondence network to identify pixel-wise correspondence between the input 2D image and a 3D template model, and transfers the input 2D image to a U-V texture map. Then we complete the invisible and occluded areas in the U-V texture map using an inpainting network. To get the 3D facial geometries, we predict coarse shape (U-V position maps) from the segmented face from the correspondence network using a shape network, and then refine the 3D coarse shape by regressing the U-V displacement map from the completed U-V texture map in a pixel-to-pixel way. We examine our methods on 3D reconstruction tasks as well as face frontalization and pose invariant face recognition tasks, using both in-the-lab datasets (MICC, MultiPIE) and in-the-wild datasets (CFP). The qualitative and quantitative results demonstrate the effectiveness of our methods on inferring 3D facial geometry and complete texture; they outperform or are comparable to the state-of-the-art.

scholarly journals PCDRN: Progressive Cascade Deep Residual Network for Pansharpening

2020 ◽  
Vol 12 (4) ◽  
pp. 676 ◽  
Author(s):  
Yong Yang ◽  
Wei Tu ◽  
Shuying Huang ◽  
Hangyuan Lu
Keyword(s):  
High Resolution ◽  
Loss Function ◽  
High Frequency ◽  
Experimental Results ◽  
Superior Performance ◽  
Residual Network ◽  
High Quality ◽  
Convolution Approach ◽  
Visual Assessments ◽  
Coarse To Fine

Pansharpening is the process of fusing a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image. In the process of pansharpening, the LRMS image is often directly upsampled by a scale of 4, which may result in the loss of high-frequency details in the fused high-resolution multispectral (HRMS) image. To solve this problem, we put forward a novel progressive cascade deep residual network (PCDRN) with two residual subnetworks for pansharpening. The network adjusts the size of an MS image to the size of a PAN image twice and gradually fuses the LRMS image with the PAN image in a coarse-to-fine manner. To prevent an overly-smooth phenomenon and achieve high-quality fusion results, a multitask loss function is defined to train our network. Furthermore, to eliminate checkerboard artifacts in the fusion results, we employ a resize-convolution approach instead of transposed convolution for upsampling LRMS images. Experimental results on the Pléiades and WorldView-3 datasets prove that PCDRN exhibits superior performance compared to other popular pansharpening methods in terms of quantitative and visual assessments.

scholarly journals Deeply Learned Pose Invariant Image Analysis with Applications in 3D Face Recognition

2019 ◽  
Vol 2019 ◽  
pp. 1-21 ◽  
Author(s):  
Naeem Ratyal ◽  
Imtiaz Ahmad Taj ◽  
Muhammad Sajid ◽  
Anzar Mahmood ◽  
Sohail Razzaq ◽  
...  
Keyword(s):  
Face Recognition ◽  
Superior Performance ◽  
Support Vector ◽  
Complex Nature ◽  
Alignment Algorithm ◽  
3D Face Recognition ◽  
3D Face ◽  
Norm Minimization ◽  
Wide Range ◽  
The Face

Face recognition aims to establish the identity of a person based on facial characteristics and is a challenging problem due to complex nature of the facial manifold. A wide range of face recognition applications are based on classification techniques and a class label is assigned to the test image that belongs to the unknown class. In this paper, a pose invariant deeply learned multiview 3D face recognition approach is proposed and aims to address two problems: face alignment and face recognition through identification and verification setups. The proposed alignment algorithm is capable of handling frontal as well as profile face images. It employs a nose tip heuristic based pose learning approach to estimate acquisition pose of the face followed by coarse to fine nose tip alignment using L2 norm minimization. The whole face is then aligned through transformation using knowledge learned from nose tip alignment. Inspired by the intrinsic facial symmetry of the Left Half Face (LHF) and Right Half Face (RHF), Deeply learned (d) Multi-View Average Half Face (d-MVAHF) features are employed for face identification using deep convolutional neural network (dCNN). For face verification d-MVAHF-Support Vector Machine (d-MVAHF-SVM) approach is employed. The performance of the proposed methodology is demonstrated through extensive experiments performed on four databases: GavabDB, Bosphorus, UMB-DB, and FRGC v2.0. The results show that the proposed approach yields superior performance as compared to existing state-of-the-art methods.

scholarly journals Experimental validation of new self-voltage balanced 9L-ANPC inverter for photovoltaic applications

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
M. Jagabar Sathik ◽  
Dhafer J. Almakhles ◽  
N. Sandeep ◽  
Marif Daula Siddique
Keyword(s):  
Output Voltage ◽  
Pulse Width Modulation ◽  
Superior Performance ◽  
Modulation Scheme ◽  
Switching Frequency ◽  
Switched Capacitor ◽  
Renewable Energy Resources ◽  
High Quality ◽  
Multilevel Inverters ◽  
Blocking Voltage

AbstractMultilevel inverters play an important role in extracting the power from renewable energy resources and delivering the output voltage with high quality to the load. This paper proposes a new single-stage switched capacitor nine-level inverter, which comprises an improved T-type inverter, auxiliary switch, and switched cell unit. The proposed topology effectively reduces the DC-link capacitor voltage and exhibits superior performance over recently switched-capacitor inverter topologies in terms of the number of power components and blocking voltage of the switches. A level-shifted multilevel pulse width modulation scheme with a modified triangular carrier wave is implemented to produce a high-quality stepped output voltage waveform with low switching frequency. The proposed nine-level inverter’s effectiveness, driven by the recommended modulation technique, is experimentally verified under varying load conditions. The power loss and efficiency for the proposed nine-level inverter are thoroughly discussed with different loads.

Sign in / Sign up

Export Citation Format

Share Document