Adaptive 3D Model-Based Facial Expression Synthesis and Pose Frontalization

Facial expressions are one of the important non-verbal ways used to understand human emotions during communication. Thus, acquiring and reproducing facial expressions is helpful in analyzing human emotional states. However, owing to complex and subtle facial muscle movements, facial expression modeling from images with face poses is difficult to achieve. To handle this issue, we present a method for acquiring facial expressions from a non-frontal single photograph using a 3D-aided approach. In addition, we propose a contour-fitting method that improves the modeling accuracy by automatically rearranging 3D contour landmarks corresponding to fixed 2D image landmarks. The acquired facial expression input can be parametrically manipulated to create various facial expressions through a blendshape or expression transfer based on the FACS (Facial Action Coding System). To achieve a realistic facial expression synthesis, we propose an exemplar-texture wrinkle synthesis method that extracts and synthesizes appropriate expression wrinkles according to the target expression. To do so, we constructed a wrinkle table of various facial expressions from 400 people. As one of the applications, we proved that the expression-pose synthesis method is suitable for expression-invariant face recognition through a quantitative evaluation, and showed the effectiveness based on a qualitative evaluation. We expect our system to be a benefit to various fields such as face recognition, HCI, and data augmentation for deep learning.

FACIAL METAMORPHOSIS USING GEOMETRICAL METHODS FOR BIOMETRIC APPLICATIONS

Facial expression modeling has been a popular topic in biometrics for many years. One of the emerging recent trends is capturing subtle details such as wrinkles, creases and minor imperfections that are highly important for biometric modeling as well as matching. In this paper, we suggest a novel approach to the problem of expression modeling and morphing based on a geometry-based paradigm. In 2D image space, a distance-based morphing system is utilized to create a line drawing style facial animation from two input images representing frontal and profile views of the face. Aging wrinkles and expression lines are extracted and mapped back to the synthesized facial NPR (nonphotorealistic) sketches. In 3D object space, we present a metamorphosis system that combines the traditional free-form deformation (FFD) model with data interpolation techniques based on the proximity preserving Voronoi diagram. With feature points selected from two images of the target face, the proposed system generates the 3D target facial model by transforming a generic model. Experimental results demonstrate that morphing sequences generated by our systems are of convincing quality.