High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies

We present a fully automatic system that can produce high-fidelity, photo-realistic three-dimensional (3D) digital human heads with a consumer RGB-D selfie camera. The system only needs the user to take a short selfie RGB-D video while rotating his/her head and can produce a high-quality head reconstruction in less than 30 s. Our main contribution is a new facial geometry modeling and reflectance synthesis procedure that significantly improves the state of the art. Specifically, given the input video a two-stage frame selection procedure is first employed to select a few high-quality frames for reconstruction. Then a differentiable renderer-based 3D Morphable Model (3DMM) fitting algorithm is applied to recover facial geometries from multiview RGB-D data, which takes advantages of a powerful 3DMM basis constructed with extensive data generation and perturbation. Our 3DMM has much larger expressive capacities than conventional 3DMM, allowing us to recover more accurate facial geometry using merely linear basis. For reflectance synthesis, we present a hybrid approach that combines parametric fitting and Convolutional Neural Networks (CNNs) to synthesize high-resolution albedo/normal maps with realistic hair/pore/wrinkle details. Results show that our system can produce faithful 3D digital human faces with extremely realistic details. The main code and the newly constructed 3DMM basis is publicly available.

Multi-Frequency Image Completion via a Biologically-Inspired Sub-Riemannian Model with Frequency and Phase

We present a novel cortically-inspired image completion algorithm. It uses five-dimensional sub-Riemannian cortical geometry, modeling the orientation, spatial frequency and phase-selective behavior of the cells in the visual cortex. The algorithm extracts the orientation, frequency and phase information existing in a given two-dimensional corrupted input image via a Gabor transform and represents those values in terms of cortical cell output responses in the model geometry. Then, it performs completion via a diffusion concentrated in a neighborhood along the neural connections within the model geometry. The diffusion models the activity propagation integrating orientation, frequency and phase features along the neural connections. Finally, the algorithm transforms the diffused and completed output responses back to the two-dimensional image plane.

A Robust Rigid Registration Framework of 3D Indoor Scene Point Clouds Based on RGB-D Information

Rigid registration of 3D indoor scenes is a fundamental yet vital task in various fields that include remote sensing (e.g., 3D reconstruction of indoor scenes), photogrammetry measurement, geometry modeling, etc. Nevertheless, state-of-the-art registration approaches still have defects when dealing with low-quality indoor scene point clouds derived from consumer-grade RGB-D sensors. The major challenge is accurately extracting correspondences between a pair of low-quality point clouds when they contain considerable noise, outliers, or weak texture features. To solve the problem, we present a point cloud registration framework in view of RGB-D information. First, we propose a point normal filter for effectively removing noise and simultaneously maintaining sharp geometric features and smooth transition regions. Second, we design a correspondence extraction scheme based on a novel descriptor encoding textural and geometry information, which can robustly establish dense correspondences between a pair of low-quality point clouds. Finally, we propose a point-to-plane registration technology via a nonconvex regularizer, which can further diminish the influence of those false correspondences and produce an exact rigid transformation between a pair of point clouds. Compared to existing state-of-the-art techniques, intensive experimental results demonstrate that our registration framework is excellent visually and numerically, especially for dealing with low-quality indoor scenes.

A 3D nano scale IGA for free vibration and buckling analyses of multi-directional FGM nanoshells

In this article, we explore a three-dimensional solid isogeometric analysis (3D-IGA) approach based on a nonlocal elasticity theory to investigate size effects on natural frequency and critical buckling load for multi-directional functionally graded (FG) nanoshells. The multi-directional FG material uses a power law rule with three power exponent indexes concerning three parametric coordinates. Nanoshell's geometries include the square plate, cylindrical and spherical panels with the side length considered in a nanoscale. Because 3D-IGA utilizes an approximation of NURBS basic functions to integrate from geometry modeling to discretized domain, it is the best promising candidate to fulfill a higher-order derivative requirement of the nonlocal theory on nanoshells. The numerical solutions are verified by those published in several pieces of literature to determine the current approach's accuracy and reliability. After a convergence solution is examined, a quartic NURBS basic function can yield ultra-converged and high-accurate results with a low computational cost. The findings show the size effect parameters which significantly impact the frequencies and the critical buckling factors of the multi-directional FG nanoshells. Generally, increases in the size effect parameters will cause declines in the frequencies and the critical buckling factors of the nanoshells.

An improved Loop subdivision to coordinate the smoothness and the number of faces via multi-objective optimization

3D mesh subdivision is essential for geometry modeling of complex surfaces, which benefits many important applications in the fields of multimedia such as computer animation. However, in the ordinary adaptive subdivision, with the deepening of the subdivision level, the benefits gained from the improvement of smoothness cannot keep pace with the cost caused by the incremental number of faces. To mitigate the gap between the smoothness and the number of faces, this paper devises a novel improved mesh subdivision method to coordinate the smoothness and the number of faces in a harmonious way. First, this paper introduces a variable threshold, rather than a constant threshold used in existing adaptive subdivision methods, to reduce the number of redundant faces while keeping the smoothness in each subdivision iteration. Second, to achieve the above goal, a new crack-solving method is developed to remove the cracks by refining the adjacent faces of the subdivided area. Third, as a result, the problem of coordinating the smoothness and the number of faces can be formulated as a multi-objective optimization problem, in which the possible threshold sequences constitute the solution space. Finally, the Non-dominated sorting genetic algorithm II (NSGA-II) is improved to efficiently search the Pareto frontier. Extensive experiments demonstrate that the proposed method consistently outperforms existing mesh subdivision methods in different settings.