Joint 3D facial shape reconstruction and texture completion from a single image

Recent years have witnessed significant progress in image-based 3D face reconstruction using deep convolutional neural networks. However, current reconstruction methods often perform improperly in self-occluded regions and can lead to inaccurate correspondences between a 2D input image and a 3D face template, hindering use in real applications. To address these problems, we propose a deep shape reconstruction and texture completion network, SRTC-Net, which jointly reconstructs 3D facial geometry and completes texture with correspondences from a single input face image. In SRTC-Net, we leverage the geometric cues from completed 3D texture to reconstruct detailed structures of 3D shapes. The SRTC-Net pipeline has three stages. The first introduces a correspondence network to identify pixel-wise correspondence between the input 2D image and a 3D template model, and transfers the input 2D image to a U-V texture map. Then we complete the invisible and occluded areas in the U-V texture map using an inpainting network. To get the 3D facial geometries, we predict coarse shape (U-V position maps) from the segmented face from the correspondence network using a shape network, and then refine the 3D coarse shape by regressing the U-V displacement map from the completed U-V texture map in a pixel-to-pixel way. We examine our methods on 3D reconstruction tasks as well as face frontalization and pose invariant face recognition tasks, using both in-the-lab datasets (MICC, MultiPIE) and in-the-wild datasets (CFP). The qualitative and quantitative results demonstrate the effectiveness of our methods on inferring 3D facial geometry and complete texture; they outperform or are comparable to the state-of-the-art.

Physically Implied Surfaces

In addition to seeing objects that are directly in view, we also represent objects that are merely implied (e.g., by occlusion, motion, and other cues). What can imply the presence of an object? Here, we explored (in three preregistered experiments; N = 360 adults) the role of physical interaction in creating impressions of objects that are not actually present. After seeing an actor collide with an invisible wall or step onto an invisible box, participants gave facilitated responses to actual, visible surfaces that appeared where the implied wall or box had been—a Stroop-like pattern of facilitation and interference that suggested automatic inferences about the relevant implied surfaces. Follow-up experiments ruled out confounding geometric cues and anticipatory responses. We suggest that physical interactions can trigger representations of the participating surfaces such that we automatically infer the presence of objects implied only by their physical consequences.

Clustering-Based Plane Refitting of Non-planar Patches for Voxel-Based 3D Point Cloud Segmentation Using K-Means Clustering

Point cloud processing is a struggled field because the points in the clouds are three-dimensional and irregular distributed signals. For this reason, the points in the point clouds are mostly sampled into regularly distributed voxels in the literature. Voxelization as a pretreatment significantly accelerates the process of segmenting surfaces. The geometric cues such as plane directions (normals) in the voxels are mostly used to segment the local surfaces. However, the sampling process may include a non-planar point group (patch), which is mostly on the edges and corners, in a voxel. These voxels can cause misleading the segmentation process. In this paper, we separate the non-planar patches into planar sub-patches using k-means clustering. The largest one among the planar sub-patches replaces the normal and barycenter properties of the voxel with those of itself. We have tested this process in a successful point cloud segmentation method and measure the effects of the proposed method on two point cloud segmentation datasets (Mosque and Train Station). The method increases the accuracy success of the Mosque dataset from 83.84% to 87.86% and that of the Train Station dataset from 85.36% to 87.07%.

Spindle scaling is governed by cell boundary regulation of microtubule nucleation

Cellular organelles such as the mitotic spindle adjust their size to the dimensions of the cell. It is widely understood that spindle scaling is governed by regulation of microtubule polymerization. Here we use quantitative microscopy in living zebrafish embryos and Xenopus egg extracts in combination with theory to show that microtubule polymerization dynamics are insufficient to scale spindles and only contribute below a critical cell size. In contrast, microtubule nucleation governs spindle scaling for all cell sizes. We show that this hierarchical regulation arises from the partitioning of a nucleation inhibitor to the cell membrane. Our results reveal that cells differentially regulate microtubule number and length using distinct geometric cues to maintain a functional spindle architecture over a large range of cell sizes.

Vacant Parking Slot Detection in the Around View Image Based on Deep Learning

Due to the complex visual environment, such as lighting variations, shadows, and limitations of vision, the accuracy of vacant parking slot detection for the park assist system (PAS) with a standalone around view monitor (AVM) needs to be improved. To address this problem, we propose a vacant parking slot detection method based on deep learning, namely VPS-Net. VPS-Net converts the vacant parking slot detection into a two-step problem, including parking slot detection and occupancy classification. In the parking slot detection stage, we propose a parking slot detection method based on YOLOv3, which combines the classification of the parking slot with the localization of marking points so that various parking slots can be directly inferred using geometric cues. In the occupancy classification stage, we design a customized network whose size of convolution kernel and number of layers are adjusted according to the characteristics of the parking slot. Experiments show that VPS-Net can detect various vacant parking slots with a precision rate of 99.63% and a recall rate of 99.31% in the ps2.0 dataset, and has a satisfying generalizability in the PSV dataset. By introducing a multi-object detection network and a classification network, VPS-Net can detect various vacant parking slots robustly.

Modulation of spatial cue processing across the lifespan: a geometric polarization of space restores allocentric navigation strategies in children and older adults

The impact of development and healthy aging on spatial cognition has been traditionally attributed to a difficulty in using allocentric strategies and a preference for egocentric ones. An alternative possibility, suggested by our previous works, is that this preference is actually conditioned by the spatial cues (e.g. geometric of landmark cues) present in the environment rather than a strategic choice per se. We tested this prediction by having 79 subjects (children, young and older adults) navigating a Y-maze composed either of landmarks or geometric cues, with an immersive head-mounted display that allows us to record both head and eye movements. Our results show that when the performance is based on landmarks solely, children and older adults exhibit a deficit in using allocentric strategies when compared to young adults. Hence, an inverted U-profile of allocentric strategies was observed across the lifespan. This was not due to a default of attention to the landmarks, as evidenced by analysis of gaze dynamics. When geometric were provided, however, older adults and children used allocentric strategies in the same proportion as young adults. They were, in addition, as efficient and quick to implement the strategy. We thus propose a reinterpretation of the previous data in the literature, whereby reference to geometric cues is the default mode for spatial representations, which is immune to age, whereas spatial representations fail to be anchored on landmarks early in development and later in aging. This new interpretation has the potential to reunify several data from the literature, ranging from spatial cues processing to strategy preference, and including other spatial skills like path integration and route learning.