Bas-relief Modeling from RGB Monocular Images with Regional Division Characteristics

Bas-relief, a form of sculpture art representation, has the general characteristics of sculpture art and satisfies people’s visual and tactile feelings by fully leveraging the advantages of painting art in composition, subject matter, and spatial processing. Bas-relief modeling using images is generally classified into the method based on the three-dimensional (3D) model, that based on the image depth restoration, and that based on multi-images. The 3D model method requires the 3D model of the object in advance. Bas-relief modeling based on the image depth restoration method usually either uses a depth camera to obtain object depth information or restores the depth information of pixels through the image. Bas-relief modeling based on the multi-image requires a short running time and has high efficiency in processing high resolution level images. Our method can automatically obtain the pixel height of each area in the image and can adjust the concave–convex relationship of each image area to obtain a bas-relief model based on the RGB monocular image. First, the edge contour of an RGB monocular image is extracted and refined by the Gauss difference algorithm based on tangential flow. Subsequently, the complete image contour information is extracted and the region-based image segmentation is used to calibrate the region. This method has improved running speed and stability compared with the traditional algorithm. Second, the regions of the RGB monocular image are divided by the improved connected-component labeling algorithm. In the traditional region calibration algorithm, the contour search strategy and the inner and outer contour definition rules of the image considered result in a low region division efficiency. This study uses an improved contour-based calibration algorithm. Then, the 3D pixel point cloud of each region is calculated by the shape-from-shading algorithm. The concave–convex relationships among these regions can be adjusted by human–computer interaction to form a reasonable bas-relief model. Lastly, the bas-relief model is obtained through triangular reconstruction using the Delaunay triangulation algorithm. The final bas-relief modeling effect is displayed by OpenGL. In this study, six groups of images are selected for conducting regional division tests, and the results obtained by the proposed method and other existing methods are compared. The proposed algorithm shows improved image processing running time for different complexity levels compared with the traditional two-pass scanning method and seed filling method (by approximately 2 s) and with the contour tracking method (by approximately 4 s). Next, image depth recovery experiments are conducted on four sets of images, namely the “ treasure seal,” “Wen Emperor seal,” “lily pattern,” and “peacock pattern,” and the results are compared. The depth of the image obtained by the traditional algorithm is generally lower than the actual plane, and the relative height of each region is not consistent with the actual situation. The proposed algorithm provides height values consistent with the height value information judged in the original image and adjusts the accurate concave–convex relationships. Moreover, the noise in the image is reduced and relatively smooth surfaces are obtained, with accurate concave–convex relationships. The proposed bas-relief model based on RGB monocular images can automatically determine the pixel height of each image area in the image and adjust the concave–convex relationship of each image area. In addition, it can recover the 3D model of the object from the image, enrich the object of bas-relief modeling, and expand the creation space of bas-relief, thereby improving the production efficiency of the bas-relief model based on RGB monocular images. The method has certain shortcomings, which require further exploration. For example, during the process of image contour extraction for region division, small differences exist between the obtained result and the actual situation, which can in turn affect the image depth recovery in the later stage. In addition, partial distortion may occur in the process of 3D reconstruction, which requires further research on point cloud data processing to reconstruct a high-quality three-dimensional surface.

EfficientNet-B0 Based Monocular Dense-Depth Map Estimation

Monocular depth estimation is a hot research topic in autonomous car driving. Deep convolution neural networks (DCNN) comprising encoder and decoder with transfer learning are exploited in the proposed work for monocular depth map estimation of two-dimensional images. Extracted CNN features from initial stages are later upsampled using a sequence of Bilinear UpSampling and convolution layers to reconstruct the depth map. The encoder forms the feature extraction part, and the decoder forms the image reconstruction part. EfficientNetB0, a new architecture is used with pretrained weights as encoder. It is a revolutionary architecture with smaller model parameters yet achieving higher efficiencies than the architectures of state-of-the-art, pretrained networks. EfficientNet-B0 is compared with two other pretrained networks, the DenseNet-121 and ResNet50 models. Each of these three models are used in encoding stage for features extraction followed by bilinear method of UpSampling in the decoder. The Monocular image is an ill-posed problem and is thus considered as a regression problem. So the metrics used in the proposed work are F1-score, Jaccard score and Mean Actual Error (MAE) etc., between the original and the reconstructed image. The results convey that EfficientNet-B0 outperforms in validation loss, F1-score and Jaccard score compared to DenseNet-121 and ResNet-50 models.

Interocular conflict from a monocular augmented reality display: Impact of visual characteristics on performance

In monocular see-through augmented reality systems, each eye is stimulated differently by a monocular image that is superimposed on the binocular background. This can impair binocular fusion, due to interocular conflict. As a function of visual characteristics, the latter can have a greater or lesser impact on user comfort and performance. This study tested several visual characteristics of a binocular background and a monocular element during an exposure that reproduced the interocular conflict induced by a monocular see-through near-eye display. The aim was to identify which factors impact the user’s performance. Performance was measured as target tracking and event detection, identification, fixation time, and latency. Our results demonstrate that performance is a function of the binocular background. Furthermore, exogenous attentional stimulation, in the form of a pulse with different levels of contrast applied to the monocular display, appears to preserve performance in most background conditions.

Efficient Monocular Depth Estimation with Transfer Feature Enhancement

Estimating the depth of the scene from a monocular image is an essential step for image semantic understanding. Practically, some existing methods for this highly ill-posed issue are still in lack of robustness and efficiency. This paper proposes a novel end-to-end depth esti- mation model with skip connections from a pre- trained Xception model for dense feature extrac- tion, and three new modules are designed to im- prove the upsampling process. In addition, ELU activation and convolutions with smaller kernel size are added to improve the pixel-wise regres- sion process. The experimental results show that our model has fewer network parameters, a lower error rate than the most advanced networks and requires only half the training time. The evalu- ation is based on the NYU v2 dataset, and our proposed model can achieve clearer boundary de- tails with state-of-the-art effects and robustness.