Bas-relief Modeling from RGB Monocular Images with Regional Division Characteristics

Bas-relief, a form of sculpture art representation, has the general characteristics of sculpture art and satisfies people’s visual and tactile feelings by fully leveraging the advantages of painting art in composition, subject matter, and spatial processing. Bas-relief modeling using images is generally classified into the method based on the three-dimensional (3D) model, that based on the image depth restoration, and that based on multi-images. The 3D model method requires the 3D model of the object in advance. Bas-relief modeling based on the image depth restoration method usually either uses a depth camera to obtain object depth information or restores the depth information of pixels through the image. Bas-relief modeling based on the multi-image requires a short running time and has high efficiency in processing high resolution level images. Our method can automatically obtain the pixel height of each area in the image and can adjust the concave–convex relationship of each image area to obtain a bas-relief model based on the RGB monocular image. First, the edge contour of an RGB monocular image is extracted and refined by the Gauss difference algorithm based on tangential flow. Subsequently, the complete image contour information is extracted and the region-based image segmentation is used to calibrate the region. This method has improved running speed and stability compared with the traditional algorithm. Second, the regions of the RGB monocular image are divided by the improved connected-component labeling algorithm. In the traditional region calibration algorithm, the contour search strategy and the inner and outer contour definition rules of the image considered result in a low region division efficiency. This study uses an improved contour-based calibration algorithm. Then, the 3D pixel point cloud of each region is calculated by the shape-from-shading algorithm. The concave–convex relationships among these regions can be adjusted by human–computer interaction to form a reasonable bas-relief model. Lastly, the bas-relief model is obtained through triangular reconstruction using the Delaunay triangulation algorithm. The final bas-relief modeling effect is displayed by OpenGL. In this study, six groups of images are selected for conducting regional division tests, and the results obtained by the proposed method and other existing methods are compared. The proposed algorithm shows improved image processing running time for different complexity levels compared with the traditional two-pass scanning method and seed filling method (by approximately 2 s) and with the contour tracking method (by approximately 4 s). Next, image depth recovery experiments are conducted on four sets of images, namely the “ treasure seal,” “Wen Emperor seal,” “lily pattern,” and “peacock pattern,” and the results are compared. The depth of the image obtained by the traditional algorithm is generally lower than the actual plane, and the relative height of each region is not consistent with the actual situation. The proposed algorithm provides height values consistent with the height value information judged in the original image and adjusts the accurate concave–convex relationships. Moreover, the noise in the image is reduced and relatively smooth surfaces are obtained, with accurate concave–convex relationships. The proposed bas-relief model based on RGB monocular images can automatically determine the pixel height of each image area in the image and adjust the concave–convex relationship of each image area. In addition, it can recover the 3D model of the object from the image, enrich the object of bas-relief modeling, and expand the creation space of bas-relief, thereby improving the production efficiency of the bas-relief model based on RGB monocular images. The method has certain shortcomings, which require further exploration. For example, during the process of image contour extraction for region division, small differences exist between the obtained result and the actual situation, which can in turn affect the image depth recovery in the later stage. In addition, partial distortion may occur in the process of 3D reconstruction, which requires further research on point cloud data processing to reconstruct a high-quality three-dimensional surface.

Vit-GAN: Image-to-image Translation with Vision Transformes and Conditional GANS

In this paper, we have developed a general-purpose architecture, Vit-Gan, capable of performing most of the image-to-image translation tasks from semantic image segmentation to single image depth perception. This paper is a follow-up paper, an extension of generator based model [1] in which the obtained results were very promising. This opened the possibility of further improvements with adversarial architecture. We used a unique vision transformers-based generator architecture and Conditional GANs(cGANs) with a Markovian Discriminator (PatchGAN) (https://github.com/YigitGunduc/vit-gan). In the present work, we use images as conditioning arguments. It is observed that the obtained results are more realistic than the commonly used architectures.

Vit-GAN: Image-to-image Translation with Vision Transformes and Conditional GANS

In this paper, we have developed a general-purpose architecture, Vit-Gan, capable of performing most of the image-to-image translation tasks from semantic image segmentation to single image depth perception. This paper is a follow-up paper, an extension of generator based model [1] in which the obtained results were very promising. This opened the possibility of further improvements with adversarial architecture. We used a unique vision transformers-based generator architecture and Conditional GANs(cGANs) with a Markovian Discriminator (PatchGAN) (https://github.com/YigitGunduc/vit-gan). In the present work, we use images as conditioning arguments. It is observed that the obtained results are more realistic than the commonly used architectures.

The Synthesis of Unpaired Underwater Images for Monocular Underwater Depth Prediction

Underwater depth prediction plays an important role in underwater vision research. Because of the complex underwater environment, it is extremely difficult and expensive to obtain underwater datasets with reliable depth annotation. Thus, underwater depth map estimation with a data-driven manner is still a challenging task. To tackle this problem, we propose an end-to-end system including two different modules for underwater image synthesis and underwater depth map estimation, respectively. The former module aims to translate the hazy in-air RGB-D images to multi-style realistic synthetic underwater images while retaining the objects and the structural information of the input images. Then we construct a semi-real RGB-D underwater dataset using the synthesized underwater images and the original corresponding depth maps. We conduct supervised learning to perform depth estimation through the pseudo paired underwater RGB-D images. Comprehensive experiments have demonstrated that the proposed method can generate multiple realistic underwater images with high fidelity, which can be applied to enhance the performance of monocular underwater image depth estimation. Furthermore, the trained depth estimation model can be applied to real underwater image depth map estimation. We will release our codes and experimental setting in https://github.com/ZHAOQIII/UW_depth.

Image Enhancement Algorithm Based on Depth Difference and Illumination Adjustment

In order to improve the clarity and color fidelity of traffic images under the complex environment of haze and uneven illumination and promote road traffic safety monitoring, a traffic image enhancement model based on illumination adjustment and depth of field difference is proposed. The algorithm is based on Retinex theory, uses dark channel principle to obtain image depth of the field, and uses spectral clustering algorithm to cluster image depth. After the subimages are divided, the local haze concentration is estimated according to the depth of field and the subimages are adaptively enhanced and fused. In addition, the illumination component is obtained by multiscale guided filtering to maintain the edge characteristics of the image, and the uneven illumination problem is solved by adjusting the curve function. The experimental results show that the proposed model can effectively enhance the uneven illumination and haze weather image in the traffic scene and the visual effect of the images is good. The generated image has rich details, improves the quality of traffic images, and can meet the needs of traffic practical application.

Investigation of Single Image Depth Prediction Under Different Lighting Conditions

This article investigates the limitations of single image depth prediction (SIDP) under different lighting conditions. Besides that, it also offers a new approach to obtain the ideal condition for SIDP. To satisfy the data requirement, we exploit a photometric stereo dataset consisting of several images of an object under different light properties. In this work, we used a dataset of ancient Roman coins captured under 54 different lighting conditions to illustrate how the approach is affected by them. This dataset emulates many lighting variances with a different state of shading and reflectance common in the natural environment. The ground truth depth data in the dataset was obtained using the stereo photometric method and used as training data. We investigated the capabilities of three different state-of-the-art methods to reconstruct ancient Roman coins with different lighting scenarios. The first investigation compares the performance of a given network using previously trained data to check cross-domains performance. Second, the model is fine-tuned from pre-trained data and trained using 70% of the ancient Roman coin dataset. Both models are tested on the remaining 30% of the data. As evaluation metrics, root mean square error and visual inspection are used. As a result, the methods show different characteristic results based on the lighting condition of the test data. Overall, they perform better at 51° and 71° angles of light, so-called ideal condition afterward. However, they perform worse at 13° and 32° because of the high density of shadows. They also cannot reach the best performance at 82° caused by the reflection that appears on the image. Based on these findings, we propose a new approach to reduce the shadows and reflections on the image using intrinsic image decomposition to achieve a synthetic ideal condition. Based on the results of synthetic images, this approach can enhance the performance of SIDP. For some state-of-the-art methods, it also achieves better results than previous original RGB images.