Bokeh Effect Rendering with Vision Transformers

Bokeh effect is growing to be an important feature in photography, essentially to choose an object of interest to be in focus with the rest of the background being blurred. While naturally rendering this effect requires a DSLR with large diameter of aperture, with the current advancements in Deep Learning, this effect can also be produced in mobile cameras. Most of the existing methods use Convolutional Neural Networks while some relying on the depth map to render this effect. In this paper, we propose an end-to-end Vision Transformer model for Bokeh rendering of images from monocular camera. This architecture uses vision transformers as backbone, thus learning from the entire image rather than just the parts from the filters in a CNN. This property of retaining global information coupled with initial training of the model for image restoration before training to render the blur effect for the background, allows our method to produce clearer images and outperform the current state-of-the-art models on the EBB! Data set. The code to our proposed method can be found at: https://github.com/Soester10/ Bokeh-Rendering-with-Vision-Transformers.

Bokeh Effect Rendering with Vision Transformers

Bokeh effect is growing to be an important feature in photography, essentially to choose an object of interest to be in focus with the rest of the background being blurred. While naturally rendering this effect requires a DSLR with large diameter of aperture, with the current advancements in Deep Learning, this effect can also be produced in mobile cameras. Most of the existing methods use Convolutional Neural Networks while some relying on the depth map to render this effect. In this paper, we propose an end-to-end Vision Transformer model for Bokeh rendering of images from monocular camera. This architecture uses vision transformers as backbone, thus learning from the entire image rather than just the parts from the filters in a CNN. This property of retaining global information coupled with initial training of the model for image restoration before training to render the blur effect for the background, allows our method to produce clearer images and outperform the current state-of-the-art models on the EBB! Data set. The code to our proposed method can be found at: https://github.com/Soester10/ Bokeh-Rendering-with-Vision-Transformers.

Faster than Real-Time Surface Pose Estimation with Application to Autonomous Robotic Grasping

Motivated by grasp planning applications within cluttered environments, this paper presents a novel approach to performing real-time surface segmentations of never-before-seen objects scattered across a given scene. This approach utilizes an input 2D depth map, where a first principles-based algorithm is utilized to exploit the fact that continuous surfaces are bounded by contours of high gradient. From these regions, the associated object surfaces can be isolated and further adapted for grasp planning. This paper also provides details for extracting the six-DOF pose for an isolated surface and presents the case of leveraging such a pose to execute planar grasping to achieve both force and torque closure. As a consequence of the highly parallel software implementation, the algorithm is shown to outperform prior approaches across all notable metrics and is also shown to be invariant to object rotation, scale, orientation relative to other objects, clutter, and varying degree of noise. This allows for a robust set of operations that could be applied to many areas of robotics research. The algorithm is faster than real time in the sense that it is nearly two times faster than the sensor rate of 30 fps.

Obstacle Avoidance and Environmental Adaptability Analysis of Snake-like Robot Based on Deep Learning

Aiming at the problems of high complexity and low accuracy of visual depth map feature recognition, a graph recognition algorithm based on principal component direction depth gradient histogram (pca-hodg) is designed in this study. In order to obtain high-quality depth map, it is necessary to calculate the parallax of the visual image. At the same time, in order to obtain the quantized regional shape histogram, it is necessary to carry out edge detection and gradient calculation on the depth map, then reduce the dimension of the depth map combined with the principal component, and use the sliding window detection method to reduce the dimension again to realize the feature extraction of the depth map. The results show that compared with other algorithms, the pca-hodg algorithm designed in this study improves the average classification accuracy and significantly reduces the average running time. This shows that the algorithm can reduce the running time by reducing the dimension, extract the depth map features more accurately, and has good robustness.

3D reconstruction of human body model based on incremental Motion recovery structure

3D reconstruction of human body model is a very important research topic in 3D reconstruction and also a challenging research direction in engineering field. In this paper, the whole pipeline flow of 3D reconstruction of human body model based on incremental motion recovery structure is proposed. Use mobile phone to collect images from different angles and screen them; Secondly, feature extraction and matching under SIFT operator, sparse reconstruction of incremental motion recovery structure, dense reconstruction based on depth map and other processes are carried out. Poisson surface reconstruction is finally carried out to achieve model reconstruction. Experiments show that the effect subject of the reconstructed model is clear.

EANet: Depth Estimation Based on EPI of Light Field

The light field is an important way to record the spatial information of the target scene. The purpose of this paper is to obtain depth information through the processing of light field information and provide a basis for intelligent medical treatment. In this paper, we first design an attention module to extract the features of light field images and connect all the features as a feature map to generate an attention image. Then, the attention map is integrated with the convolution layer in the neural network in the form of weights to enhance the weight of the subaperture viewpoint, which is more meaningful for depth estimation. Finally, the obtained initial depth results were optimized. The experimental results show that the MSE, PSNR, and SSIM of the depth map obtained by this method are increased by about 13%, 10 dB, and 4%, respectively, in some scenarios with good performance.

Active IR location as the roadway profilometry method

This article observe a using of active infrared beam location as roadway surface quality control. Changes in the spatial structure of the emitted IR radiation by surfaces within the capture scene allow creating a depth map of this scene. An optical camera makes it possible to use classical computer vision methods for stitching a depth map. For testing the possibility of using this approach, we made statistical studies on a multiple sample of distance measurements. Here we explain two experimental schemes with a programmable mechanical scanning system. The first one, we had determined the distance, which the image is capture accurately. The second, we measure the planar resolution, a minimum size of the defect that recognize by the infrared beam location system.

Per-pixel displacement mapping using cone tracing with correct silhouette

Per-pixel displacement mapping is a texture mapping technique that adds the microrelief effect to 3D surfaces without increasing the density of their corresponding meshes. This technique relies on ray tracing algorithms to find the intersection point between the viewing ray and the microrelief stored in a 2D texture called a depth map. This intersection makes it possible to deter- mine the corresponding pixel to produce an illusion of surface displacement instead of a real one. Cone tracing is one of the per-pixel displacement map- ping techniques for real-time rendering that relies on the encoding of the empty space around each pixel of the depth map. During the preprocessing stage, this space is encoded in the form of top-opened cones and then stored in a 2D texture, and during the rendering stage, it is used to converge more quickly to the intersection point. Cone tracing technique produces satisfacto- ry results in the case of flat surfaces, but when it comes to curved surfaces, it does not support the silhouette at the edges of the 3D mesh, that is to say, the relief merges with the surface of the object, and in this case, it will not be rendered correctly. To overcome this limitation, we have presented two new cone tracing algorithms that allow taking into consideration the curvature of the 3D surface to determine the fragments belonging to the silhouette. These two algorithms are based on a quadratic approximation of the object geometry at each vertex of the 3D mesh. The main objective of this paper is to achieve a texture mapping with a realistic appearance and at a low cost so that the rendered objects will have real and complex details that are vis- ible on their entire surface and without modifying their geometry. Based on the ray-tracing algorithm, our contribution can be useful for current graphics card generation, since the programmable units and the frameworks associat- ed with the new graphics cards integrate today the technology of ray tracing.