scholarly journals Depth Map Renement Using Reliability Based Joint Trilateral Filter

1970 ◽  
Vol 7 (2) ◽  
pp. 107-116
Author(s):  
Takuya Matsuo ◽  
Naoki Kodera ◽  
Norishige Fukushima ◽  
Yutaka Ishibashi
Keyword(s):  
Lower Cost ◽  
Computational Cost ◽  
Depth Map ◽  
Optimization Methods ◽  
Depth Image ◽  
Depth Maps ◽  
Trilateral Filter ◽  
Depth Image Based Rendering ◽  
Depth Ranges

In this paper, we propose a renement lter for depth maps. The lter convolutes an image and a depth map with a cross computed kernel. We call the lter joint trilateral lter. Main advantages of the proposed method are that the lter ts outlines of objects in the depth map to silhouettes in the im- age, and the lter reduces Gaussian noise in other areas. The eects reduce rendering artifacts when a free viewpoint image is generated by point cloud ren- dering and depth image based rendering techniques. Additionally, their computational cost is independent of depth ranges. Thus we can obtain accurate depth maps with the lower cost than the conventional ap- proaches, which require Markov random eld based optimization methods. Experimental results show that the accuracy of the depth map in edge areas goes up and its running time decreases. In addition, the lter improves the accuracy of edges in the depth map from Kinect sensor. As results, the quality of the rendering image is improved.

scholarly journals No-Reference Depth Map Quality Evaluation Model Based on Depth Map Edge Confidence Measurement in Immersive Video Applications

2019 ◽  
Vol 11 (10) ◽  
pp. 204 ◽  
Author(s):  
Dogan ◽  
Haddad ◽  
Ekmekcioglu ◽  
Kondoz
Keyword(s):  
Quality Evaluation ◽  
Evaluation Model ◽  
Depth Map ◽  
Depth Image ◽  
Depth Maps ◽  
Model Based ◽  
Objective Quality ◽  
Quality Evaluation Model ◽  
Confidence Measurement

When it comes to evaluating perceptual quality of digital media for overall quality of experience assessment in immersive video applications, typically two main approaches stand out: Subjective and objective quality evaluation. On one hand, subjective quality evaluation offers the best representation of perceived video quality assessed by the real viewers. On the other hand, it consumes a significant amount of time and effort, due to the involvement of real users with lengthy and laborious assessment procedures. Thus, it is essential that an objective quality evaluation model is developed. The speed-up advantage offered by an objective quality evaluation model, which can predict the quality of rendered virtual views based on the depth maps used in the rendering process, allows for faster quality assessments for immersive video applications. This is particularly important given the lack of a suitable reference or ground truth for comparing the available depth maps, especially when live content services are offered in those applications. This paper presents a no-reference depth map quality evaluation model based on a proposed depth map edge confidence measurement technique to assist with accurately estimating the quality of rendered (virtual) views in immersive multi-view video content. The model is applied for depth image-based rendering in multi-view video format, providing comparable evaluation results to those existing in the literature, and often exceeding their performance.

scholarly journals A Residual Network and FPGA Based Real-Time Depth Map Enhancement System

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 546
Author(s):  
Zhenni Li ◽  
Haoyi Sun ◽  
Yuliang Gao ◽  
Jiao Wang
Keyword(s):  
Real Time ◽  
Super Resolution ◽  
Depth Map ◽  
Acquisition System ◽  
Depth Image ◽  
Fpga Design ◽  
Depth Sensing ◽  
Residual Network ◽  
Real Time Processing ◽  
Depth Maps

Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 fps@ (1280 × 960 + 328 × 248 × 3).

scholarly journals Quality Assessment of 3D Synthesized Images Based on Textural and Structural Distortion Estimation

2021 ◽  
Vol 11 (6) ◽  
pp. 2666
Author(s):  
Hafiz Muhammad Usama Hassan Alvi ◽  
Muhammad Shahid Farid ◽  
Muhammad Hassan Khan ◽  
Marcin Grzegorzek
Keyword(s):  
Quality Assessment ◽  
Mixed Reality ◽  
Depth Estimation ◽  
Poor Quality ◽  
Depth Image ◽  
Depth Maps ◽  
Structural Distortions ◽  
Image Quality Metric ◽  
Technological Constraints

Emerging 3D-related technologies such as augmented reality, virtual reality, mixed reality, and stereoscopy have gained remarkable growth due to their numerous applications in the entertainment, gaming, and electromedical industries. In particular, the 3D television (3DTV) and free-viewpoint television (FTV) enhance viewers’ television experience by providing immersion. They need an infinite number of views to provide a full parallax to the viewer, which is not practical due to various financial and technological constraints. Therefore, novel 3D views are generated from a set of available views and their depth maps using depth-image-based rendering (DIBR) techniques. The quality of a DIBR-synthesized image may be compromised for several reasons, e.g., inaccurate depth estimation. Since depth is important in this application, inaccuracies in depth maps lead to different textural and structural distortions that degrade the quality of the generated image and result in a poor quality of experience (QoE). Therefore, quality assessment DIBR-generated images are essential to guarantee an appreciative QoE. This paper aims at estimating the quality of DIBR-synthesized images and proposes a novel 3D objective image quality metric. The proposed algorithm aims to measure both textural and structural distortions in the DIBR image by exploiting the contrast sensitivity and the Hausdorff distance, respectively. The two measures are combined to estimate an overall quality score. The experimental evaluations performed on the benchmark MCL-3D dataset show that the proposed metric is reliable and accurate, and performs better than existing 2D and 3D quality assessment metrics.

scholarly journals High-Fidelity Depth Upsampling Using the Self-Learning Framework

Sensors ◽  
2018 ◽  
Vol 19 (1) ◽  
pp. 81
Author(s):  
Inwook Shim ◽  
Tae-Hyun Oh ◽  
In Kweon
Keyword(s):  
Stereo Matching ◽  
Depth Map ◽  
The Self ◽  
Sensor Data ◽  
High Fidelity ◽  
Learning Framework ◽  
Depth Maps ◽  
Potential Applications ◽  
Self Learning

This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points.

scholarly journals Artifact Handling Based on Depth Image for View Synthesis

2019 ◽  
Vol 9 (9) ◽  
pp. 1834 ◽  
Author(s):  
Xiaodong Chen ◽  
Haitao Liang ◽  
Huaiyuan Xu ◽  
Siyu Ren ◽  
Huaiyu Cai ◽  
...  
Keyword(s):  
Objective Evaluation ◽  
Depth Image ◽  
Reference Image ◽  
Depth Information ◽  
Virtual Image ◽  
Hole Filling ◽  
Virtual View ◽  
Depth Image Based Rendering ◽  
Handling Method

The depth image based rendering (DIBR) is a popular technology for 3D video and free viewpoint video (FVV) synthesis, by which numerous virtual views can be generated from a single reference view and its depth image. However, some artifacts are produced in the DIBR process and reduce the visual quality of virtual view. Due to the diversity of artifacts, effectively handling them becomes a challenging task. In this paper, an artifact handling method based on depth image is proposed. The reference image and its depth image are extended to fill the holes that belong to the out-of-field regions. A depth image preprocessing method is applied to project the ghosts to their correct place. The 3D warping process is optimized by an adaptive one-to-four method to deal with the cracks and pixel overlapping. For disocclusions, we calculate depth and background terms of the filling priority based on depth information. The search for the best matching patch is performed simultaneously in the reference image and the virtual image. Moreover, adaptive patch size is used in all hole-filling processes. Experimental results demonstrate the effectiveness of the proposed method, which has better performance compared with previous methods in subjective and objective evaluation.

Spatial Distance-based Interpolation Algorithm for Computer Generated 2D+Z Images

2020 ◽  
Vol 2020 (2) ◽  
pp. 140-1-140-6
Author(s):  
Yuzhong Jiao ◽  
Kayton Wai Keung Cheung ◽  
Mark Ping Chan Mok ◽  
Yiu Kei Li
Keyword(s):  
Depth Map ◽  
Linear Interpolation ◽  
Spatial Distance ◽  
Depth Image ◽  
Interpolation Algorithm ◽  
3D Display ◽  
Common Input ◽  
Depth Maps ◽  
High Resolution Images ◽  
Rgb Images

Computer generated 2D plus Depth (2D+Z) images are common input data for 3D display with depth image-based rendering (DIBR) technique. Due to their simplicity, linear interpolation methods are usually used to convert low-resolution images into high-resolution images for not only depth maps but also 2D RGB images. However linear methods suffer from zigzag artifacts in both depth map and RGB images, which severely affects the 3D visual experience. In this paper, spatial distance-based interpolation algorithm for computer generated 2D+Z images is proposed. The method interpolates RGB images with the help of depth and edge information from depth maps. Spatial distance from interpolated pixel to surrounding available pixels is utilized to obtain the weight factors of surrounding pixels. Experiment results show that such spatial distance-based interpolation can achieve sharp edges and less artifacts for 2D RGB images. Naturally, it can improve the performance of 3D display. Since bilinear interpolation is used in homogenous areas, the proposed algorithm keeps low computational complexity.

scholarly journals ANALYSIS AND ENHANCEMENT OF THE DENOISING DEPTH DATA USING KINECT THROUGH ITERATIVE TECHNIQUE

2016 ◽  
Vol 78 (9) ◽  
Author(s):  
Mostafa Karbasi ◽  
Sara Bilal ◽  
Reza Aghababaeyan ◽  
Abdolvahab Ehsani Rad ◽  
Zeeshan Bhatti ◽  
...  
Keyword(s):  
Background Noise ◽  
Depth Map ◽  
Depth Image ◽  
Shadow Removal ◽  
3D Image ◽  
High Quality ◽  
Depth Data ◽  
Different Types ◽  
Accuracy And Stability

Since the release of Kinect by Microsoft, the, accuracy and stability of Kinect data-such as depth map, has been essential and important element of research and data analysis. In order to develop efficient means of analyzing and using the kinnect data, researchers require high quality of depth data during the preprocessing step, which is very crucial for accurate results. One of the most important concerns of researchers is to eliminate image noise and convert image and video to the best quality. In this paper, different types of the noise for Kinect are analyzed and a unique technique is used, to reduce the background noise based on distance between Kinect devise and the user. Whereas, for shadow removal, the iterative method is used to eliminate the shadow casted by the Kinect. A 3D depth image is obtained as a result with good quality and accuracy. Further, the results of this present study reveal that the image background is eliminated completely and the 3D image quality in depth map has been enhanced.

Sign in / Sign up

Export Citation Format

Share Document