Algorithm for correcting the image depth map based on the points brightness and their distance from the observation point

Objective. Modeling the human head is a significant problem that arises in a wide variety of fields of science and technology. Existing active technologies for reconstruction and modeling of the object under study require expensive equipment and trained personnel. Methods. An alternative is to use passive methods that perform image processing using special mathematical algorithms. One of these methods is the stereo vision, which is based on the use of paired images taken simultaneously with several cameras positioned and calibrated in a certain way. However, a common drawback of stereo vision methods is the possibility of obtaining erroneous depth maps due to poorquality source images or incorrect camera and lighting settings. Results. Procedures were developed that use additional parameters of image points, which can be used to correct depth maps to avoid the appearance of defects. To achieve this objective, the existing mathematical software for processing photo and video materials is analyzed; methods for suppressing noise in the image, obtaining an image contour, as well as a method for obtaining a 3D object matrix based on changing the direction of illumination are proposed; the algorithm is tested on a test example. Conclusion. The developed technique should improve the quality of the depth map of the processed image and thus make the modeling procedures more efficient.

Download Full-text

No-Reference Depth Map Quality Evaluation Model Based on Depth Map Edge Confidence Measurement in Immersive Video Applications

Future Internet ◽

10.3390/fi11100204 ◽

2019 ◽

Vol 11 (10) ◽

pp. 204 ◽

Cited By ~ 3

Author(s):

Dogan ◽

Haddad ◽

Ekmekcioglu ◽

Kondoz

Keyword(s):

Quality Evaluation ◽

Evaluation Model ◽

Depth Map ◽

Depth Image ◽

Depth Maps ◽

Model Based ◽

Objective Quality ◽

Quality Evaluation Model ◽

Confidence Measurement

When it comes to evaluating perceptual quality of digital media for overall quality of experience assessment in immersive video applications, typically two main approaches stand out: Subjective and objective quality evaluation. On one hand, subjective quality evaluation offers the best representation of perceived video quality assessed by the real viewers. On the other hand, it consumes a significant amount of time and effort, due to the involvement of real users with lengthy and laborious assessment procedures. Thus, it is essential that an objective quality evaluation model is developed. The speed-up advantage offered by an objective quality evaluation model, which can predict the quality of rendered virtual views based on the depth maps used in the rendering process, allows for faster quality assessments for immersive video applications. This is particularly important given the lack of a suitable reference or ground truth for comparing the available depth maps, especially when live content services are offered in those applications. This paper presents a no-reference depth map quality evaluation model based on a proposed depth map edge confidence measurement technique to assist with accurately estimating the quality of rendered (virtual) views in immersive multi-view video content. The model is applied for depth image-based rendering in multi-view video format, providing comparable evaluation results to those existing in the literature, and often exceeding their performance.

Download Full-text

High-Fidelity Depth Upsampling Using the Self-Learning Framework

Sensors ◽

10.3390/s19010081 ◽

2018 ◽

Vol 19 (1) ◽

pp. 81

Author(s):

Inwook Shim ◽

Tae-Hyun Oh ◽

In Kweon

Keyword(s):

Stereo Matching ◽

Depth Map ◽

The Self ◽

Sensor Data ◽

High Fidelity ◽

Learning Framework ◽

Depth Maps ◽

Potential Applications ◽

Self Learning

This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points.

Download Full-text

ENHANCEMENT OF DEPTH MAP BY FUSION USING ADAPTIVE AND SEMANTIC-GUIDED SPATIOTEMPORAL FILTERING

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-3-2020-227-2020 ◽

2020 ◽

Vol V-3-2020 ◽

pp. 227-232

Author(s):

H. Albanwan ◽

R. Qin

Keyword(s):

Depth Map ◽

Spatiotemporal Analysis ◽

Stereo Image ◽

Single Pair ◽

Depth Maps ◽

3D Model Reconstruction ◽

Digital Elevation ◽

Image Pairs ◽

Spatiotemporal Filters

Abstract. Extracting detailed geometric information about a scene relies on the quality of the depth maps (e.g. Digital Elevation Surfaces, DSM) to enhance the performance of 3D model reconstruction. Elevation information from LiDAR is often expensive and hard to obtain. The most common approach to generate depth maps is through multi-view stereo (MVS) methods (e.g. dense stereo image matching). The quality of single depth maps, however, is often prone to noise, outliers, and missing data points due to the quality of the acquired image pairs. A reference multi-view image pair must be noise-free and clear to ensure high-quality depth maps. To avoid such a problem, current researches are headed toward fusing multiple depth maps to recover the shortcomings of single-depth maps resulted from a single pair of multi-view images. Several approaches tackled this problem by merging and fusing depth maps, using probabilistic and deterministic methods, but few discussed how these fused depth maps can be refined through adaptive spatiotemporal analysis algorithms (e.g. spatiotemporal filters). The motivation is to push towards preserving the high precision and detail level of depth maps while optimizing the performance, robustness, and efficiency of the algorithm.

Download Full-text

Modular Stereo Vision

Depth Map and 3D Imaging Applications ◽

10.4018/978-1-61350-326-3.ch013 ◽

2011 ◽

pp. 245-267

Author(s):

Ng Oon-Ee ◽

Velappa Ganapathy ◽

S.G. Ponnambalam

Keyword(s):

Stereo Vision ◽

Depth Map ◽

Standard Practice ◽

Entire Process ◽

Single Process ◽

Vision Algorithm ◽

New Algorithms

The two-frame stereo vision algorithm is typically conceived of and implemented as a single process. The standard practice is to categorize individual algorithms according to the ‘type’ of process used. Evaluation is done based on the quality of the depth map produced. In this chapter, we demonstrate that the stereo vision process is actually composed of a number of inter-linked processes. Stereo vision is shown to be modular in nature; algorithms implementing it typically implement distinct stages of the entire process. The modularity of stereo vision implies that the specific methods used in different algorithms can be combined to produce new algorithms. We present a model describing stereo vision in a modular manner. We also provide examples of the stereo vision process being implemented in a modular manner, with practical example code. The purpose of this chapter is to present this model and implementation for the use of researchers in the field of computational stereo vision.

Download Full-text

Depth Map Renement Using Reliability Based Joint Trilateral Filter

ECTI Transactions on Computer and Information Technology (ECTI-CIT) ◽

10.37936/ecti-cit.201372.54362 ◽

1970 ◽

Vol 7 (2) ◽

pp. 107-116

Author(s):

Takuya Matsuo ◽

Naoki Kodera ◽

Norishige Fukushima ◽

Yutaka Ishibashi

Keyword(s):

Lower Cost ◽

Computational Cost ◽

Depth Map ◽

Optimization Methods ◽

Depth Image ◽

Depth Maps ◽

Trilateral Filter ◽

Depth Image Based Rendering ◽

Depth Ranges

In this paper, we propose a renement lter for depth maps. The lter convolutes an image and a depth map with a cross computed kernel. We call the lter joint trilateral lter. Main advantages of the proposed method are that the lter ts outlines of objects in the depth map to silhouettes in the im- age, and the lter reduces Gaussian noise in other areas. The eects reduce rendering artifacts when a free viewpoint image is generated by point cloud ren- dering and depth image based rendering techniques. Additionally, their computational cost is independent of depth ranges. Thus we can obtain accurate depth maps with the lower cost than the conventional ap- proaches, which require Markov random eld based optimization methods. Experimental results show that the accuracy of the depth map in edge areas goes up and its running time decreases. In addition, the lter improves the accuracy of edges in the depth map from Kinect sensor. As results, the quality of the rendering image is improved.

Download Full-text

Depth Map Quality Evaluation for Photographic Applications

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.9.iqsp-370 ◽

2020 ◽

Vol 2020 (9) ◽

pp. 370-1-370-7

Author(s):

Eloi Zalczer ◽

François-Xavier Thomas ◽

Laurent Chanas ◽

Gabriele Facciolo ◽

Frédéric Guichard

Keyword(s):

Augmented Reality ◽

Quality Evaluation ◽

Depth Map ◽

Depth Imaging ◽

Depth Maps ◽

Spatial Alignment ◽

Need To Evaluate ◽

New Challenges ◽

Consumer Devices

As depth imaging is integrated into more and more consumer devices, manufacturers have to tackle new challenges. Applica- tions such as computational bokeh and augmented reality require dense and precisely segmented depth maps to achieve good re- sults. Modern devices use a multitude of different technologies to estimate depth maps, such as time-of-flight sensors, stereoscopic cameras, structured light sensors, phase-detect pixels or a com- bination thereof. Therefore, there is a need to evaluate the quality of the depth maps, regardless of the technology used to produce them. The aim of our work is to propose an end-result evalua- tion method based on a single scene, using a specifically designed chart. We consider the depth maps embedded in the photographs, which are not visible to the user but are used by specialized soft- ware, in association with the RGB pictures. Some of the aspects considered are spatial alignment between RGB and depth, depth consistency, and robustness to texture variations. This work also provides a comparison of perceptual and automatic evaluations.

Download Full-text

The Design and Implementation of Postprocessing for Depth Map on Real-Time Extraction System

The Scientific World JOURNAL ◽

10.1155/2014/363287 ◽

2014 ◽

Vol 2014 ◽

pp. 1-10 ◽

Cited By ~ 2

Author(s):

Zhiwei Tang ◽

Bin Li ◽

Huosheng Li ◽

Zheng Xu

Keyword(s):

Real Time ◽

Stereo Matching ◽

Depth Map ◽

Depth Estimation ◽

Consistency Check ◽

Key Technology ◽

Design And Implementation ◽

Depth Maps ◽

Matching Cost

Depth estimation becomes the key technology to resolve the communications of the stereo vision. We can get the real-time depth map based on hardware, which cannot implement complicated algorithm as software, because there are some restrictions in the hardware structure. Eventually, some wrong stereo matching will inevitably exist in the process of depth estimation by hardware, such as FPGA. In order to solve the problem a postprocessing function is designed in this paper. After matching cost unique test, the both left-right and right-left consistency check solutions are implemented, respectively; then, the cavities in depth maps can be filled by right depth values on the basis of right-left consistency check solution. The results in the experiments have shown that the depth map extraction and postprocessing function can be implemented in real time in the same system; what is more, the quality of the depth maps is satisfactory.

Download Full-text

A Residual Network and FPGA Based Real-Time Depth Map Enhancement System

Entropy ◽

10.3390/e23050546 ◽

2021 ◽

Vol 23 (5) ◽

pp. 546

Author(s):

Zhenni Li ◽

Haoyi Sun ◽

Yuliang Gao ◽

Jiao Wang

Keyword(s):

Real Time ◽

Super Resolution ◽

Depth Map ◽

Acquisition System ◽

Depth Image ◽

Fpga Design ◽

Depth Sensing ◽

Residual Network ◽

Real Time Processing ◽

Depth Maps

Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 fps@ (1280 × 960 + 328 × 248 × 3).

Download Full-text

Quality Assessment of 3D Synthesized Images Based on Textural and Structural Distortion Estimation

Applied Sciences ◽

10.3390/app11062666 ◽

2021 ◽

Vol 11 (6) ◽

pp. 2666

Author(s):

Hafiz Muhammad Usama Hassan Alvi ◽

Muhammad Shahid Farid ◽

Muhammad Hassan Khan ◽

Marcin Grzegorzek

Keyword(s):

Quality Assessment ◽

Mixed Reality ◽

Depth Estimation ◽

Poor Quality ◽

Depth Image ◽

Depth Maps ◽

Structural Distortions ◽

Image Quality Metric ◽

Technological Constraints

Emerging 3D-related technologies such as augmented reality, virtual reality, mixed reality, and stereoscopy have gained remarkable growth due to their numerous applications in the entertainment, gaming, and electromedical industries. In particular, the 3D television (3DTV) and free-viewpoint television (FTV) enhance viewers’ television experience by providing immersion. They need an infinite number of views to provide a full parallax to the viewer, which is not practical due to various financial and technological constraints. Therefore, novel 3D views are generated from a set of available views and their depth maps using depth-image-based rendering (DIBR) techniques. The quality of a DIBR-synthesized image may be compromised for several reasons, e.g., inaccurate depth estimation. Since depth is important in this application, inaccuracies in depth maps lead to different textural and structural distortions that degrade the quality of the generated image and result in a poor quality of experience (QoE). Therefore, quality assessment DIBR-generated images are essential to guarantee an appreciative QoE. This paper aims at estimating the quality of DIBR-synthesized images and proposes a novel 3D objective image quality metric. The proposed algorithm aims to measure both textural and structural distortions in the DIBR image by exploiting the contrast sensitivity and the Hausdorff distance, respectively. The two measures are combined to estimate an overall quality score. The experimental evaluations performed on the benchmark MCL-3D dataset show that the proposed metric is reliable and accurate, and performs better than existing 2D and 3D quality assessment metrics.

Download Full-text

Estimation of Soil Surface Roughness Using Stereo Vision Approach

Sensors ◽

10.3390/s21134386 ◽

2021 ◽

Vol 21 (13) ◽

pp. 4386

Author(s):

Afshin Azizi ◽

Yousef Abbaspour-Gilandeh ◽

Tarahom Mesri-Gundoshmian ◽

Aitazaz A. Farooque ◽

Hassan Afzaal

Keyword(s):

Surface Roughness ◽

Stereo Vision ◽

Agricultural Soils ◽

Depth Map ◽

Soil Surface ◽

Regression Equations ◽

Height Profile ◽

Peak Fitting ◽

Soil Surface Roughness ◽

Tillage Operations

Soil roughness is one of the most challenging issues in the agricultural domain and plays a crucial role in soil quality. The objective of this research was to develop a computerized method based on stereo vision technique to estimate the roughness formed on the agricultural soils. Additionally, soil till quality was investigated by analyzing the height of plow layers. An image dataset was provided in the real conditions of the field. For determining the soil surface roughness, the elevation of clods obtained from tillage operations was computed using a depth map. This map was obtained by extracting and matching corresponding keypoints as super pixels of images. Regression equations and coefficients of determination between the measured and estimated values indicate that the proposed method has a strong potential for the estimation of soil shallow roughness as an important physical parameter in tillage operations. In addition, peak fitting of tilled layers was applied to the height profile to evaluate the till quality. The results of this suggest that the peak fitting is an effective method of judging tillage quality in the fields.

Download Full-text