scholarly journals ENHANCEMENT OF DEPTH MAP BY FUSION USING ADAPTIVE AND SEMANTIC-GUIDED SPATIOTEMPORAL FILTERING

2020 ◽  
Vol V-3-2020 ◽  
pp. 227-232
Author(s):  
H. Albanwan ◽  
R. Qin
Keyword(s):  
Depth Map ◽  
Spatiotemporal Analysis ◽  
Stereo Image ◽  
Single Pair ◽  
Depth Maps ◽  
3D Model Reconstruction ◽  
Digital Elevation ◽  
Image Pairs ◽  
Spatiotemporal Filters

Abstract. Extracting detailed geometric information about a scene relies on the quality of the depth maps (e.g. Digital Elevation Surfaces, DSM) to enhance the performance of 3D model reconstruction. Elevation information from LiDAR is often expensive and hard to obtain. The most common approach to generate depth maps is through multi-view stereo (MVS) methods (e.g. dense stereo image matching). The quality of single depth maps, however, is often prone to noise, outliers, and missing data points due to the quality of the acquired image pairs. A reference multi-view image pair must be noise-free and clear to ensure high-quality depth maps. To avoid such a problem, current researches are headed toward fusing multiple depth maps to recover the shortcomings of single-depth maps resulted from a single pair of multi-view images. Several approaches tackled this problem by merging and fusing depth maps, using probabilistic and deterministic methods, but few discussed how these fused depth maps can be refined through adaptive spatiotemporal analysis algorithms (e.g. spatiotemporal filters). The motivation is to push towards preserving the high precision and detail level of depth maps while optimizing the performance, robustness, and efficiency of the algorithm.

scholarly journals Constructing Stereo Presentations of Textual Information on an Autostereoscopic Monitor

2019 ◽  
Author(s):  
Сергей Андреев ◽  
Sergey Andreev ◽  
Николь Бондарева ◽  
Nicole Bondareva
Keyword(s):  
Numerical Simulation ◽  
Scientific Research ◽  
Practical Experience ◽  
Stereo Image ◽  
Stereo Images ◽  
General Study ◽  
Textual Information ◽  
Stereoscopic Images ◽  
Depth Maps

This paper presents practical experience in constructing stereo presentations of texts and formulas on an autostereoscopic monitor in stereo presentations designed to display the results of numerical simulation. The task of constructing stereo images of texts and formulas is a structural subtask of a general study devoted to the development of methods and algorithms for constructing stereo presentations of the results of scientific research. This paper discusses the construction of stereoscopic images on an autostereoscopic monitor. The autostereoscopic monitor allows one to observe a stereo image without glasses, while ensuring the quality of the stereo image, which is not inferior to the quality of the stereo image, presented using a classic 3D projection stereo system. Various methods of obtaining stereo images supported by the monitor were tested, namely, the multi-view presentation of the object and the construction of depth maps. The results for both methods are presented.

scholarly journals No-Reference Depth Map Quality Evaluation Model Based on Depth Map Edge Confidence Measurement in Immersive Video Applications

2019 ◽  
Vol 11 (10) ◽  
pp. 204 ◽  
Author(s):  
Dogan ◽  
Haddad ◽  
Ekmekcioglu ◽  
Kondoz
Keyword(s):  
Quality Evaluation ◽  
Evaluation Model ◽  
Depth Map ◽  
Depth Image ◽  
Depth Maps ◽  
Model Based ◽  
Objective Quality ◽  
Quality Evaluation Model ◽  
Confidence Measurement

When it comes to evaluating perceptual quality of digital media for overall quality of experience assessment in immersive video applications, typically two main approaches stand out: Subjective and objective quality evaluation. On one hand, subjective quality evaluation offers the best representation of perceived video quality assessed by the real viewers. On the other hand, it consumes a significant amount of time and effort, due to the involvement of real users with lengthy and laborious assessment procedures. Thus, it is essential that an objective quality evaluation model is developed. The speed-up advantage offered by an objective quality evaluation model, which can predict the quality of rendered virtual views based on the depth maps used in the rendering process, allows for faster quality assessments for immersive video applications. This is particularly important given the lack of a suitable reference or ground truth for comparing the available depth maps, especially when live content services are offered in those applications. This paper presents a no-reference depth map quality evaluation model based on a proposed depth map edge confidence measurement technique to assist with accurately estimating the quality of rendered (virtual) views in immersive multi-view video content. The model is applied for depth image-based rendering in multi-view video format, providing comparable evaluation results to those existing in the literature, and often exceeding their performance.

DEM Generation and Accuracy Assessment Using ZY-3 Stereo Image Pairs

2015 ◽  
Vol 738-739 ◽  
pp. 613-617 ◽  
Author(s):  
Guo Yin Cai ◽  
Jie Huan ◽  
Yang Liu ◽  
Ming Yi Du
Keyword(s):  
3D Visualization ◽  
Accuracy Assessment ◽  
Satellite Image ◽  
Stereo Image ◽  
Stereo Pair ◽  
Topographic Analysis ◽  
Digital Elevation ◽  
Data Source ◽  
Image Pairs ◽  
Elevation Model

Digital Elevation Model (DEM) is an important data source for topographic analysis, 3D visualization and satellite image ortho-rectification. This paper focused on the DEM extraction and accuracy assessment from ZY-3 satellite with 3 stereo images. DEM was extracted using three different stereo pair image groups composed of forward and nadir view images, nadir and backward view images as well as forward and backward view images. The accuracy of the DEM was indicated by root-mean-square error (RMSE) values. The results showed that the stereo pair of nadir and forward view images achieved the best accuracy, while the pair of forward and backward view images obtained the worst. This might be useful for the selection of the stereo pair images for extracting DEM using ZY-3 satellite images.

scholarly journals High-Fidelity Depth Upsampling Using the Self-Learning Framework

Sensors ◽  
2018 ◽  
Vol 19 (1) ◽  
pp. 81
Author(s):  
Inwook Shim ◽  
Tae-Hyun Oh ◽  
In Kweon
Keyword(s):  
Stereo Matching ◽  
Depth Map ◽  
The Self ◽  
Sensor Data ◽  
High Fidelity ◽  
Learning Framework ◽  
Depth Maps ◽  
Potential Applications ◽  
Self Learning

This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points.

scholarly journals Algorithm for correcting the image depth map based on the points brightness and their distance from the observation point

2020 ◽  
Vol 47 (3) ◽  
pp. 82-92
Author(s):  
S. I. Korotkevich ◽  
Yu. V. Minaeva
Keyword(s):  
Stereo Vision ◽  
Depth Map ◽  
Human Head ◽  
Observation Point ◽  
Mathematical Software ◽  
Depth Maps ◽  
Trained Personnel ◽  
Mathematical Algorithms ◽  
Image Contour

Objective. Modeling the human head is a significant problem that arises in a wide variety of fields of science and technology. Existing active technologies for reconstruction and modeling of the object under study require expensive equipment and trained personnel. Methods. An alternative is to use passive methods that perform image processing using special mathematical algorithms. One of these methods is the stereo vision, which is based on the use of paired images taken simultaneously with several cameras positioned and calibrated in a certain way. However, a common drawback of stereo vision methods is the possibility of obtaining erroneous depth maps due to poorquality source images or incorrect camera and lighting settings. Results. Procedures were developed that use additional parameters of image points, which can be used to correct depth maps to avoid the appearance of defects. To achieve this objective, the existing mathematical software for processing photo and video materials is analyzed; methods for suppressing noise in the image, obtaining an image contour, as well as a method for obtaining a 3D object matrix based on changing the direction of illumination are proposed; the algorithm is tested on a test example. Conclusion. The developed technique should improve the quality of the depth map of the processed image and thus make the modeling procedures more efficient. 

scholarly journals Depth Map Renement Using Reliability Based Joint Trilateral Filter

1970 ◽  
Vol 7 (2) ◽  
pp. 107-116
Author(s):  
Takuya Matsuo ◽  
Naoki Kodera ◽  
Norishige Fukushima ◽  
Yutaka Ishibashi
Keyword(s):  
Lower Cost ◽  
Computational Cost ◽  
Depth Map ◽  
Optimization Methods ◽  
Depth Image ◽  
Depth Maps ◽  
Trilateral Filter ◽  
Depth Image Based Rendering ◽  
Depth Ranges

In this paper, we propose a renement lter for depth maps. The lter convolutes an image and a depth map with a cross computed kernel. We call the lter joint trilateral lter. Main advantages of the proposed method are that the lter ts outlines of objects in the depth map to silhouettes in the im- age, and the lter reduces Gaussian noise in other areas. The eects reduce rendering artifacts when a free viewpoint image is generated by point cloud ren- dering and depth image based rendering techniques. Additionally, their computational cost is independent of depth ranges. Thus we can obtain accurate depth maps with the lower cost than the conventional ap- proaches, which require Markov random eld based optimization methods. Experimental results show that the accuracy of the depth map in edge areas goes up and its running time decreases. In addition, the lter improves the accuracy of edges in the depth map from Kinect sensor. As results, the quality of the rendering image is improved.

scholarly journals Depth Map Quality Evaluation for Photographic Applications

2020 ◽  
Vol 2020 (9) ◽  
pp. 370-1-370-7
Author(s):  
Eloi Zalczer ◽  
François-Xavier Thomas ◽  
Laurent Chanas ◽  
Gabriele Facciolo ◽  
Frédéric Guichard
Keyword(s):  
Augmented Reality ◽  
Quality Evaluation ◽  
Depth Map ◽  
Depth Imaging ◽  
Depth Maps ◽  
Spatial Alignment ◽  
Need To Evaluate ◽  
New Challenges ◽  
Consumer Devices

As depth imaging is integrated into more and more consumer devices, manufacturers have to tackle new challenges. Applica- tions such as computational bokeh and augmented reality require dense and precisely segmented depth maps to achieve good re- sults. Modern devices use a multitude of different technologies to estimate depth maps, such as time-of-flight sensors, stereoscopic cameras, structured light sensors, phase-detect pixels or a com- bination thereof. Therefore, there is a need to evaluate the quality of the depth maps, regardless of the technology used to produce them. The aim of our work is to propose an end-result evalua- tion method based on a single scene, using a specifically designed chart. We consider the depth maps embedded in the photographs, which are not visible to the user but are used by specialized soft- ware, in association with the RGB pictures. Some of the aspects considered are spatial alignment between RGB and depth, depth consistency, and robustness to texture variations. This work also provides a comparison of perceptual and automatic evaluations.

scholarly journals The Design and Implementation of Postprocessing for Depth Map on Real-Time Extraction System

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Zhiwei Tang ◽  
Bin Li ◽  
Huosheng Li ◽  
Zheng Xu
Keyword(s):  
Real Time ◽  
Stereo Matching ◽  
Depth Map ◽  
Depth Estimation ◽  
Consistency Check ◽  
Key Technology ◽  
Design And Implementation ◽  
Depth Maps ◽  
Matching Cost

Depth estimation becomes the key technology to resolve the communications of the stereo vision. We can get the real-time depth map based on hardware, which cannot implement complicated algorithm as software, because there are some restrictions in the hardware structure. Eventually, some wrong stereo matching will inevitably exist in the process of depth estimation by hardware, such as FPGA. In order to solve the problem a postprocessing function is designed in this paper. After matching cost unique test, the both left-right and right-left consistency check solutions are implemented, respectively; then, the cavities in depth maps can be filled by right depth values on the basis of right-left consistency check solution. The results in the experiments have shown that the depth map extraction and postprocessing function can be implemented in real time in the same system; what is more, the quality of the depth maps is satisfactory.

scholarly journals The performance of landslide susceptibility models critically depends on the quality of digital elevation models

2020 ◽  
Vol 11 (1) ◽  
pp. 1075-1092 ◽  
Author(s):  
Jonas Brock ◽  
Patrick Schratz ◽  
Helene Petschko ◽  
Jannes Muenchow ◽  
Mihai Micu ◽  
...  
Keyword(s):  
Landslide Susceptibility ◽  
Digital Elevation Models ◽  
Digital Elevation

scholarly journals A Residual Network and FPGA Based Real-Time Depth Map Enhancement System

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 546
Author(s):  
Zhenni Li ◽  
Haoyi Sun ◽  
Yuliang Gao ◽  
Jiao Wang
Keyword(s):  
Real Time ◽  
Super Resolution ◽  
Depth Map ◽  
Acquisition System ◽  
Depth Image ◽  
Fpga Design ◽  
Depth Sensing ◽  
Residual Network ◽  
Real Time Processing ◽  
Depth Maps

Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 fps@ (1280 × 960 + 328 × 248 × 3).

Sign in / Sign up

Export Citation Format

Share Document