A Disparity Refinement in Stereo Matching based on Mean-shift Segmentation and Spatiotemporal Domain

2020 ◽  
Vol 64 (2) ◽  
pp. 20505-1-20505-12
Author(s):  
Hui-Yu Huang ◽  
Zhe-Hao Liu
Keyword(s):  
Stereo Matching ◽  
Mean Shift ◽  
Disparity Map ◽  
Video Streams ◽  
Segmentation Strategy ◽  
Information Matching ◽  
Color Similarity ◽  
Spatial Domains ◽  
Disparity Maps ◽  
The Cost

Abstract A stereo matching algorithm is used to find the best match between a pair of images. To compute the cost of the matching points from the sequence of images, the disparity maps from video streams are estimated. However, the estimated disparity sequences may cause undesirable flickering errors. These errors result in low visibility of the synthesized video and reduce video coding. In order to solve this problem, in this article, the authors propose a spatiotemporal disparity refinement on local stereo matching based on the segmentation strategy. Based on segmentation information, matching point searching, and color similarity, adaptive disparity values to recover the disparity errors in disparity sequences can be obtained. The flickering errors are also effectively removed, and the boundaries of objects are well preserved. The procedures of the proposed approach consist of a segmentation process, matching point searching, and refinement in the temporal and spatial domains. Experimental results verify that the proposed approach can yield a high quantitative evaluation and a high-quality disparity map compared with other methods.

scholarly journals Stereo Matching with Spatiotemporal Disparity Refinement Using Simple Linear Iterative Clustering Segmentation

Electronics ◽  
2021 ◽  
Vol 10 (6) ◽  
pp. 717
Author(s):  
Hui-Yu Huang ◽  
Zhe-Hao Liu
Keyword(s):  
Outlier Detection ◽  
Stereo Matching ◽  
3D Visualization ◽  
Signal To Noise Ratio ◽  
Three Dimensional ◽  
Segmentation Strategy ◽  
Color Similarity ◽  
Simple Linear Iterative Clustering ◽  
Disparity Maps ◽  
Public Datasets

Stereo matching is a challenging problem, especially for computer vision, e.g., three-dimensional television (3DTV) or 3D visualization. The disparity maps from the video streams must be estimated. However, the estimated disparity sequences may cause undesirable flickering errors. These errors result in poor visual quality for the synthesized video and reduce the video coding information. In order to solve this problem, we here propose a spatiotemporal disparity refinement method for local stereo matching using the simple linear iterative clustering (SLIC) segmentation strategy, outlier detection, and refinements of the temporal and spatial domains. In the outlier detection, the segmented region in the initial disparity is used to distinguish errors in the binocular disparity. Based on the color similarity and disparity difference, we recalculate the aggregated cost to determine adaptive disparities to recover the disparity errors in disparity sequences. The flickering errors are also effectively removed, and the object boundaries are well preserved. Experiments using public datasets demonstrated that our proposed method creates high-quality disparity maps and obtains a high peak signal-to-noise ratio compared to state-of-the-art methods.

scholarly journals mproved Stereo Matching Algorithm based on Census Transform and Dynamic Histogram Cost Computation

2021 ◽  
Vol 11 (8) ◽  
pp. 48-57
Author(s):  
A. F. Kadmin ◽  
◽  
R. A. Hamzah ◽  
M. N. Abd Manap ◽  
M. S. Hamid ◽  
...  
Keyword(s):  
Stereo Matching ◽  
Disparity Map ◽  
Local Method ◽  
Matching Algorithm ◽  
Census Transform ◽  
Traditional Taxonomy ◽  
Matching Cost ◽  
Winner Takes All ◽  
The Cost ◽  
Vision Algorithm

Stereo matching is a significant subject in the stereo vision algorithm. Traditional taxonomy composition consists of several issues in the stereo correspondences process such as radiometric distortion, discontinuity, and low accuracy at the low texture regions. This new taxonomy improves the local method of stereo matching algorithm based on the dynamic cost computation for disparity map measurement. This method utilised modified dynamic cost computation in the matching cost stage. A modified Census Transform with dynamic histogram is used to provide the cost volume. An adaptive bilateral filtering is applied to retain the image depth and edge information in the cost aggregation stage. A Winner Takes All (WTA) optimisation is applied in the disparity selection and a left-right check with an adaptive bilateral median filtering are employed for final refinement. Based on the dataset of standard Middlebury, the taxonomy has better accuracy and outperformed several other state-ofthe-art algorithms. Keywords—Stereo matching, disparity map, dynamic cost, census transform, local method

A New Basic Correlation Measurement for Stereo Matching

2019 ◽  
Author(s):  
Bowen Shi ◽  
Shan Shi ◽  
Junhua Wu ◽  
Musheng Chen
Keyword(s):  
Time Complexity ◽  
Stereo Matching ◽  
Computation Method ◽  
Local Correlation ◽  
Matching Algorithm ◽  
Cost Aggregation ◽  
Image Patches ◽  
Disparity Maps ◽  
Aggregation Step ◽  
The Cost

In this paper, we propose a new stereo matching algorithm to measure the correlation between two rectified image patches. The difficulty near objects' boundaries and textureless areas is a widely discussed issue in local correlation-based algorithms and most approaches focus on the cost aggregation step to solve the problem. We analyze the inherent limitations of sum of absolute differences (SAD) and sum of squared differences (SSD), then propose a new difference computation method to restrain the noise near objects' boundaries and enlarge the intensity variations in textureless areas. The proposed algorithm can effectively deal with the problems and generate more accurate disparity maps than SAD and SSD without time complexity increasing. Furthermore, proved by experiments, the algorithm can also be applied in some SAD-based and SSD-based algorithms to achieve better results than the original.

COMPUTATION AND PERFORMANCE ANALYSIS OF DOUBLE STAGE FILTER FOR IMAGE PROCESSING

2015 ◽  
Vol 77 (19) ◽  
Author(s):  
Teo Chee Huat ◽  
Nurulfajar Abdul Manap ◽  
Masrullizam Mat Ibrahim
Keyword(s):  
Stereo Matching ◽  
Ground Truth ◽  
Block Matching ◽  
Basic Block ◽  
Median Filtering ◽  
Disparity Map ◽  
Evaluation Functions ◽  
Map Segmentation ◽  
Disparity Maps ◽  
And Performance

Double Stage Filter (DSF) is a hybrid stereo matching algorithm which consists of basic block matching and dynamic programming algorithms, basic median filtering and new technique of segmentation. The algorithm acquire disparity maps which will be analyzed by using evaluation functions such as PSNR, MSE and SSIM. The computation of DSF and existing algorithms are presented in this paper. The Phase 2 in DSF is to remove the unwanted aspects such as depth discontinuities and holes from occlusion from the raw disparity map. Segmentation, merging and median filtering are the major parts for post processing of DSF algorithm. From the results of evaluation functions, the disparity maps attained by DSF is closer to the ground truth compared to other algorithms while its computation takes only few seconds longer than DP algorithm but its capable to obtain better results of disparity map.

scholarly journals A Survey of Learning Approaches and Application for 3D Vision

2018 ◽  
Vol 173 ◽  
pp. 03053
Author(s):  
Luanhao Lu
Keyword(s):  
Stereo Matching ◽  
Mean Shift ◽  
Three Dimensional ◽  
Image Synthesis ◽  
Learning Approaches ◽  
Median Filtering ◽  
Stereo Images ◽  
Disparity Map ◽  
3D Vision ◽  
2D Images

Three-dimensional (3D) vision extracted from the stereo images or reconstructed from the two-dimensional (2D) images is the most effective topic in computer vision and video surveillance. Three-dimensional scene is constructed through two stereo images which existing disparity map by Stereo vision. Many methods of Stereo matching which contains median filtering, mean-shift segmentation, guided filter and joint trilateral filters [1] are used in many algorithms to construct the precise disparity map. These methods committed to figure out the image synthesis range in different Stereo matching fields and among these techniques cannot perform perfectly every turn. The paper focuses on 3D vision, introduce the background and process of 3D vision, reviews several classical datasets in the field of 3D vision, based on which the learning approaches and several types of applications of 3D vision were evaluated and analyzed.

scholarly journals A Line-Based Adaptive-Weight Matching Algorithm Using Loopy Belief Propagation

2015 ◽  
Vol 2015 ◽  
pp. 1-13
Author(s):  
Hui Li ◽  
Xiao-Guang Zhang ◽  
Zheng Sun
Keyword(s):  
Stereo Matching ◽  
Belief Propagation ◽  
Mean Shift ◽  
Spatial Proximity ◽  
Consistency Checking ◽  
Disparity Map ◽  
Matching Effect ◽  
Matching Algorithm ◽  
Adaptive Weight ◽  
Loopy Belief Propagation

In traditional adaptive-weight stereo matching, the rectangular shaped support region requires excess memory consumption and time. We propose a novel line-based stereo matching algorithm for obtaining a more accurate disparity map with low computation complexity. This algorithm can be divided into two steps: disparity map initialization and disparity map refinement. In the initialization step, a new adaptive-weight model based on the linear support region is put forward for cost aggregation. In this model, the neural network is used to evaluate the spatial proximity, and the mean-shift segmentation method is used to improve the accuracy of color similarity; the Birchfield pixel dissimilarity function and the census transform are adopted to establish the dissimilarity measurement function. Then the initial disparity map is obtained by loopy belief propagation. In the refinement step, the disparity map is optimized by iterative left-right consistency checking method and segmentation voting method. The parameter values involved in this algorithm are determined with many simulation experiments to further improve the matching effect. Simulation results indicate that this new matching method performs well on standard stereo benchmarks and running time of our algorithm is remarkably lower than that of algorithm with rectangle-shaped support region.

scholarly journals HYBRID-BASED DENSE STEREO MATCHING

2016 ◽  
Vol XLI-B3 ◽  
pp. 495-501 ◽  
Author(s):  
T. Y. Chuang ◽  
H. W. Ting ◽  
J. J. Jaw
Keyword(s):  
Stereo Matching ◽  
Penalty Parameter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Satisfactory Performance ◽  
Dense Stereo Matching ◽  
Illumination Changes ◽  
Global Matching ◽  
Disparity Maps ◽  
Image Edges

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

scholarly journals HYBRID-BASED DENSE STEREO MATCHING

2016 ◽  
Vol XLI-B3 ◽  
pp. 495-501
Author(s):  
T. Y. Chuang ◽  
H. W. Ting ◽  
J. J. Jaw
Keyword(s):  
Stereo Matching ◽  
Penalty Parameter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Satisfactory Performance ◽  
Dense Stereo Matching ◽  
Illumination Changes ◽  
Global Matching ◽  
Disparity Maps ◽  
Image Edges

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

scholarly journals Application of morphological processing methods in computer stereo vision

2021 ◽  
Vol 6 (131) ◽  
pp. 18-27
Author(s):  
Oleh Prokopchuk ◽  
Serhii Vovk
Keyword(s):  
Mathematical Morphology ◽  
Stereo Vision ◽  
Real World ◽  
Stereo Matching ◽  
Morphological Processing ◽  
Morphological Operations ◽  
Disparity Map ◽  
Real Objects ◽  
Disparity Maps ◽  
The Impact

Computer vision algorithms are important for many areas of human activity. In particular, the number of applications related to the need to process images of real-world objects with computerized tools and the subsequent use of descriptive information in a variety of interactive and automated decision-making systems is increased. An important tool for analyzing real-world scenes are approaches to the application of stereo vision algorithms. The important step of many stereo matching algorithms is a disparity map. Depending on the content of the observed scene, part of the values on the disparity map can be immediately attributed to background values on a certain basis, or form a "natural" background, which is characterized by loss of informative data due to unacceptable error of subsequent resultant distance values. The calculated disparity map of any algorithm may contain some shortcomings in the form of discontinuities of continuous information areas caused by the complexity of shooting conditions, the impact of noise of various natures, hardware imperfections, and so on. An approach to mitigating the undesirable influence of negative factors on the resulting disparity is the use of mathematical morphology operations to process disparity maps at the post-processing stage. This paper presents information technology for increasing the content of disparity maps based on the mathematical morphology methods. The technology is based on a combination of morphological operations of erosion and dilation, which eliminates the typical problems of discontinuities of monotone regions and erroneous values on disparity maps. The proposed approach allows reducing the impact of common problems that arise during the operation of stereo matching algorithms, as well as increase the overall informativeness of disparity maps for images of real objects in the absence of partial or complete initial data on the characteristics of the observed scene. The results of testing morphological operations with disparity maps for real objects allow us to conclude about the possibility of partial restoration of areas of disparity maps with gaps in continuous information areas, as well as to reduce the impact of random anomalous values on the overall content of the disparity maps.

scholarly journals AMBIGUITY CONCEPT IN STEREO MATCHING PIPELINE

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 383-390
Author(s):  
E. Sarrazin ◽  
M. Cournet ◽  
L. Dumas ◽  
V. Defonte ◽  
Q. Fardet ◽  
...  
Keyword(s):  
Stereo Matching ◽  
Performance Estimation ◽  
Cost Curve ◽  
Disparity Map ◽  
Integral Measure ◽  
Global Matching ◽  
Good Potential ◽  
The Cost ◽  
The Impact

Abstract. In a 3D reconstruction pipeline, stereo matching step aims at computing a disparity map representing the depth between image pair. The evaluation of the disparity map can be done through the estimation of a confidence metric. In this article, we propose a new confidence metric, named ambiguity integral metric, to assess the quality of the produced disparity map. This metric is derived from the concept of ambiguity, which characterizes the property of the cost curve profile. It aims to quantify the difficulty in identifying the correct disparity to select. The quality of ambiguity integral metric is evaluated through the ROC curve methodology and compared with other confidence measures. In regards to other measures, the ambiguity integral measure shows a good potential. We also integrate this measure through various steps of the stereo matching pipeline in order to improve the performance estimation of the disparity map. First, we include ambiguity integral measure during the Semi Global Matching optimization step. The objective is to weight, by ambiguity integral measure, the influence of points in the SGM regularization to reduce the impact of ambiguous points. Secondly, we use ambiguity as an input of a disparity refinement deep learning architecture in order to easily locate noisy area and preserve details.

Sign in / Sign up

Export Citation Format

Share Document