scholarly journals A Joint 2D-3D Complementary Network for Stereo Matching

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1430
Author(s):  
Xiaogang Jia ◽  
Wei Chen ◽  
Zhengfa Liang ◽  
Xin Luo ◽  
Mingfei Wu ◽  
...  
Keyword(s):  
Stereo Matching ◽  
Computational Cost ◽  
Research Field ◽  
Disparity Map ◽  
Improve Performance ◽  
Cost Aggregation ◽  
Disparity Range ◽  
Public Datasets ◽  
Coarse To Fine ◽  
Speed And Accuracy

Stereo matching is an important research field of computer vision. Due to the dimension of cost aggregation, current neural network-based stereo methods are difficult to trade-off speed and accuracy. To this end, we integrate fast 2D stereo methods with accurate 3D networks to improve performance and reduce running time. We leverage a 2D encoder-decoder network to generate a rough disparity map and construct a disparity range to guide the 3D aggregation network, which can significantly improve the accuracy and reduce the computational cost. We use a stacked hourglass structure to refine the disparity from coarse to fine. We evaluated our method on three public datasets. According to the KITTI official website results, Our network can generate an accurate result in 80 ms on a modern GPU. Compared to other 2D stereo networks (AANet, DeepPruner, FADNet, etc.), our network has a big improvement in accuracy. Meanwhile, it is significantly faster than other 3D stereo networks (5× than PSMNet, 7.5× than CSN and 22.5× than GANet, etc.), demonstrating the effectiveness of our method.

A Parallel Stereo Matching Algorithm Core for FPGA Modeled by DSP Builder

2014 ◽  
Vol 536-537 ◽  
pp. 67-76
Author(s):  
Xiang Zhang ◽  
Zhang Wei Chen
Keyword(s):  
Stereo Matching ◽  
Cost Calculation ◽  
Stereo Pair ◽  
Disparity Map ◽  
High Definition ◽  
Matching Algorithm ◽  
Census Transform ◽  
Cost Aggregation ◽  
Dsp Builder ◽  
Fpga Chip

This paper proposes a FPGA implementation to apply a stereo matching algorithm based on a kind of sparse census transform in a FPGA chip which can provide a high-definition dense disparity map in real-time. The parallel stereo matching algorithm core involves census transform, cost calculation and cost aggregation modules. The circuits of the algorithm core are modeled by the Matlab/Simulink-based tool box: DSP Builder. The system can process many different sizes of stereo pair images through a configuration interface. The maximum horizon resolution of stereo images is 2048.

scholarly journals Improved Cost Computation and Adaptive Shape Guided Filter for Local Stereo Matching of Low Texture Stereo Images

2020 ◽  
Vol 10 (5) ◽  
pp. 1869
Author(s):  
Hua Liu ◽  
Rui Wang ◽  
Yuanping Xia ◽  
Xiaoming Zhang
Keyword(s):  
Cost Function ◽  
Stereo Matching ◽  
Guided Filter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Dense Stereo Matching ◽  
Gradient Based ◽  
Matching Cost ◽  
Cost Measurement ◽  
Image Pairs

Dense stereo matching has been widely used in photogrammetry and computer vision applications. Even though it has a long research history, dense stereo matching is still challenging for occluded, textureless and discontinuous regions. This paper proposed an efficient and effective matching cost measurement and an adaptive shape guided filter-based matching cost aggregation method to improve the stereo matching performance for large textureless regions. At first, an efficient matching cost function combining enhanced image gradient-based matching cost and improved census transform-based matching cost is introduced. This proposed matching cost function is robust against radiometric variations and textureless regions. Following this, an adaptive shape cross-based window is constructed for each pixel and a modified guided filter based on this adaptive shape window is implemented for cost aggregation. The final disparity map is obtained after disparity selection and multiple steps disparity refinement. Experiments were conducted on the Middlebury benchmark dataset to evaluate the effectiveness of the proposed cost measurement and cost aggregation strategy. The experimental results demonstrated that the average matching error rate on Middlebury standard image pairs is 9.40%. Compared with the traditional guided filter-based stereo matching method, the proposed method achieved a better matching result in textureless regions.

scholarly journals HYBRID-BASED DENSE STEREO MATCHING

2016 ◽  
Vol XLI-B3 ◽  
pp. 495-501 ◽  
Author(s):  
T. Y. Chuang ◽  
H. W. Ting ◽  
J. J. Jaw
Keyword(s):  
Stereo Matching ◽  
Penalty Parameter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Satisfactory Performance ◽  
Dense Stereo Matching ◽  
Illumination Changes ◽  
Global Matching ◽  
Disparity Maps ◽  
Image Edges

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

scholarly journals HYBRID-BASED DENSE STEREO MATCHING

2016 ◽  
Vol XLI-B3 ◽  
pp. 495-501
Author(s):  
T. Y. Chuang ◽  
H. W. Ting ◽  
J. J. Jaw
Keyword(s):  
Stereo Matching ◽  
Penalty Parameter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Satisfactory Performance ◽  
Dense Stereo Matching ◽  
Illumination Changes ◽  
Global Matching ◽  
Disparity Maps ◽  
Image Edges

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

scholarly journals Fast global stereo matching via energy pyramid minimization

2014 ◽  
Vol II-3 ◽  
pp. 41-48
Author(s):  
B. Conejo ◽  
S. Leprince ◽  
F. Ayoub ◽  
J. P. Avouac
Keyword(s):  
Random Fields ◽  
Stereo Matching ◽  
Conditional Random Fields ◽  
Optimal Solution ◽  
Stereo Pair ◽  
Optimization Approach ◽  
Disparity Map ◽  
Image Pyramids ◽  
Global Matching ◽  
Coarse To Fine

We define a global matching framework based on energy pyramid, the Global Matching via Energy Pyramid (GM-EP) algorithm, which estimates the disparity map from a single stereo-pair by solving an energy minimization problem. We efficiently address this minimization by globally optimizing a coarse to fine sequence of sparse Conditional Random Fields (CRF) directly defined on the energy. This global discrete optimization approach guarantees that at each scale we obtain a near optimal solution, and we demonstrate its superiority over state of the art image pyramid approaches through application to real stereo-pairs. We conclude that multiscale approaches should be build on energy pyramids rather than on image pyramids.

scholarly journals Accurate hierarchical stereo matching based on 3D plane labeling of superpixel for stereo images from rovers

2021 ◽  
Vol 18 (2) ◽  
pp. 172988142110021
Author(s):  
Haichao Li ◽  
Zhi Li ◽  
Jianbin Huang ◽  
Bo Meng ◽  
Zhimin Zhang
Keyword(s):  
Large Scale ◽  
Stereo Matching ◽  
State Of The Art ◽  
The State ◽  
Normal Vector ◽  
Stereo Images ◽  
Disparity Map ◽  
Matching Method ◽  
Search Range ◽  
Coarse To Fine

An accurate hierarchical stereo matching method is proposed based on continuous 3D plane labeling of superpixel for rover’s stereo images. This method can infer the 3D plane label of each pixel combined with the slanted-patch matching strategy and coarse-to-fine constraints, which is especially suitable for large-scale scene matching with low-texture or textureless regions. At every level, the stereo matching method based on superpixel segmentation makes the iteration convergence faster and avoids huge redundant computations. In the coarse-to-fine matching scheme, we propose disparity constraint and 3D normal vector constraint between adjacent levels through which the disparity map and 3D normal vector map at a coarser level are used to restrict the search range of disparity and normal vector at a fine level. The experimental results with the Chang’e-3 rover dataset and the KITTI dataset show that the proposed stereo matching method is efficiently and accurately compared with the state-of-the-art 3D labeling algorithm, especially in low-texture or textureless regions. The computational efficiency of this method is about five to six times faster than the state-of-the-art 3D labeling method, and the accuracy is better.

scholarly journals An Efficient Stereo Matching Network Using Sequential Feature Fusion

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1045
Author(s):  
Jaecheol Jeong ◽  
Suyeon Jeon ◽  
Yong Seok Heo
Keyword(s):  
Building Block ◽  
Stereo Matching ◽  
Feature Fusion ◽  
Mobile Environments ◽  
Disparity Map ◽  
Matching Network ◽  
Main Building ◽  
Matching Networks ◽  
Disparity Range ◽  
The Cost

Recent stereo matching networks adopt 4D cost volumes and 3D convolutions for processing those volumes. Although these methods show good performance in terms of accuracy, they have an inherent disadvantage in that they require great deal of computing resources and memory. These requirements limit their applications for mobile environments, which are subject to inherent computing hardware constraints. Both accuracy and consumption of computing resources are important, and improving both at the same time is a non-trivial task. To deal with this problem, we propose a simple yet efficient network, called Sequential Feature Fusion Network (SFFNet) which sequentially generates and processes the cost volume using only 2D convolutions. The main building block of our network is a Sequential Feature Fusion (SFF) module which generates 3D cost volumes to cover a part of the disparity range by shifting and concatenating the target features, and processes the cost volume using 2D convolutions. A series of the SFF modules in our SFFNet are designed to gradually cover the full disparity range. Our method prevents heavy computations and allows for efficient generation of an accurate final disparity map. Various experiments show that our method has an advantage in terms of accuracy versus efficiency compared to other networks.

Detail Preserving Coarse-to-Fine Matching for Stereo Matching and Optical Flow

2021 ◽  
pp. 1-1
Author(s):  
Yong Deng ◽  
Jimin Xiao ◽  
Steven Zhiying Zhou ◽  
Jiashi Feng
Keyword(s):  
Optical Flow ◽  
Stereo Matching ◽  
Coarse To Fine

scholarly journals An Edge-Sense Bidirectional Pyramid Network for Stereo Matching of VHR Remote Sensing Images

2020 ◽  
Vol 12 (24) ◽  
pp. 4025
Author(s):  
Rongshu Tao ◽  
Yuming Xiang ◽  
Hongjian You
Keyword(s):  
Remote Sensing ◽  
Stereo Matching ◽  
Tall Buildings ◽  
Disparity Estimation ◽  
Complex Structures ◽  
Learning Networks ◽  
Remote Sensing Images ◽  
Essential Step ◽  
Disparity Range ◽  
The Cost

As an essential step in 3D reconstruction, stereo matching still faces unignorable problems due to the high resolution and complex structures of remote sensing images. Especially in occluded areas of tall buildings and textureless areas of waters and woods, precise disparity estimation has become a difficult but important task. In this paper, we develop a novel edge-sense bidirectional pyramid stereo matching network to solve the aforementioned problems. The cost volume is constructed from negative to positive disparities since the disparity range in remote sensing images varies greatly and traditional deep learning networks only work well for positive disparities. Then, the occlusion-aware maps based on the forward-backward consistency assumption are applied to reduce the influence of the occluded area. Moreover, we design an edge-sense smoothness loss to improve the performance of textureless areas while maintaining the main structure. The proposed network is compared with two baselines. The experimental results show that our proposed method outperforms two methods, DenseMapNet and PSMNet, in terms of averaged endpoint error (EPE) and the fraction of erroneous pixels (D1), and the improvements in occluded and textureless areas are significant.

Sign in / Sign up

Export Citation Format

Share Document