Depth estimation via light field camera with a hybrid stereo matching method

To fulfill the applications on robot vision, the commonly used stereo matching method for depth estimation is supposed to be efficient in terms of running speed and disparity accuracy. Based on this requirement, Delaunay-based stereo matching method is proposed to achieve the aforementioned standards in this paper. First, a Canny edge operator is used to detect the edge points of an image as supporting points. Those points are then processed using a Delaunay triangulation algorithm to divide the whole image into a series of linked triangular facets. A proposed module composed of these facets performs a rude estimation of image disparity. According to the triangular property of shared vertices, the estimated disparity is then refined to generate the disparity map. The method is tested on Middlebury stereo pairs. The running time of the proposed method is about 1 s and the matching accuracy is 93%. Experimental results show that the proposed method improves both running speed and disparity accuracy, which forms a steady foundation and good application prospect for a robot’s path planning system with stereo camera devices.

Download Full-text

Structured Light Field Generated by Two Projectors for High-Speed Three Dimensional Measurement

Journal of Robotics and Mechatronics ◽

10.20965/jrm.2016.p0523 ◽

2016 ◽

Vol 28 (4) ◽

pp. 523-532 ◽

Cited By ~ 2

Author(s):

Akihiro Obara ◽

◽

Xu Yang ◽

Hiromasa Oku ◽

Keyword(s):

High Speed ◽

Light Field ◽

Stereo Matching ◽

Structured Light ◽

Three Dimensional ◽

Depth Estimation ◽

Image Features ◽

Depth Information ◽

Speed Tracking ◽

Three Dimensional Measurement

[abstFig src='/00280004/10.jpg' width='300' text='Concept of SLF generated by two projectors' ] Triangulation is commonly used to restore 3D scenes, but its frame of less than 30 fps due to time-consuming stereo-matching is an obstacle for applications requiring that results be fed back in real time. The structured light field (SLF) our group proposed previously reduced the amount of calculation in 3D restoration, realizing high-speed measurement. Specifically, the SLF estimates depth information by projecting information on distance directly to a target. The SLF synthesized as reported, however, presents difficulty in extracting image features for depth estimation. In this paper, we propose synthesizing the SLF using two projectors with a certain layout. Our proposed SLF’s basic properties are based on an optical model. We evaluated the SLF’s performance using a prototype we developed and applied to the high-speed depth estimation of a target moving randomly at a speed of 1000 Hz. We demonstrate the target’s high-speed tracking based on high-speed depth information feedback.

Download Full-text

FAST STEREO MATCHING METHOD BASED ON OPTIMIZED CORRELATION ALGORITHM FOR FACE DEPTH ESTIMATION

Proceedings of the International Conference on Computer Vision Theory and Applications ◽

10.5220/0003845703770380 ◽

2012 ◽

Keyword(s):

Stereo Matching ◽

Depth Estimation ◽

Matching Method ◽

Correlation Algorithm

Download Full-text

Baseline-Adaptive Light Field Depth Estimation Based on Stereo Matching

10.1109/iscipt53667.2021.00176 ◽

2021 ◽

Author(s):

Yu Mo ◽

Chao Xiao ◽

Jungang Yang ◽

Wei An

Keyword(s):

Light Field ◽

Stereo Matching ◽

Depth Estimation

Download Full-text

Depth Estimation for Light-Field Images Using Stereo Matching and Convolutional Neural Networks

Sensors ◽

10.3390/s20216188 ◽

2020 ◽

Vol 20 (21) ◽

pp. 6188

Author(s):

Ségolène Rogge ◽

Ionut Schiopu ◽

Adrian Munteanu

Keyword(s):

Machine Learning ◽

Light Field ◽

Stereo Matching ◽

Estimation Method ◽

Similarity Index ◽

Depth Estimation ◽

Residual Error ◽

Disparity Estimation ◽

Machine Learning Techniques ◽

Disparity Map

The paper presents a novel depth-estimation method for light-field (LF) images based on innovative multi-stereo matching and machine-learning techniques. In the first stage, a novel block-based stereo matching algorithm is employed to compute the initial estimation. The proposed algorithm is specifically designed to operate on any pair of sub-aperture images (SAIs) in the LF image and to compute the pair’s corresponding disparity map. For the central SAI, a disparity fusion technique is proposed to compute the initial disparity map based on all available pairwise disparities. In the second stage, a novel pixel-wise deep-learning (DL)-based method for residual error prediction is employed to further refine the disparity estimation. A novel neural network architecture is proposed based on a new structure of layers. The proposed DL-based method is employed to predict the residual error of the initial estimation and to refine the final disparity map. The experimental results demonstrate the superiority of the proposed framework and reveal that the proposed method achieves an average improvement of 15.65% in root mean squared error (RMSE), 43.62% in mean absolute error (MAE), and 5.03% in structural similarity index (SSIM) over machine-learning-based state-of-the-art methods.

Download Full-text

DEPTH ESTIMATION IN LIGHT FIELD CAMERA ARRAYS BASED ON MULTI-STEREO MATCHING AND BELIEF PROPAGATION

2018 - 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON) ◽

10.1109/3dtv.2018.8478503 ◽

2018 ◽

Author(s):

Segolene Rogge ◽

Adrian Munteanu

Keyword(s):

Light Field ◽

Stereo Matching ◽

Belief Propagation ◽

Depth Estimation ◽

Camera Arrays

Download Full-text

A Semi-Supervised Monocular Stereo Matching Method

Symmetry ◽

10.3390/sym11050690 ◽

2019 ◽

Vol 11 (5) ◽

pp. 690

Author(s):

Zhimin Zhang ◽

Jianzhong Qiao ◽

Shukuan Lin

Keyword(s):

Stereo Matching ◽

Ground Truth ◽

Depth Estimation ◽

Estimation Methods ◽

Estimation Accuracy ◽

Stereo Pair ◽

Single Image ◽

Matching Method ◽

Matching Model ◽

Monocular Depth

Supervised monocular depth estimation methods based on learning have shown promising results compared with the traditional methods. However, these methods require a large number of high-quality corresponding ground truth depth data as supervision labels. Due to the limitation of acquisition equipment, it is expensive and impractical to record ground truth depth for different scenes. Compared to supervised methods, the self-supervised monocular depth estimation method without using ground truth depth is a promising research direction, but self-supervised depth estimation from a single image is geometrically ambiguous and suboptimal. In this paper, we propose a novel semi-supervised monocular stereo matching method based on existing approaches to improve the accuracy of depth estimation. This idea is inspired by the experimental results of the paper that the depth estimation accuracy of a stereo pair as input is better than that of a monocular view as input in the same self-supervised network model. Therefore, we decompose the monocular depth estimation problem into two sub-problems, a right view synthesized process followed by a semi-supervised stereo matching process. In order to improve the accuracy of the synthetic right view, we innovate beyond the existing view synthesis method Deep3D by adding a left-right consistency constraint and a smoothness constraint. To reduce the error caused by the reconstructed right view, we propose a semi-supervised stereo matching model that makes use of disparity maps generated by a self-supervised stereo matching model as the supervision cues and joint self-supervised cues to optimize the stereo matching network. In the test, the two networks are able to predict the depth map directly from a single image by pipeline connecting. Both procedures not only obey geometric principles, but also improve estimation accuracy. Test results on the KITTI dataset show that this method is superior to the current mainstream monocular self-supervised depth estimation methods under the same condition.

Download Full-text

Spatial Detection of Vehicles in Images using Convolutional Neural Networks and Stereo Matching

Journal of Computational Vision and Imaging Systems ◽

10.15353/vsnl.v2i1.101 ◽

2016 ◽

Vol 2 (1) ◽

Author(s):

Jeremy Pinto ◽

Nolan Lunscher ◽

Georges Younes ◽

David Abou Chacra ◽

Henry Leopold ◽

...

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Stereo Matching ◽

Mean Absolute Error ◽

Depth Estimation ◽

Absolute Error ◽

Stereo Images ◽

Matching Method ◽

Spatial Detection ◽

State Of Art

Convolutional Neural Networks combined with a state of the artstereo-matching method are used to find and estimate the 3D positionof vehicles in pairs of stereo images. Pixel positions of vehiclesare first estimated separately in pairs of stereo images usinga Convolutional Neural Network for regression. These coordinatesare then combined with a state-of-art stereo-matching method todetermine the depth, and thus the 3D location, of the vehicles. Weshow in this paper that cars can be detected with a combined accuracyof approximately 90% with a tolerated radius error of 5%,and a Mean Absolute Error of 5.25m on depth estimation for carsup to 50m away.

Download Full-text