scholarly journals Enhanced Soft 3D Reconstruction Method with an Iterative Matching Cost Update Using Object Surface Consensus

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6680
Author(s):  
Min-Jae Lee ◽  
Gi-Mun Um ◽  
Joungil Yun ◽  
Won-Sik Cheong ◽  
Soon-Yong Park
Keyword(s):  
3D Reconstruction ◽  
Stereo Matching ◽  
View Synthesis ◽  
Depth Image ◽  
Reconstruction Method ◽  
Object Surface ◽  
Plane Sweep ◽  
Depth Images ◽  
Research Areas ◽  
Matching Cost

In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.

scholarly journals AN AUTOMATIC 3D RECONSTRUCTION METHOD BASED ON MULTI-VIEW STEREO VISION FOR THE MOGAO GROTTOES

2015 ◽  
Vol XL-4/W5 ◽  
pp. 171-176 ◽  
Author(s):  
J. Xiong ◽  
S. Zhong ◽  
L. Zheng
Keyword(s):  
3D Reconstruction ◽  
Stereo Vision ◽  
Measurement System ◽  
Stereo Matching ◽  
Three Dimensional ◽  
Texture Mapping ◽  
Reconstruction Method ◽  
Epipolar Constraint ◽  
Mogao Grottoes ◽  
Binocular Stereo

This paper presents an automatic three-dimensional reconstruction method based on multi-view stereo vision for the Mogao Grottoes. 3D digitization technique has been used in cultural heritage conservation and replication over the past decade, especially the methods based on binocular stereo vision. However, mismatched points are inevitable in traditional binocular stereo matching due to repeatable or similar features of binocular images. In order to reduce the probability of mismatching greatly and improve the measure precision, a portable four-camera photographic measurement system is used for 3D modelling of a scene. Four cameras of the measurement system form six binocular systems with baselines of different lengths to add extra matching constraints and offer multiple measurements. Matching error based on epipolar constraint is introduced to remove the mismatched points. Finally, an accurate point cloud can be generated by multi-images matching and sub-pixel interpolation. Delaunay triangulation and texture mapping are performed to obtain the 3D model of a scene. The method has been tested on 3D reconstruction several scenes of the Mogao Grottoes and good results verify the effectiveness of the method.

scholarly journals Three-Dimensional Reconstruction Method of Rapeseed Plants in the Whole Growth Period Using RGB-D Camera

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4628
Author(s):  
Xiaowen Teng ◽  
Guangsheng Zhou ◽  
Yuxuan Wu ◽  
Chenglong Huang ◽  
Wanjing Dong ◽  
...  
Keyword(s):  
3D Reconstruction ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Depth Image ◽  
Scanning System ◽  
Reconstruction Method ◽  
Three Dimensional Reconstruction ◽  
Hardware Cost ◽  
Dimensional Reconstruction

The three-dimensional reconstruction method using RGB-D camera has a good balance in hardware cost and point cloud quality. However, due to the limitation of inherent structure and imaging principle, the acquired point cloud has problems such as a lot of noise and difficult registration. This paper proposes a 3D reconstruction method using Azure Kinect to solve these inherent problems. Shoot color images, depth images and near-infrared images of the target from six perspectives by Azure Kinect sensor with black background. Multiply the binarization result of the 8-bit infrared image with the RGB-D image alignment result provided by Microsoft corporation, which can remove ghosting and most of the background noise. A neighborhood extreme filtering method is proposed to filter out the abrupt points in the depth image, by which the floating noise point and most of the outlier noise will be removed before generating the point cloud, and then using the pass-through filter eliminate rest of the outlier noise. An improved method based on the classic iterative closest point (ICP) algorithm is presented to merge multiple-views point clouds. By continuously reducing both the size of the down-sampling grid and the distance threshold between the corresponding points, the point clouds of each view are continuously registered three times, until get the integral color point cloud. Many experiments on rapeseed plants show that the success rate of cloud registration is 92.5% and the point cloud accuracy obtained by this method is 0.789 mm, the time consuming of a integral scanning is 302 seconds, and with a good color restoration. Compared with a laser scanner, the proposed method has considerable reconstruction accuracy and a significantly ahead of the reconstruction speed, but the hardware cost is much lower when building a automatic scanning system. This research shows a low-cost, high-precision 3D reconstruction technology, which has the potential to be widely used for non-destructive measurement of rapeseed and other crops phenotype.

scholarly journals 3D Reconstruction Algorithm of Weld Pool Surface Based on Computer Vision Technology

2015 ◽  
Vol 9 (1) ◽  
pp. 820-825
Author(s):  
Zhen-Hai Mu
Keyword(s):  
3D Reconstruction ◽  
Weld Pool ◽  
Stereo Matching ◽  
Vision System ◽  
Reconstruction Algorithm ◽  
Welding Parameters ◽  
Matching Algorithm ◽  
Pool Surface ◽  
Matching Cost ◽  
Weld Pool Surface

As is well known that sensing and measuring the weld pool surface is very important to design intelligent welding machines which is able to imitate a skilled human welder who can choose suitable welding parameters. Therefore, in this paper, we focused on the problem of weld pool surface 3D reconstruction, which is a key issue in intelligent welding machines development. Firstly, the framework of the weld pool surface 3D reconstruction system is described. The weld pool surface 3D reconstruction system uses a single camera stereo vision system to extract original data from weld pool, and then the left and right images are collected. Afterward, we utilize Pixel difference square and matching algorithm and Stereo matching algorithm to process images. Next, the 3D reconstruction of weld pool surface is constructed using the point cloud data. Secondly, stereo matching based weld pool surface 3D reconstruction algorithm is illustrated. In this algorithm, the matching cost function is computed through the Markov random field, and then the weighted matching cost is calculated via the guided filter. Thirdly, to test the performance of our proposed algorithm, we develop an experimental platform to measure weld pool width, length, convexity and the previous inputs based on a linear model predictive controller. Experimental results demonstrate that the proposed 3D reconstruction algorithm of weld pool surface can achieve high quality under both current disturbance and speed disturbance.

scholarly journals RobotP: A Benchmark Dataset for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1299
Author(s):  
Honglin Yuan ◽  
Tim Hoogenkamp ◽  
Remco C. Veltkamp
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
3D Models ◽  
Depth Image ◽  
Great Success ◽  
Estimation Algorithms ◽  
Depth Images ◽  
Object Pose Estimation ◽  
Image Pairs ◽  
Bounding Boxes

Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.

scholarly journals HRDepthNet: Depth Image-Based Marker-Less Tracking of Body Joints

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1356
Author(s):  
Linda Christin Büker ◽  
Finnja Zuber ◽  
Andreas Hein ◽  
Sebastian Fudickar
Keyword(s):  
Color Images ◽  
Depth Image ◽  
Accuracy Evaluation ◽  
Timed Up And Go ◽  
Position Errors ◽  
Depth Images ◽  
Upper And Lower Extremities ◽  
Rgb Images ◽  
Human Joints ◽  
Body Joints

With approaches for the detection of joint positions in color images such as HRNet and OpenPose being available, consideration of corresponding approaches for depth images is limited even though depth images have several advantages over color images like robustness to light variation or color- and texture invariance. Correspondingly, we introduce High- Resolution Depth Net (HRDepthNet)—a machine learning driven approach to detect human joints (body, head, and upper and lower extremities) in purely depth images. HRDepthNet retrains the original HRNet for depth images. Therefore, a dataset is created holding depth (and RGB) images recorded with subjects conducting the timed up and go test—an established geriatric assessment. The images were manually annotated RGB images. The training and evaluation were conducted with this dataset. For accuracy evaluation, detection of body joints was evaluated via COCO’s evaluation metrics and indicated that the resulting depth image-based model achieved better results than the HRNet trained and applied on corresponding RGB images. An additional evaluation of the position errors showed a median deviation of 1.619 cm (x-axis), 2.342 cm (y-axis) and 2.4 cm (z-axis).

3D RECONSTRUCTION OF BREAST CANCER FROM MAMMOGRAMS USING VOLUME RENDERING TECHNIQUES

2015 ◽  
Vol 75 (2) ◽  
Author(s):  
Ho Wei Yong ◽  
Abdullah Bade ◽  
Rajesh Kumar Muniandy
Keyword(s):  
Breast Cancer ◽  
3D Reconstruction ◽  
Computational Cost ◽  
Reconstruction Method ◽  
Cancer Model ◽  
Volume Reconstruction ◽  
The Past ◽  
Breast Cancer Model ◽  
Reconstruction Software ◽  
To Come

Over the past thirty years, a number of researchers have investigated on 3D organ reconstruction from medical images and there are a few 3D reconstruction software available on the market. However, not many researcheshave focused on3D reconstruction of breast cancer’s tumours. Due to the method complexity, most 3D breast cancer’s tumours reconstruction were done based on MRI slices dataeven though mammogram is the current clinical practice for breast cancer screening. Therefore, this research will investigate the process of creating a method that will be able to reconstruct 3D breast cancer’s tumours from mammograms effectively.  Several steps were proposed for this research which includes data acquisition, volume reconstruction, andvolume rendering. The expected output from this research is the 3D breast cancer’s tumours model that is generated from correctly registered mammograms. The main purpose of this research is to come up with a 3D reconstruction method that can produce good breast cancer model from mammograms while using minimal computational cost.

Sign in / Sign up

Export Citation Format

Share Document