Refraction-Based Bundle Adjustment for Scale Reconstructible Structure from Motion

Structure from motion is a three-dimensional (3D) reconstruction method that uses one camera. However, the absolute scale of objects cannot be reconstructed by the conventional structure from motion method. In our previous studies, to solve this problem by using refraction, we proposed a scale reconstructible structure from motion method. In our measurement system, a refractive plate is fixed in front of a camera and images are captured through this plate. To overcome the geometrical constraints, we derived an extended essential equation by theoretically considering the effect of refraction. By applying this formula to 3D measurements, the absolute scale of an object could be obtained. However, this method was verified only by a simulation under ideal conditions, for example, by not taking into account real phenomena such as noise or occlusion, which are necessarily caused in actual measurements. In this study, to robustly apply this method to an actual measurement with real images, we introduced a novel bundle adjustment method based on the refraction effect. This optimization technique can reduce the 3D reconstruction errors caused by measurement noise in actual scenes. In particular, we propose a new error function considering the effect of refraction. By minimizing the value of this error function, accurate 3D reconstruction results can be obtained. To evaluate the effectiveness of the proposed method, experiments using both simulations and real images were conducted. The results of the simulation show that the proposed method is theoretically accurate. The results of the experiments using real images show that the proposed method is effective for real 3D measurements.

Download Full-text

CROWD-SOURCED VISUAL DATA COLLECTION FOR MONITORING INDOOR CONSTRUCTION IN 3D

Proceedings of International Structural Engineering and Construction ◽

10.14455/isec.res.2018.15 ◽

2018 ◽

Vol 5 (1) ◽

Author(s):

Fouad Amer ◽

Mani Golparvar-Fard

Keyword(s):

Data Collection ◽

3D Reconstruction ◽

Structure From Motion ◽

Reconstruction Algorithm ◽

Point Clouds ◽

Reconstruction Method ◽

Visual Data ◽

Mapping Algorithm ◽

3D Point Clouds ◽

Localization And Mapping

Complete and accurate 3D monitoring of indoor construction progress using visual data is challenging. It requires (a) capturing a large number of overlapping images, which is time-consuming and labor-intensive to collect, and (b) processing using Structure from Motion (SfM) algorithms, which can be computationally expensive. To address these inefficiencies, this paper proposes a hybrid SfM-SLAM 3D reconstruction algorithm along with a decentralized data collection workflow to map indoor construction work locations in 3D and any desired frequency. The hybrid 3D reconstruction method is composed of a pipeline of Structure from Motion (SfM) coupled with Multi-View Stereo (MVS) to generate 3D point clouds and a SLAM (Simultaneous Localization and Mapping) algorithm to register the separately formed models together. Our SfM and SLAM pipelines are built on binary Oriented FAST and Rotated BRIEF (ORB) descriptors to tightly couple these two separate reconstruction workflows and enable fast computation. To elaborate the data capture workflow and validate the proposed method, a case study was conducted on a real-world construction site. Compared to state-of-the-art methods, our preliminary results show a decrease in both registration error and processing time, demonstrating the potential of using daily images captured by different trades coupled with weekly walkthrough videos captured by a field engineer for complete 3D visual monitoring of indoor construction operations.

Download Full-text

GNSS/INS-Assisted Structure from Motion Strategies for UAV-Based Imagery over Mechanized Agricultural Fields

Remote Sensing ◽

10.3390/rs12030351 ◽

2020 ◽

Vol 12 (3) ◽

pp. 351 ◽

Cited By ~ 3

Author(s):

Seyyed Meghdad Hasheminasab ◽

Tian Zhou ◽

Ayman Habib

Keyword(s):

3D Reconstruction ◽

Structure From Motion ◽

Point Clouds ◽

Bundle Adjustment ◽

Agricultural Fields ◽

System Calibration ◽

3D Point Clouds ◽

Digital Agriculture ◽

Orientation Parameters ◽

Calibration Parameters

Acquired imagery by unmanned aerial vehicles (UAVs) has been widely used for three-dimensional (3D) reconstruction/modeling in various digital agriculture applications, such as phenotyping, crop monitoring, and yield prediction. 3D reconstruction from well-textured UAV-based images has matured and the user community has access to several commercial and opensource tools that provide accurate products at a high level of automation. However, in some applications, such as digital agriculture, due to repetitive image patterns, these approaches are not always able to produce reliable/complete products. The main limitation of these techniques is their inability to establish a sufficient number of correctly matched features among overlapping images, causing incomplete and/or inaccurate 3D reconstruction. This paper provides two structure from motion (SfM) strategies, which use trajectory information provided by an onboard survey-grade global navigation satellite system/inertial navigation system (GNSS/INS) and system calibration parameters. The main difference between the proposed strategies is that the first one—denoted as partially GNSS/INS-assisted SfM—implements the four stages of an automated triangulation procedure, namely, imaging matching, relative orientation parameters (ROPs) estimation, exterior orientation parameters (EOPs) recovery, and bundle adjustment (BA). The second strategy— denoted as fully GNSS/INS-assisted SfM—removes the EOPs estimation step while introducing a random sample consensus (RANSAC)-based strategy for removing matching outliers before the BA stage. Both strategies modify the image matching by restricting the search space for conjugate points. They also implement a linear procedure for ROPs’ refinement. Finally, they use the GNSS/INS information in modified collinearity equations for a simpler BA procedure that could be used for refining system calibration parameters. Eight datasets over six agricultural fields are used to evaluate the performance of the developed strategies. In comparison with a traditional SfM framework and Pix4D Mapper Pro, the proposed strategies are able to generate denser and more accurate 3D point clouds as well as orthophotos without any gaps.

Download Full-text

Bundle Adjustment and 3D Reconstruction Method for Underwater Sonar Image

The Journal of Korea Robotics Society ◽

10.7746/jkros.2016.11.2.051 ◽

2016 ◽

Vol 11 (2) ◽

pp. 51-59 ◽

Cited By ~ 3

Author(s):

Young-Sik Shin ◽

◽

Yeong-jun Lee ◽

Hyun-Taek Choi ◽

Ayoung Kim

Keyword(s):

3D Reconstruction ◽

Bundle Adjustment ◽

Reconstruction Method ◽

Sonar Image

Download Full-text

A Practical 3D Reconstruction Method for Weak Texture Scenes

Remote Sensing ◽

10.3390/rs13163103 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3103

Author(s):

Xuyuan Yang ◽

Guang Jiang

Keyword(s):

Deep Learning ◽

3D Reconstruction ◽

High Precision ◽

Structure From Motion ◽

Learning Algorithms ◽

Reconstruction Method ◽

Multiple Views ◽

Feature Points ◽

3D Reconstructions ◽

Pipe Networks

In recent years, there has been a growing demand for 3D reconstructions of tunnel pits, underground pipe networks, and building interiors. For such scenarios, weak textures, repeated textures, or even no textures are common. To reconstruct these scenes, we propose covering the lighting sources with films of spark patterns to “add” textures to the scenes. We use a calibrated camera to take pictures from multiple views and then utilize structure from motion (SFM) and multi-view stereo (MVS) algorithms to carry out a high-precision 3D reconstruction. To improve the effectiveness of our reconstruction, we combine deep learning algorithms with traditional methods to extract and match feature points. Our experiments have verified the feasibility and efficiency of the proposed method.

Download Full-text

BENDING THE DOMING EFFECT IN STRUCTURE FROM MOTION RECONSTRUCTIONS THROUGH BUNDLE ADJUSTMENT

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w6-235-2017 ◽

2017 ◽

Vol XLII-2/W6 ◽

pp. 235-241 ◽

Cited By ~ 2

Author(s):

L. Magri ◽

R. Toldo

Keyword(s):

3D Reconstruction ◽

Structure From Motion ◽

Detection Method ◽

Low Cost ◽

Critical Issue ◽

Bundle Adjustment ◽

Real Word ◽

A Posteriori ◽

Novel Method

Structure from Motion techniques provides low-cost and flexible methods that can be adopted in arial surveying to collect topographic data with accurate results. Nevertheless, the so-called “<i>doming effect</i>”, due to unfortunate acquisition conditions or unreliable modeling of radial distortion, has been recognized as a critical issue that disrupts the quality of the attained 3D reconstruction. In this paper we propose a novel method, that works effectively in the presence of a nearly flat soil, to tackle <i>a posteriori</i> the doming effect: an automatic ground detection method is used to capture the doming deformation flawing the reconstruction, which in turn is wrapped to the correct geometry by iteratively enforcing a planarity constraint through a Bundle Adjustment framework. Experiments on real word datasets demonstrate promising results.

Download Full-text

Calculating Absolute Scale and Scale Uncertainty for SfM Using Distance Sensor Measurements

Recent Advances in 3D Imaging, Modeling, and Reconstruction - Advances in Multimedia and Interactive Technologies ◽

10.4018/978-1-5225-5294-9.ch008 ◽

2020 ◽

pp. 168-192

Author(s):

Ivan Nikolov ◽

Claus B. Madsen

Keyword(s):

3D Reconstruction ◽

Cultural Heritage ◽

Structure From Motion ◽

Data Gathering ◽

Scale Uncertainty ◽

Absolute Scale ◽

Inherent Problem ◽

Distance Sensor ◽

2D Images

Capturing details of objects and surfaces using structure from motion (SfM) 3D reconstruction has become an important part of data gathering in geomapping, medicine, cultural heritage, and the energy and production industries. One inherent problem with SfM, due to its reliance on 2D images, is the ambiguity of the reconstruction's scale. Absolute scale can be calculated by using the data from additional sensors. This chapter demonstrates how distance sensors can be used to calculate the scale of a reconstructed object. In addition, the authors demonstrate that the uncertainty of the calculated scale can be computed and how it depends on the precision of the used sensors. The provided methods are straightforward and easy to integrate into the workflow of commercial SfM solutions.

Download Full-text

Research on 3D Reconstruction Method Based on Temporal Fourier Transform Profilometry

2020 2nd International Conference on Information Technology and Computer Application (ITCA) ◽

10.1109/itca52113.2020.00150 ◽

2020 ◽

Author(s):

Liao Min ◽

Luo weifeng

Keyword(s):

Fourier Transform ◽

3D Reconstruction ◽

Reconstruction Method ◽

Fourier Transform Profilometry

Download Full-text

3D RECONSTRUCTION OF BREAST CANCER FROM MAMMOGRAMS USING VOLUME RENDERING TECHNIQUES

Jurnal Teknologi ◽

10.11113/jt.v75.4978 ◽

2015 ◽

Vol 75 (2) ◽

Author(s):

Ho Wei Yong ◽

Abdullah Bade ◽

Rajesh Kumar Muniandy

Keyword(s):

Breast Cancer ◽

3D Reconstruction ◽

Computational Cost ◽

Reconstruction Method ◽

Cancer Model ◽

Volume Reconstruction ◽

The Past ◽

Breast Cancer Model ◽

Reconstruction Software ◽

To Come

Over the past thirty years, a number of researchers have investigated on 3D organ reconstruction from medical images and there are a few 3D reconstruction software available on the market. However, not many researcheshave focused on3D reconstruction of breast cancer’s tumours. Due to the method complexity, most 3D breast cancer’s tumours reconstruction were done based on MRI slices dataeven though mammogram is the current clinical practice for breast cancer screening. Therefore, this research will investigate the process of creating a method that will be able to reconstruct 3D breast cancer’s tumours from mammograms effectively. Several steps were proposed for this research which includes data acquisition, volume reconstruction, andvolume rendering. The expected output from this research is the 3D breast cancer’s tumours model that is generated from correctly registered mammograms. The main purpose of this research is to come up with a 3D reconstruction method that can produce good breast cancer model from mammograms while using minimal computational cost.

Download Full-text