Fast and robust visual odometry with a low-cost IMU in dynamic environments

2019 ◽  
Vol 46 (6) ◽  
pp. 882-894 ◽  
Author(s):  
Erliang Yao ◽  
Hexin Zhang ◽  
Haitao Song ◽  
Guoliang Zhang
Keyword(s):  
Indirect Method ◽  
Direct Method ◽  
Low Cost ◽  
Visual Odometry ◽  
Dynamic Environments ◽  
Motion Blur ◽  
Bundle Adjustment ◽  
Measurement Unit ◽  
Content Type ◽  
Camera Pose

Purpose To realize stable and precise localization in the dynamic environments, the authors propose a fast and robust visual odometry (VO) approach with a low-cost Inertial Measurement Unit (IMU) in this study. Design/methodology/approach The proposed VO incorporates the direct method with the indirect method to track the features and to optimize the camera pose. It initializes the positions of tracked pixels with the IMU information. Besides, the tracked pixels are refined by minimizing the photometric errors. Due to the small convergence radius of the indirect method, the dynamic pixels are rejected. Subsequently, the camera pose is optimized by minimizing the reprojection errors. The frames with little dynamic information are selected to create keyframes. Finally, the local bundle adjustment is performed to refine the poses of the keyframes and the positions of 3-D points. Findings The proposed VO approach is evaluated experimentally in dynamic environments with various motion types, suggesting that the proposed approach achieves more accurate and stable location than the conventional approach. Moreover, the proposed VO approach works well in the environments with the motion blur. Originality/value The proposed approach fuses the indirect method and the direct method with the IMU information, which improves the localization in dynamic environments significantly.

scholarly journals Dynamic-DSO: Direct Sparse Odometry Using Objects Semantic Information for Dynamic Environments

2020 ◽  
Vol 10 (4) ◽  
pp. 1467
Author(s):  
Chao Sheng ◽  
Shuguo Pan ◽  
Wang Gao ◽  
Yong Tan ◽  
Tao Zhao
Keyword(s):  
Semantic Information ◽  
Direct Method ◽  
Visual Odometry ◽  
Dynamic Environments ◽  
Loop Closure ◽  
Loop Closure Detection ◽  
Camera Pose ◽  
Photometric Error ◽  
Tracking Model ◽  
Dynamic Objects

Traditional Simultaneous Localization and Mapping (SLAM) (with loop closure detection), or Visual Odometry (VO) (without loop closure detection), are based on the static environment assumption. When working in dynamic environments, they perform poorly whether using direct methods or indirect methods (feature points methods). In this paper, Dynamic-DSO which is a semantic monocular direct visual odometry based on DSO (Direct Sparse Odometry) is proposed. The proposed system is completely implemented with the direct method, which is different from the most current dynamic systems combining the indirect method with deep learning. Firstly, convolutional neural networks (CNNs) are applied to the original RGB image to generate the pixel-wise semantic information of dynamic objects. Then, based on the semantic information of the dynamic objects, dynamic candidate points are filtered out in keyframes candidate points extraction; only static candidate points are reserved in the tracking and optimization module, to achieve accurate camera pose estimation in dynamic environments. The photometric error calculated by the projection points in dynamic region of subsequent frames are removed from the whole photometric error in pyramid motion tracking model. Finally, the sliding window optimization which neglects the photometric error calculated in the dynamic region of each keyframe is applied to obtain the precise camera pose. Experiments on the public TUM dynamic dataset and the modified Euroc dataset show that the positioning accuracy and robustness of the proposed Dynamic-DSO is significantly higher than the state-of-the-art direct method in dynamic environments, and the semi-dense cloud map constructed by Dynamic-DSO is clearer and more detailed.

scholarly journals Swedish Weight Sounding: A prospective portable soil investigation tools for liquefaction assessment of residential houses in Indonesia

2020 ◽  
Vol 156 ◽  
pp. 02010
Author(s):  
Yusa Muhamad ◽  
Bowman Elisabeth T. ◽  
Nugroho S.A
Keyword(s):  
Indirect Method ◽  
Direct Method ◽  
Low Cost ◽  
Fine Sand ◽  
Good Prospect ◽  
Torque Measurement ◽  
Liquefaction Potential ◽  
Potential Index ◽  
National Disaster ◽  
Liquefaction Assessment

National Disaster Management Agency (BNPB) statistics show that the majority of earthquake affected buildings are residential houses, whereas in practice, soil investigation is rarely conducted for residential houses in Indonesia. This study is preliminary work on the prospective of Swedish Weight Sounding (SWST) for liquefaction assessment for residential houses. Material used is poorly graded sand. The number of half turns from SWST (NSW) per meter for very loose and loose clean fine sand ranges from 4 to 168 (equivalent to SPT 0-30). Liquefaction potential was assessed using an indirect method by converting NSW into equivalent NSPT and direct method. In general, the factor of safety obtained from the direct method is more conservative (thus giving lower liquefaction potential index) than the indirect method. Torque measured for material in this study ranged from 6-54 Nm, equivalent to a specific energy range from 7-70 N/mm2. Liquefaction assessment using SWST data with torque measurement also indicated the soil is liquefiable. SWST also may be able to detect sand ageing. In summary SWS has a good prospect as a highly portable and low cost investigation tool for liquefaction assessment of residential houses in Indonesia.

scholarly journals Stereo Visual Odometry for Indoor Localization of Ship Model

2020 ◽  
Vol 58 (1) ◽  
pp. 57-75
Author(s):  
Mario Kučić ◽  
Marko Valčić
Keyword(s):  
Autonomous Navigation ◽  
Inertial Measurement Unit ◽  
Indoor Localization ◽  
Low Cost ◽  
Visual Odometry ◽  
Measurement Unit ◽  
Ship Model ◽  
Open Sea ◽  
Autonomous Cars ◽  
Stereo Cameras

Typically, ships are designed for open sea navigation and thus research of autonomous ships is mostly done for that particular area. This paper explores the possibility of using low-cost sensors for localization inside the small navigation area. The localization system is based on the technology used for developing autonomous cars. The main part of the system is visual odometry using stereo cameras fused with Inertial Measurement Unit (IMU) data coupled with Kalman and particle filters to get decimetre level accuracy inside a basin for different surface conditions. The visual odometry uses cropped frames for stereo cameras and Good features to track algorithm for extracting features to get depths for each feature that is used for estimation of ship model movement. Experimental results showed that the proposed system could localize itself within a decimetre accuracy implying that there is a real possibility for ships in using visual odometry for autonomous navigation on narrow waterways, which can have a significant impact on future transportation.

scholarly journals ASSESSING THE RELIABILITY AND THE ACCURACY OF ATTITUDE EXTRACTED FROM VISUAL ODOMETRY FOR LIDAR DATA GEOREFERENCING

2017 ◽  
Vol XLII-2/W6 ◽  
pp. 201-208
Author(s):  
B. Leroux ◽  
J. Cali ◽  
J. Verdun ◽  
L. Morel ◽  
H. He
Keyword(s):  
Test Bench ◽  
Low Cost ◽  
Control Point ◽  
Visual Odometry ◽  
Airborne Lidar ◽  
Measurement Unit ◽  
Navigation Systems ◽  
Control Points ◽  
Ground Control Points ◽  
Orientation Parameters

Airborne LiDAR systems require the use of Direct Georeferencing (DG) in order to compute the coordinates of the surveyed point in the mapping frame. An UAV platform does not derogate to this need, but its payload has to be lighter than this installed onboard so the manufacturer needs to find an alternative to heavy sensors and navigation systems. For the georeferencing of these data, a possible solution could be to replace the Inertial Measurement Unit (IMU) by a camera and record the optical flow. The different frames would then be processed thanks to photogrammetry so as to extract the External Orientation Parameters (EOP) and, therefore, the path of the camera. The major advantages of this method called Visual Odometry (VO) is low cost, no drifts IMU-induced, option for the use of Ground Control Points (GCPs) such as on airborne photogrammetry surveys. In this paper we shall present a test bench designed to assess the reliability and accuracy of the attitude estimated from VO outputs. The test bench consists of a trolley which embeds a GNSS receiver, an IMU sensor and a camera. The LiDAR is replaced by a tacheometer in order to survey the control points already known. We have also developped a methodology applied to this test bench for the calibration of the external parameters and the computation of the surveyed point coordinates. Several tests have revealed a difference about 2–3 centimeters between the control point coordinates measured and those already known.

scholarly journals A FEASIBILITY STUDY ON INCREMENTAL BUNDLE ADJUSTMENT WITH FISHEYE IMAGES AND LOW-COST SENSORS

2019 ◽  
Vol XLII-2/W18 ◽  
pp. 167-171
Author(s):  
A. M. G. Tommaselli ◽  
M. B. Campos ◽  
L. F. Castanheiro ◽  
E. Honkavaara
Keyword(s):  
Microelectromechanical Systems ◽  
Low Cost ◽  
Space Structure ◽  
Bundle Adjustment ◽  
Global Navigation Satellite Systems ◽  
Measurement Unit ◽  
Navigation Systems ◽  
Positional Accuracy ◽  
Integrated Sensor ◽  
Sensor Orientation

Abstract. Low cost imaging and positioning sensors are opening new frontiers for applications in near real-time Photogrammetry. Omnidirectional cameras acquiring images with 360° coverage, when combined with information coming from GNSS (Global Navigation Satellite Systems) and IMU (Inertial Measurement Unit), can efficiently estimate orientation and object space structure. However, several challenges remain in the use of low-cost sensors and image observations acquired by sensors with non-perspective inner geometry. The accuracy of the measurement using low-cost sensors is affected by different sources of errors and sensor stability. Microelectromechanical systems (MEMS) present a large gap between predicted and actual accuracy. This work presents a study on the performance of an integrated sensor orientation approach to estimate sensor orientation and 3D sparse point cloud, using an incremental bundle adjustment strategy and data coming from a low-cost portable mobile terrestrial system composed by off-theshelf navigation systems and a poly-dioptric system (Ricoh Theta S). Experiments were performed in an outdoor area (sidewalk), achieving a trajectory positional accuracy of 0.33 m and a meter level 3D reconstruction.

scholarly journals Forward collision warning system for motorcyclist using smart phone sensors based on time-to-collision and trajectory prediction

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Qun Lim ◽  
Yi Lim ◽  
Hafiz Muhammad ◽  
Dylan Wei Ming Tan ◽  
U-Xuan Tan
Keyword(s):  
Short Term Memory ◽  
Low Cost ◽  
Video Data ◽  
Measurement Unit ◽  
Trajectory Prediction ◽  
Data Set ◽  
Content Type ◽  
Collision Warning ◽  
Time To Collision ◽  
Forward Collision Warning

Purpose The purpose of this paper is to develop a proof-of-concept (POC) Forward Collision Warning (FWC) system for the motorcyclist, which determines a potential clash based on time-to-collision and trajectory of both the detected and ego vehicle (motorcycle). Design/methodology/approach This comes in three approaches. First, time-to-collision value is to be calculated based on low-cost camera video input. Second, the trajectory of the detected vehicle is predicted based on video data in the 2 D pixel coordinate. Third, the trajectory of the ego vehicle is predicted via the lean direction of the motorcycle from a low-cost inertial measurement unit sensor. Findings This encompasses a comprehensive Advanced FWC system which is an amalgamation of the three approaches mentioned above. First, to predict time-to-collision, nested Kalman filter and vehicle detection is used to convert image pixel matrix to relative distance, velocity and time-to-collision data. Next, for trajectory prediction of detected vehicles, a few algorithms were compared, and it was found that long short-term memory performs the best on the data set. The last finding is that to determine the leaning direction of the ego vehicle, it is better to use lean angle measurement compared to riding pattern classification. Originality/value The value of this paper is that it provides a POC FWC system that considers time-to-collision and trajectory of both detected and ego vehicle (motorcycle).

scholarly journals Forward and Backward Visual Fusion Approach to Motion Estimation with High Robustness and Low Cost

2019 ◽  
Vol 11 (18) ◽  
pp. 2139
Author(s):  
Ke Wang ◽  
Xin Huang ◽  
JunLan Chen ◽  
Chuan Cao ◽  
Zhoubing Xiong ◽  
...  
Keyword(s):  
Low Cost ◽  
Visual Odometry ◽  
Optimization Method ◽  
Measurement Unit ◽  
Ground Vehicles ◽  
Unit Quaternion ◽  
Trajectory Estimation ◽  
Evaluation Indexes ◽  
Error Space ◽  
Self Motion

We present a novel low-cost visual odometry method of estimating the ego-motion (self-motion) for ground vehicles by detecting the changes that motion induces on the images. Different from traditional localization methods that use differential global positioning system (GPS), precise inertial measurement unit (IMU) or 3D Lidar, the proposed method only leverage data from inexpensive visual sensors of forward and backward onboard cameras. Starting with the spatial-temporal synchronization, the scale factor of backward monocular visual odometry was estimated based on the MSE optimization method in a sliding window. Then, in trajectory estimation, an improved two-layers Kalman filter was proposed including orientation fusion and position fusion . Where, in the orientation fusion step, we utilized the trajectory error space represented by unit quaternion as the state of the filter. The resulting system enables high-accuracy, low-cost ego-pose estimation, along with providing robustness capability of handing camera module degradation by automatic reduce the confidence of failed sensor in the fusion pipeline. Therefore, it can operate in the presence of complex and highly dynamic motion such as enter-in-and-out tunnel entrance, texture-less, illumination change environments, bumpy road and even one of the cameras fails. The experiments carried out in this paper have proved that our algorithm can achieve the best performance on evaluation indexes of average in distance (AED), average in X direction (AEX), average in Y direction (AEY), and root mean square error (RMSE) compared to other state-of-the-art algorithms, which indicates that the output results of our approach is superior to other methods.

scholarly journals A GNSS-FREE UNMANNED AERIAL LASER SCANNING SYSTEM

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 83-90
Author(s):  
M. Shahbazi ◽  
C. Cortes ◽  
P. Ménard ◽  
J. S. Bilodeau
Keyword(s):  
Laser Scanning ◽  
Low Cost ◽  
Point Clouds ◽  
Bundle Adjustment ◽  
Measurement Unit ◽  
Scanning System ◽  
Dual Frequency ◽  
Gnss Receiver ◽  
Intrinsic Parameters ◽  
The Individual

Abstract. In this paper, the procedure of developing and evaluating a UAV-borne mapping system is described. The system is equipped with both a LiDAR and a camera. The system mounting parameters, as well as the intrinsic parameters of the individual sensors, are calibrated rigorously. Simultaneous calibration of the LiDAR intrinsic parameters and the LiDAR-camera mounting parameters is performed in a self-calibrating bundle adjustment with additional relative orientation constraints. A visual-inertial approach is proposed to georeference the laser scans without using a GNSS receiver. This approach is motivated not only by the interest of users in low-cost systems but also by the fact that the integrity of GNSS signals might be affected under several environmental conditions, e.g., indoors, in urban canyons, under tree canopies. It is shown that a low-cost inertial measurement unit not equipped with a dual-frequency, real-time kinematic GNSS receiver is still useful for georeferencing the laser scanning data with cm-level accuracy. The scans are also textured using the images captured by the camera, which enriches the LiDAR point clouds with spectral information.

scholarly journals PHOTOGRAMMETRIC RECONSTRUCTION WITH BAYESIAN INFORMATION

2016 ◽  
Vol XLI-B1 ◽  
pp. 681-687
Author(s):  
A. Masiero ◽  
F. Fissore ◽  
A. Guarnieri ◽  
F. Pirotti ◽  
A. Vettore
Keyword(s):  
Inertial Measurement Unit ◽  
Laser Scanning ◽  
Feature Matching ◽  
Low Cost ◽  
Bundle Adjustment ◽  
Measurement Unit ◽  
Smart Devices ◽  
Reconstruction Procedure ◽  
Inertial Measurement ◽  
Qualified Personnel

Nowadays photogrammetry and laser scanning methods are the most wide spread surveying techniques. Laser scanning methods usually allow to obtain more accurate results with respect to photogrammetry, but their use have some issues, e.g. related to the high cost of the instrumentation and the typical need of high qualified personnel to acquire experimental data on the field. Differently, photogrammetric reconstruction can be achieved by means of low cost devices and by persons without specific training. Furthermore, the recent diffusion of smart devices (e.g. smartphones) embedded with imaging and positioning sensors (i.e. standard camera, GNSS receiver, inertial measurement unit) is opening the possibility of integrating more information in the photogrammetric reconstruction procedure, in order to increase its computational efficiency, its robustness and accuracy. In accordance with the above observations, this paper examines and validates new possibilities for the integration of information provided by the inertial measurement unit (IMU) into the photogrammetric reconstruction procedure, and, to be more specific, into the procedure for solving the feature matching and the bundle adjustment problems.

Development of low-cost mobile laser scanning for 3D construction indoor mapping by using inertial measurement unit, ultra-wide band and 2D laser scanner

2019 ◽  
Vol 26 (7) ◽  
pp. 1367-1386
Author(s):  
Chao Chen ◽  
Llewellyn Tang ◽  
Craig Matthew Hancock ◽  
Penghe Zhang
Keyword(s):  
Inertial Measurement Unit ◽  
Indoor Environment ◽  
Laser Scanning ◽  
Low Cost ◽  
Laser Scanner ◽  
Wide Band ◽  
Measurement Unit ◽  
Content Type ◽  
Inertial Measurement ◽  
Mls Method

Purpose The purpose of this paper is to introduce the development of an innovative mobile laser scanning (MLS) method for 3D indoor mapping. The generally accepted and used procedure for this type of mapping is usually performed using static terrestrial laser scanning (TLS) which is high-cost and time-consuming. Compared with conventional TLS, the developed method proposes a new idea with advantages of low-cost, high mobility and time saving on the implementation of a 3D indoor mapping. Design/methodology/approach This method integrates a low-cost 2D laser scanner with two indoor positioning techniques – ultra-wide band (UWB) and an inertial measurement unit (IMU), to implement a 3D MLS for reality captures from an experimental indoor environment through developed programming algorithms. In addition, a reference experiment by using conventional TLS was also conducted under the same conditions for scan result comparison to validate the feasibility of the developed method. Findings The findings include: preset UWB system integrated with a low-cost IMU can provide a reliable positioning method for indoor environment; scan results from a portable 2D laser scanner integrated with a motion trajectory from the IMU/UWB positioning approach is able to generate a 3D point cloud based in an indoor environment; and the limitations on hardware, accuracy, automation and the positioning approach are also summarized in this study. Research limitations/implications As the main advantage of the developed method is low-cost, it may limit the automation of the method due to the consideration of the cost control. Robotic carriers and higher-performance 2D laser scanners can be applied to realize panoramic and higher-quality scan results for improvements of the method. Practical implications Moreover, during the practical application, the UWB system can be disturbed by variances of the indoor environment, which can affect the positioning accuracy in practice. More advanced algorithms are also needed to optimize the automatic data processing for reducing errors caused by manual operations. Originality/value The development of this MLS method provides a novel idea that integrates data from heterogeneous systems or sensors to realize a practical aim of indoor mapping, and meanwhile promote the current laser scanning technology to a lower-cost, more flexible, more portable and less time-consuming trend.

Sign in / Sign up

Export Citation Format

Share Document