scholarly journals REAL-TIME ON-BOARD OBSTACLE AVOIDANCE FOR UAVS BASED ON EMBEDDED STEREO VISION

2018 ◽  
Vol XLII-1 ◽  
pp. 363-370 ◽  
Author(s):  
B. Ruf ◽  
S. Monka ◽  
M. Kollmann ◽  
M. Grinberg
Keyword(s):  
Obstacle Avoidance ◽  
Critical Distance ◽  
Obstacle Detection ◽  
Disparity Estimation ◽  
Disparity Map ◽  
Application Development ◽  
Development Costs ◽  
Global Matching ◽  
Disparity Maps ◽  
High Level

<p><strong>Abstract.</strong> In order to improve usability and safety, modern unmanned aerial vehicles (UAVs) are equipped with sensors to monitor the environment, such as laser-scanners and cameras. One important aspect in this monitoring process is to detect obstacles in the flight path in order to avoid collisions. Since a large number of consumer UAVs suffer from tight weight and power constraints, our work focuses on obstacle avoidance based on a lightweight stereo camera setup. We use disparity maps, which are computed from the camera images, to locate obstacles and to automatically steer the UAV around them. For disparity map computation we optimize the well-known semi-global matching (SGM) approach for the deployment on an embedded FPGA. The disparity maps are then converted into simpler representations, the so called U-/V-Maps, which are used for obstacle detection. Obstacle avoidance is based on a reactive approach which finds the shortest path around the obstacles as soon as they have a critical distance to the UAV. One of the fundamental goals of our work was the reduction of development costs by closing the gap between application development and hardware optimization. Hence, we aimed at using high-level synthesis (HLS) for porting our algorithms, which are written in C/C++, to the embedded FPGA. We evaluated our implementation of the disparity estimation on the KITTI Stereo 2015 benchmark. The integrity of the overall realtime reactive obstacle avoidance algorithm has been evaluated by using Hardware-in-the-Loop testing in conjunction with two flight simulators.</p>

scholarly journals Lidar Based Intelligent Obstacle Avoidance System for Autonomous Ground Vehicles

2020 ◽  
Vol 8 (6) ◽  
pp. 2466-2472
Keyword(s):  
Neural Network ◽  
Obstacle Avoidance ◽  
Obstacle Detection ◽  
Sensor Data ◽  
Ground Vehicles ◽  
Distance Information ◽  
Vast Number ◽  
Autonomous Ground Vehicles ◽  
Level Of Automation ◽  
High Level

Autonomous ground vehicles (AGVs) started occupying our day-to-day life. AGVs can be programmed to be smart with the current technological advancements. In doing so, we can apply them to assist humans in many aspects like reducing road accidents, enabling us to use cars without driving knowledge, autonomous patrolling in dangerous zones, and autonomous farming. For AGVs to operate at this level of automation, it must be equipped with sensory perception devices to be aware of its surroundings, and also, a way to perceives this data is crucial. As a first step towards this, researchers have developed a vast number of camera vision-based efficient neural network algorithms for detecting and avoiding obstacles. Unfortunately, an AGV cannot survive only with computer vision as it suffers from several effects like night driving and erroneous estimation of distance information. Camera vision and lidar vision together is suitable for AGVs to operate in all conditions like day, night, and fog. We propose a novel neural network model, which transforms the lidar sensor data into obstacle avoidance decisions, which is integrated into the hybrid vision of any AGV. Existing lidar sensor-based obstacle detection and avoidance systems like 2D collision cone approaches are not suitable for real-time applications, as they lag in providing accurate and quick responses, which leads to collisions. The proposed intelligent Field of View (FOV) mechanism replaces classical mathematical approaches, which accurately mimics the behavior of human drivers. The model quickly takes decisions with a high level of accuracy to command the AGV upon being obstructed with obstacles in the trajectory. This makes the AGV drive in obstacle rich environments without manual maneuvering autonomously.

scholarly journals HYBRID-BASED DENSE STEREO MATCHING

2016 ◽  
Vol XLI-B3 ◽  
pp. 495-501 ◽  
Author(s):  
T. Y. Chuang ◽  
H. W. Ting ◽  
J. J. Jaw
Keyword(s):  
Stereo Matching ◽  
Penalty Parameter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Satisfactory Performance ◽  
Dense Stereo Matching ◽  
Illumination Changes ◽  
Global Matching ◽  
Disparity Maps ◽  
Image Edges

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

scholarly journals HYBRID-BASED DENSE STEREO MATCHING

2016 ◽  
Vol XLI-B3 ◽  
pp. 495-501
Author(s):  
T. Y. Chuang ◽  
H. W. Ting ◽  
J. J. Jaw
Keyword(s):  
Stereo Matching ◽  
Penalty Parameter ◽  
Disparity Map ◽  
Cost Aggregation ◽  
Satisfactory Performance ◽  
Dense Stereo Matching ◽  
Illumination Changes ◽  
Global Matching ◽  
Disparity Maps ◽  
Image Edges

Stereo matching generating accurate and dense disparity maps is an indispensable technique for 3D exploitation of imagery in the fields of Computer vision and Photogrammetry. Although numerous solutions and advances have been proposed in the literature, occlusions, disparity discontinuities, sparse texture, image distortion, and illumination changes still lead to problematic issues and await better treatment. In this paper, a hybrid-based method based on semi-global matching is presented to tackle the challenges on dense stereo matching. To ease the sensitiveness of SGM cost aggregation towards penalty parameters, a formal way to provide proper penalty estimates is proposed. To this end, the study manipulates a shape-adaptive cross-based matching with an edge constraint to generate an initial disparity map for penalty estimation. Image edges, indicating the potential locations of occlusions as well as disparity discontinuities, are approved by the edge drawing algorithm to ensure the local support regions not to cover significant disparity changes. Besides, an additional penalty parameter 𝑃𝑒 is imposed onto the energy function of SGM cost aggregation to specifically handle edge pixels. Furthermore, the final disparities of edge pixels are found by weighting both values derived from the SGM cost aggregation and the U-SURF matching, providing more reliable estimates at disparity discontinuity areas. Evaluations on Middlebury stereo benchmarks demonstrate satisfactory performance and reveal the potency of the hybrid-based dense stereo matching method.

scholarly journals GROUND TRUTH GENERATION AND DISPARITY ESTIMATION FOR OPTICAL SATELLITE IMAGERY

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 127-134
Author(s):  
M. Cournet ◽  
E. Sarrazin ◽  
L. Dumas ◽  
J. Michel ◽  
J. Guinet ◽  
...  
Keyword(s):  
Deep Learning ◽  
Satellite Imagery ◽  
Learning Difficulties ◽  
Ground Truth ◽  
Disparity Estimation ◽  
Learning Network ◽  
Ground Truth Generation ◽  
Global Matching ◽  
Disparity Maps ◽  
Deep Learning Network

Abstract. Several 3D reconstruction pipelines are being developed around the world for satellite imagery. Most of them implement their own versions of Semi-Global Matching, as an option for the matching step. However, deep learning based solutions already outperform every SGM derived algorithms on Kitti and Middlebury stereo datasets. But these deep learning based solutions need huge quantities of ground truths for training. This implies that the generation of ground truth stereo datasets, from satellite imagery and lidar, seems to be of great interest for the scientific community. It will aim at reducing the potential transfer learning difficulties, that could arise from a training done on datasets such as Middlebury or Kitti. In this work, we present a new ground truth generation pipeline. It produces stereo-rectified images and ground truth disparity maps, from satellite imagery and lidar. We also assess the rectification and the disparity accuracies of these outputs. We finally train a deep learning network on our preliminary ground truth dataset.

scholarly journals REDUCTION OF THE FRONTO-PARALLEL BIAS FOR WIDE-BASELINE SEMI-GLOBAL MATCHING

2019 ◽  
Vol IV-2/W5 ◽  
pp. 69-76 ◽  
Author(s):  
L. Roth ◽  
H. Mayer
Keyword(s):  
Image Matching ◽  
Synthetic Data ◽  
Real Data ◽  
Image Warping ◽  
Disparity Map ◽  
Dense Image ◽  
Global Matching ◽  
Disparity Maps ◽  
Smoothness Term ◽  
Data Term

<p><strong>Abstract.</strong> Semi-Global Matching (SGM) is a widely-used technique for dense image matching that is popular because of its accuracy and speed. While it works well for textured scenes, it can fail on slanted surfaces particularly in wide-baseline configurations due to the so-called fronto-parallel bias. In this paper, we propose an extension of SGM that utilizes image warping to reduce the fronto-parallel bias in the data term, based on estimating dominant slanted planes. The latter are also used as surface priors improving the smoothness term. Our proposed method calculates disparity maps for each dominant slanted plane and fuses them to obtain the final disparity map. We have quantitatively evaluated our approach outperforming SGM and SGM-P on synthetic data and demonstrate its potential on real data by qualitative results. In this way, we underscore the need to tackle the fronto-parallel bias in particular for wide-baseline configurations in both the data term and the smoothness term of SGM.</p>

scholarly journals Learned Collaborative Stereo Refinement

2021 ◽  
Author(s):  
Patrick Knöbelreiter ◽  
Thomas Pock
Keyword(s):  
Structural Properties ◽  
Gradient Method ◽  
Color Image ◽  
Image Space ◽  
Activation Functions ◽  
Disparity Map ◽  
Proximal Gradient Method ◽  
Variational Structure ◽  
Disparity Maps ◽  
The Individual

AbstractIn this work, we propose a learning-based method to denoise and refine disparity maps. The proposed variational network arises naturally from unrolling the iterates of a proximal gradient method applied to a variational energy defined in a joint disparity, color, and confidence image space. Our method allows to learn a robust collaborative regularizer leveraging the joint statistics of the color image, the confidence map and the disparity map. Due to the variational structure of our method, the individual steps can be easily visualized, thus enabling interpretability of the method. We can therefore provide interesting insights into how our method refines and denoises disparity maps. To this end, we can visualize and interpret the learned filters and activation functions and prove the increased reliability of the predicted pixel-wise confidence maps. Furthermore, the optimization based structure of our refinement module allows us to compute eigen disparity maps, which reveal structural properties of our refinement module. The efficiency of our method is demonstrated on the publicly available stereo benchmarks Middlebury 2014 and Kitti 2015.

scholarly journals A State-of-the-Art Analysis of Obstacle Avoidance Methods from the Perspective of an Agricultural Sprayer UAV’s Operation Scenario

Agronomy ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 1069
Author(s):  
Shibbir Ahmed ◽  
Baijing Qiu ◽  
Fiaz Ahmad ◽  
Chun-Wei Kong ◽  
Huang Xin
Keyword(s):  
Collision Avoidance ◽  
Obstacle Avoidance ◽  
State Of The Art ◽  
Obstacle Detection ◽  
Crop Monitoring ◽  
Data Intensive ◽  
Structural Details ◽  
Dynamic Operating ◽  
New Generation ◽  
The Internet Of Things

Over the last decade, Unmanned Aerial Vehicles (UAVs), also known as drones, have been broadly utilized in various agricultural fields, such as crop management, crop monitoring, seed sowing, and pesticide spraying. Nonetheless, autonomy is still a crucial limitation faced by the Internet of Things (IoT) UAV systems, especially when used as sprayer UAVs, where data needs to be captured and preprocessed for robust real-time obstacle detection and collision avoidance. Moreover, because of the objective and operational difference between general UAVs and sprayer UAVs, not every obstacle detection and collision avoidance method will be sufficient for sprayer UAVs. In this regard, this article seeks to review the most relevant developments on all correlated branches of the obstacle avoidance scenarios for agricultural sprayer UAVs, including a UAV sprayer’s structural details. Furthermore, the most relevant open challenges for current UAV sprayer solutions are enumerated, thus paving the way for future researchers to define a roadmap for devising new-generation, affordable autonomous sprayer UAV solutions. Agricultural UAV sprayers require data-intensive algorithms for the processing of the images acquired, and expertise in the field of autonomous flight is usually needed. The present study concludes that UAV sprayers are still facing obstacle detection challenges due to their dynamic operating and loading conditions.

scholarly journals A Hands-Free Obstacle Detection and Avoidance System

2015 ◽  
Vol 5 (3) ◽  
pp. 801-804
Author(s):  
M. Abdul-Niby ◽  
M. Alameen ◽  
O. Irscheid ◽  
M. Baidoun ◽  
H. Mourtada
Keyword(s):  
Obstacle Avoidance ◽  
Visually Impaired ◽  
Low Cost ◽  
Cost Effective ◽  
Obstacle Detection ◽  
Text To Speech ◽  
Voice Signal ◽  
Visually Impaired People ◽  
Impaired People ◽  
The Voice

In this paper, we present a low cost hands-free detection and avoidance system designed to provide mobility assistance for visually impaired people. An ultrasonic sensor is attached to the jacket of the user and detects the obstacles in front. The information obtained is transferred to the user through audio messages and also by a vibration. The range of the detection is user-defined. A text-to-speech module is employed for the voice signal. The proposed obstacle avoidance device is cost effective, easy to use and easily upgraded.

scholarly journals Disparity estimation using Graph cuts for road applications

2021 ◽  
Vol 297 ◽  
pp. 01055
Author(s):  
Mohamed El Ansari ◽  
Ilyas El Jaafari ◽  
Lahcen Koutti
Keyword(s):  
Stereo Matching ◽  
Optimal Solution ◽  
Similarity Criterion ◽  
Graph Cuts ◽  
Search Space ◽  
Disparity Estimation ◽  
Stereo Images ◽  
Disparity Map ◽  
Current Frame ◽  
Edge Points

This paper proposes a new edge based stereo matching approach for road applications. The new approach consists in matching the edge points extracted from the input stereo images using temporal constraints. At the current frame, we propose to estimate a disparity range for each image line based on the disparity map of its preceding one. The stereo images are divided into multiple parts according to the estimated disparity ranges. The optimal solution of each part is independently approximated via the state-of-the-art energy minimization approach Graph cuts. The disparity search space at each image part is very small compared to the global one, which improves the results and reduces the execution time. Furthermore, as a similarity criterion between corresponding edge points, we propose a new cost function based on the intensity, the gradient magnitude and gradient orientation. The proposed method has been tested on virtual stereo images, and it has been compared to a recently proposed method and the results are satisfactory.

Sign in / Sign up

Export Citation Format

Share Document