scholarly journals SELF-SUPERVISED LEARNING FOR MONOCULAR DEPTH ESTIMATION FROM AERIAL IMAGERY

2020 ◽  
Vol V-2-2020 ◽  
pp. 357-364
Author(s):  
M. Hermann ◽  
B. Ruf ◽  
M. Weinmann ◽  
S. Hinz
Keyword(s):  
Supervised Learning ◽  
Image Matching ◽  
Ground Truth ◽  
Depth Estimation ◽  
Training Data ◽  
Aerial Imagery ◽  
Small Model ◽  
Conventional Methods ◽  
Monocular Depth ◽  
Real Time Application

Abstract. Supervised learning based methods for monocular depth estimation usually require large amounts of extensively annotated training data. In the case of aerial imagery, this ground truth is particularly difficult to acquire. Therefore, in this paper, we present a method for self-supervised learning for monocular depth estimation from aerial imagery that does not require annotated training data. For this, we only use an image sequence from a single moving camera and learn to simultaneously estimate depth and pose information. By sharing the weights between pose and depth estimation, we achieve a relatively small model, which favors real-time application. We evaluate our approach on three diverse datasets and compare the results to conventional methods that estimate depth maps based on multi-view geometry. We achieve an accuracy δ1:25 of up to 93.5 %. In addition, we have paid particular attention to the generalization of a trained model to unknown data and the self-improving capabilities of our approach. We conclude that, even though the results of monocular depth estimation are inferior to those achieved by conventional methods, they are well suited to provide a good initialization for methods that rely on image matching or to provide estimates in regions where image matching fails, e.g. occluded or texture-less regions.

scholarly journals MULTI-TASK LEARNING FROM FIXED-WING UAV IMAGES FOR 2D/3D CITY MODELLING

2021 ◽  
Vol XLIV-M-3-2021 ◽  
pp. 1-5
Author(s):  
M. R. Bayanlou ◽  
M. Khoshboresh-Masouleh
Keyword(s):  
Scene Understanding ◽  
Depth Estimation ◽  
Traffic Monitoring ◽  
Training Data ◽  
Specific Information ◽  
Infrastructure Development ◽  
Task Learning ◽  
Monocular Depth ◽  
Uav Images ◽  
3D City Modelling

Abstract. Single-task learning in artificial neural networks will be able to learn the model very well, and the benefits brought by transferring knowledge thus become limited. In this regard, when the number of tasks increases (e.g., semantic segmentation, panoptic segmentation, monocular depth estimation, and 3D point cloud), duplicate information may exist across tasks, and the improvement becomes less significant. Multi-task learning has emerged as a solution to knowledge-transfer issues and is an approach to scene understanding which involves multiple related tasks each with potentially limited training data. Multi-task learning improves generalization by leveraging the domain-specific information contained in the training data of related tasks. In urban management applications such as infrastructure development, traffic monitoring, smart 3D cities, and change detection, automated multi-task data analysis for scene understanding based on the semantic, instance, and panoptic annotation, as well as monocular depth estimation, is required to generate precise urban models. In this study, a common framework for the performance assessment of multi-task learning methods from fixed-wing UAV images for 2D/3D city modelling is presented.

scholarly journals Self-Supervised Monocular Depth Learning in Low-Texture Areas

2021 ◽  
Vol 13 (9) ◽  
pp. 1673
Author(s):  
Wanpeng Xu ◽  
Ling Zou ◽  
Lingda Wu ◽  
Zhipeng Fu
Keyword(s):  
Supervised Learning ◽  
Network Architecture ◽  
Depth Estimation ◽  
Reference Image ◽  
Camera Motion ◽  
Feature Maps ◽  
Data Set ◽  
Edge Location ◽  
Monocular Depth ◽  
Texture Boundaries

For the task of monocular depth estimation, self-supervised learning supervises training by calculating the pixel difference between the target image and the warped reference image, obtaining results comparable to those with full supervision. However, the problematic pixels in low-texture regions are ignored, since most researchers think that no pixels violate the assumption of camera motion, taking stereo pairs as the input in self-supervised learning, which leads to the optimization problem in these regions. To tackle this problem, we perform photometric loss using the lowest-level feature maps instead and implement first- and second-order smoothing to the depth, ensuring consistent gradients ring optimization. Given the shortcomings of ResNet as the backbone, we propose a new depth estimation network architecture to improve edge location accuracy and obtain clear outline information even in smoothed low-texture boundaries. To acquire more stable and reliable quantitative evaluation results, we introce a virtual data set in the self-supervised task because these have dense depth maps corresponding to pixel by pixel. We achieve performance that exceeds that of the prior methods on both the Eigen Splits of the KITTI and VKITTI2 data sets taking stereo pairs as the input.

Attention-Based Self-Supervised Learning Monocular Depth Estimation With Edge Refinement

2021 ◽  
Author(s):  
Chenweinan Jiang ◽  
Haichun Liu ◽  
Lanzhen Li ◽  
Changchun Pan
Keyword(s):  
Supervised Learning ◽  
Depth Estimation ◽  
Monocular Depth

scholarly journals Real-time 3D Perception of Scene with Monocular Camera

2020 ◽  
Vol 7 (2) ◽  
pp. 4-7
Author(s):  
Shadi Saleh ◽  
Shanmugapriyan Manoharan ◽  
Wolfram Hardt
Keyword(s):  
Supervised Learning ◽  
Ground Truth ◽  
Depth Estimation ◽  
Depth Information ◽  
Learning Approach ◽  
Single Image ◽  
Learning To Learn ◽  
3D Perception ◽  
Training Time ◽  
Visual Appearance

Depth is a vital prerequisite for the fulfillment of various tasks such as perception, navigation, and planning. Estimating depth using only a single image is a challenging task since the analytic mapping is not available between the intensity image and its depth where the features cue of the context is usually absent in the single image. Furthermore, most current researchers rely on the supervised Learning approach to handle depth estimation. Therefore, the demand for recorded ground truth depth is important at the training time, which is actually tricky and costly. This study presents two approaches (unsupervised learning and semi-supervised learning) to learn the depth information using only a single RGB-image. The main objective of depth estimation is to extract a representation of the spatial structure of the environment and to restore the 3D shape and visual appearance of objects in imagery.

Self-Supervised Learning for Monocular Depth Estimation on Minimally Invasive Surgery Scenes

2021 ◽  
Author(s):  
Shuwei Shao ◽  
Zhongcai Pei ◽  
Weihai Chen ◽  
Baochang Zhang ◽  
Xingming Wu ◽  
...  
Keyword(s):  
Minimally Invasive Surgery ◽  
Minimally Invasive ◽  
Supervised Learning ◽  
Invasive Surgery ◽  
Depth Estimation ◽  
Monocular Depth

Self-Supervised Learning of Monocular Depth Estimation Based on Progressive Strategy

2021 ◽  
pp. 1-1
Author(s):  
Huachun Wang ◽  
Xinzhu Sang ◽  
Duo Chen ◽  
Peng Wang ◽  
Binbin Yan ◽  
...  
Keyword(s):  
Supervised Learning ◽  
Depth Estimation ◽  
Monocular Depth

scholarly journals Unsupervised Monocular Depth Estimation Based on Residual Neural Network of Coarse–Refined Feature Extractions for Drone

Electronics ◽  
2019 ◽  
Vol 8 (10) ◽  
pp. 1179 ◽  
Author(s):  
Tao Huang ◽  
Shuanfeng Zhao ◽  
Longlong Geng ◽  
Qian Xu
Keyword(s):  
Neural Network ◽  
Image Reconstruction ◽  
Depth Map ◽  
Ground Truth ◽  
Depth Estimation ◽  
Input Image ◽  
Superior Performance ◽  
Estimation Methods ◽  
Depth Information ◽  
Monocular Depth

To take full advantage of the information of images captured by drones and given that most existing monocular depth estimation methods based on supervised learning require vast quantities of corresponding ground truth depth data for training, the model of unsupervised monocular depth estimation based on residual neural network of coarse–refined feature extractions for drone is therefore proposed. As a virtual camera is introduced through a deep residual convolution neural network based on coarse–refined feature extractions inspired by the principle of binocular depth estimation, the unsupervised monocular depth estimation has become an image reconstruction problem. To improve the performance of our model for monocular depth estimation, the following innovations are proposed. First, the pyramid processing for input image is proposed to build the topological relationship between the resolution of input image and the depth of input image, which can improve the sensitivity of depth information from a single image and reduce the impact of input image resolution on depth estimation. Second, the residual neural network of coarse–refined feature extractions for corresponding image reconstruction is designed to improve the accuracy of feature extraction and solve the contradiction between the calculation time and the numbers of network layers. In addition, to predict high detail output depth maps, the long skip connections between corresponding layers in the neural network of coarse feature extractions and deconvolution neural network of refined feature extractions are designed. Third, the loss of corresponding image reconstruction based on the structural similarity index (SSIM), the loss of approximate disparity smoothness and the loss of depth map are united as a novel training loss to better train our model. The experimental results show that our model has superior performance on the KITTI dataset composed by corresponding left view and right view and Make3D dataset composed by image and corresponding ground truth depth map compared to the state-of-the-art monocular depth estimation methods and basically meet the requirements for depth information of images captured by drones when our model is trained on KITTI.

scholarly journals YOLO MDE: Object Detection with Monocular Depth Estimation

Electronics ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. 76
Author(s):  
Jongsub Yu ◽  
Hyukdoo Choi
Keyword(s):  
Risk Assessment ◽  
Object Detection ◽  
Network Architecture ◽  
Ground Truth ◽  
Depth Estimation ◽  
Autonomous Driving ◽  
Depth Prediction ◽  
Bounding Box ◽  
Monocular Depth ◽  
Bounding Boxes

This paper presents an object detector with depth estimation using monocular camera images. Previous detection studies have typically focused on detecting objects with 2D or 3D bounding boxes. A 3D bounding box consists of the center point, its size parameters, and heading information. However, predicting complex output compositions leads a model to have generally low performances, and it is not necessary for risk assessment for autonomous driving. We focused on predicting a single depth per object, which is essential for risk assessment for autonomous driving. Our network architecture is based on YOLO v4, which is a fast and accurate one-stage object detector. We added an additional channel to the output layer for depth estimation. To train depth prediction, we extract the closest depth from the 3D bounding box coordinates of ground truth labels in the dataset. Our model is compared with the latest studies on 3D object detection using the KITTI object detection benchmark. As a result, we show that our model achieves higher detection performance and detection speed than existing models with comparable depth accuracy.

Sign in / Sign up

Export Citation Format

Share Document