Realtime Depth Estimation and Obstacle Detection from Monocular Video

As difficult vision-based tasks like object detection and monocular depth estimation are making their way in real-time applications and as more light weighted solutions for autonomous vehicles navigation systems are emerging, obstacle detection and collision prediction are two very challenging tasks for small embedded devices like drones. We propose a novel light weighted and time-efficient vision-based solution to predict Time-to-Collision from a monocular video camera embedded in a smartglasses device as a module of a navigation system for visually impaired pedestrians. It consists of two modules: a static data extractor made of a convolutional neural network to predict the obstacle position and distance and a dynamic data extractor that stacks the obstacle data from multiple frames and predicts the Time-to-Collision with a simple fully connected neural network. This paper focuses on the Time-to-Collision network’s ability to adapt to new sceneries with different types of obstacles with supervised learning.

Download Full-text

Unsupervised Depth Estimation from Monocular Video based on Relative Motion

Proceedings of the 2018 International Conference on Signal Processing and Machine Learning - SPML '18 ◽

10.1145/3297067.3297094 ◽

2018 ◽

Author(s):

Hui Cao ◽

Chao Wang ◽

Ping Wang ◽

Qingquan Zou ◽

Xiao Xiao

Keyword(s):

Relative Motion ◽

Depth Estimation ◽

Monocular Video

Download Full-text

An Enhancement for the Consistent Depth Estimation of Monocular Videos using Lightweight Network

Machine Learning and Applications An International Journal ◽

10.5121/mlaij.2021.8302 ◽

2021 ◽

Vol 8 (3) ◽

pp. 15-27

Author(s):

Mohamed N. Sweilam ◽

Nikolay Tolstokulakov

Keyword(s):

Neural Network ◽

Computer Vision ◽

Mobile Phones ◽

Depth Estimation ◽

The Neural Network ◽

Monocular Video ◽

Great Progress ◽

Run Time

Depth estimation has made great progress in the last few years due to its applications in robotics science and computer vision. Various methods have been implemented and enhanced to estimate the depth without flickers and missing holes. Despite this progress, it is still one of the main challenges for researchers, especially for the video applications which have more complexity of the neural network which af ects the run time. Moreover to use such input like monocular video for depth estimation is considered an attractive idea, particularly for hand-held devices such as mobile phones, they are very popular for capturing pictures and videos, in addition to having a limited amount of RAM. Here in this work, we focus on enhancing the existing consistent depth estimation for monocular videos approach to be with less usage of RAM and with using less number of parameters without having a significant reduction in the quality of the depth estimation.

Download Full-text

J-MOD2: Joint Monocular Obstacle Detection and Depth Estimation

IEEE Robotics and Automation Letters ◽

10.1109/lra.2018.2800083 ◽

2018 ◽

Vol 3 (3) ◽

pp. 1490-1497 ◽

Cited By ~ 13

Author(s):

Michele Mancini ◽

Gabriele Costante ◽

Paolo Valigi ◽

Thomas A. Ciarfuglia

Keyword(s):

Depth Estimation ◽

Obstacle Detection

Download Full-text

Online supervised attention-based recurrent depth estimation from monocular video

PeerJ Computer Science ◽

10.7717/peerj-cs.317 ◽

2020 ◽

Vol 6 ◽

pp. e317

Author(s):

Dmitrii Maslov ◽

Ilya Makarov

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Short Term Memory ◽

Depth Estimation ◽

Autonomous Driving ◽

Temporal Information ◽

Depth Information ◽

Safe Driving ◽

Monocular Video ◽

Depth Reconstruction

Autonomous driving highly depends on depth information for safe driving. Recently, major improvements have been taken towards improving both supervised and self-supervised methods for depth reconstruction. However, most of the current approaches focus on single frame depth estimation, where quality limit is hard to beat due to limitations of supervised learning of deep neural networks in general. One of the way to improve quality of existing methods is to utilize temporal information from frame sequences. In this paper, we study intelligent ways of integrating recurrent block in common supervised depth estimation pipeline. We propose a novel method, which takes advantage of the convolutional gated recurrent unit (convGRU) and convolutional long short-term memory (convLSTM). We compare use of convGRU and convLSTM blocks and determine the best model for real-time depth estimation task. We carefully study training strategy and provide new deep neural networks architectures for the task of depth estimation from monocular video using information from past frames based on attention mechanism. We demonstrate the efficiency of exploiting temporal information by comparing our best recurrent method with existing image-based and video-based solutions for monocular depth reconstruction.

Download Full-text