scholarly journals Detail-preserving depth estimation from a single image based on modified fully convolutional residual network and gradient network

2021 ◽  
Vol 3 (12) ◽  
Author(s):  
Huihui Xu ◽  
Nan Liu
Keyword(s):  
Depth Map ◽  
Depth Estimation ◽  
Prediction Algorithm ◽  
Single Image ◽  
Residual Network ◽  
Depth Prediction ◽  
Daunting Task ◽  
Depth Gradients ◽  
Loss Term ◽  
Fusion Scheme

AbstractPredicting a convincing depth map from a monocular single image is a daunting task in the field of computer vision. In this paper, we propose a novel detail-preserving depth estimation (DPDE) algorithm based on a modified fully convolutional residual network and gradient network. Specifically, we first introduce a new deep network that combines the fully convolutional residual network (FCRN) and a U-shaped architecture to generate the global depth map. Meanwhile, an efficient feature similarity-based loss term is introduced for training this network better. Then, we devise a gradient network to generate the local details of the scene based on gradient information. Finally, an optimization-based fusion scheme is proposed to integrate the depth and depth gradients to generate a reliable depth map with better details. Three benchmark RGBD datasets are evaluated from the perspective of qualitative and quantitative, the experimental results show that the designed depth prediction algorithm is superior to several classic depth prediction approaches and can reconstruct plausible depth maps.

scholarly journals The Synthesis of Unpaired Underwater Images for Monocular Underwater Depth Prediction

2021 ◽  
Vol 8 ◽  
Author(s):  
Qi Zhao ◽  
Ziqiang Zheng ◽  
Huimin Zeng ◽  
Zhibin Yu ◽  
Haiyong Zheng ◽  
...  
Keyword(s):  
Structural Information ◽  
Depth Map ◽  
Depth Estimation ◽  
Image Synthesis ◽  
Map Estimation ◽  
Estimation Model ◽  
Depth Prediction ◽  
Underwater Image ◽  
Underwater Vision ◽  
Image Depth

Underwater depth prediction plays an important role in underwater vision research. Because of the complex underwater environment, it is extremely difficult and expensive to obtain underwater datasets with reliable depth annotation. Thus, underwater depth map estimation with a data-driven manner is still a challenging task. To tackle this problem, we propose an end-to-end system including two different modules for underwater image synthesis and underwater depth map estimation, respectively. The former module aims to translate the hazy in-air RGB-D images to multi-style realistic synthetic underwater images while retaining the objects and the structural information of the input images. Then we construct a semi-real RGB-D underwater dataset using the synthesized underwater images and the original corresponding depth maps. We conduct supervised learning to perform depth estimation through the pseudo paired underwater RGB-D images. Comprehensive experiments have demonstrated that the proposed method can generate multiple realistic underwater images with high fidelity, which can be applied to enhance the performance of monocular underwater image depth estimation. Furthermore, the trained depth estimation model can be applied to real underwater image depth map estimation. We will release our codes and experimental setting in https://github.com/ZHAOQIII/UW_depth.

scholarly journals Fast Depth Estimation in a Single Image Using Lightweight Efficient Neural Network

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4434 ◽  
Author(s):  
Sangwon Kim ◽  
Jaeyeal Nam ◽  
Byoungchul Ko
Keyword(s):  
Neural Network ◽  
Real Time ◽  
Fundamental Problem ◽  
Depth Map ◽  
Ground Truth ◽  
Depth Estimation ◽  
Depth Range ◽  
Single Image ◽  
Special Equipment ◽  
Multiple Images

Depth estimation is a crucial and fundamental problem in the computer vision field. Conventional methods re-construct scenes using feature points extracted from multiple images; however, these approaches require multiple images and thus are not easily implemented in various real-time applications. Moreover, the special equipment required by hardware-based approaches using 3D sensors is expensive. Therefore, software-based methods for estimating depth from a single image using machine learning or deep learning are emerging as new alternatives. In this paper, we propose an algorithm that generates a depth map in real time using a single image and an optimized lightweight efficient neural network (L-ENet) algorithm instead of physical equipment, such as an infrared sensor or multi-view camera. Because depth values have a continuous nature and can produce locally ambiguous results, pixel-wise prediction with ordinal depth range classification was applied in this study. In addition, in our method various convolution techniques are applied to extract a dense feature map, and the number of parameters is greatly reduced by reducing the network layer. By using the proposed L-ENet algorithm, an accurate depth map can be generated from a single image quickly and, in a comparison with the ground truth, we can produce depth values closer to those of the ground truth with small errors. Experiments confirmed that the proposed L-ENet can achieve a significantly improved estimation performance over the state-of-the-art algorithms in depth estimation based on a single image.

scholarly journals Single-Image Depth Inference Using Generative Adversarial Networks

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1708 ◽  
Author(s):  
Daniel Stanley Tan ◽  
Chih-Yuan Yao ◽  
Conrado Ruiz ◽  
Kai-Lung Hua
Keyword(s):  
Smart Cities ◽  
Depth Map ◽  
Depth Estimation ◽  
Input Image ◽  
Generative Adversarial Networks ◽  
Depth Information ◽  
Single Image ◽  
Neural Network Models ◽  
Generative Adversarial Network ◽  
Depth Sensors

Depth has been a valuable piece of information for perception tasks such as robot grasping, obstacle avoidance, and navigation, which are essential tasks for developing smart homes and smart cities. However, not all applications have the luxury of using depth sensors or multiple cameras to obtain depth information. In this paper, we tackle the problem of estimating the per-pixel depths from a single image. Inspired by the recent works on generative neural network models, we formulate the task of depth estimation as a generative task where we synthesize an image of the depth map from a single Red, Green, and Blue (RGB) input image. We propose a novel generative adversarial network that has an encoder-decoder type generator with residual transposed convolution blocks trained with an adversarial loss. Quantitative and qualitative experimental results demonstrate the effectiveness of our approach over several depth estimation works.

scholarly journals Depth Estimation from Monocular Image and Coarse Depth Points based on Conditional GAN

2018 ◽  
Vol 175 ◽  
pp. 03055 ◽  
Author(s):  
Yaoxin Li ◽  
Keyuan Qian ◽  
Tao Huang ◽  
Jingkun Zhou
Keyword(s):  
Depth Map ◽  
Depth Estimation ◽  
Point Clouds ◽  
Single Image ◽  
Depth Sensor ◽  
Depth Maps ◽  
Adversarial Networks ◽  
Novel Approach ◽  
Source Data ◽  
Monocular Image

Depth estimation has achieved considerable success with the development of the depth sensor devices and deep learning method. However, depth estimation from monocular RGB-based image will increase ambiguity and is prone to error. In this paper, we present a novel approach to produce dense depth map from a single image coupled with coarse point-cloud samples. Our approach learns to fit the distribution of the depth map from source data using conditional adversarial networks and convert the sparse point clouds to dense maps. Our experiments show that the use of the conditional adversarial networks can add full image information to the predicted depth maps and the effectiveness of our approach to predict depth in NYU-Depth-v2 indoor dataset.

scholarly journals A Residual Network and FPGA Based Real-Time Depth Map Enhancement System

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 546
Author(s):  
Zhenni Li ◽  
Haoyi Sun ◽  
Yuliang Gao ◽  
Jiao Wang
Keyword(s):  
Real Time ◽  
Super Resolution ◽  
Depth Map ◽  
Acquisition System ◽  
Depth Image ◽  
Fpga Design ◽  
Depth Sensing ◽  
Residual Network ◽  
Real Time Processing ◽  
Depth Maps

Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 fps@ (1280 × 960 + 328 × 248 × 3).

scholarly journals Monocular Depth Estimation Based on Multi-Scale Depth Map Fusion

IEEE Access ◽  
2021 ◽  
pp. 1-1
Author(s):  
Xin Yang ◽  
Qingling Chang ◽  
Xinglin Liu ◽  
Siyuan He ◽  
Yan Cui
Keyword(s):  
Depth Map ◽  
Depth Estimation ◽  
Multi Scale ◽  
Monocular Depth

Kernel-attended residual network for single image super-resolution

2021 ◽  
Vol 213 ◽  
pp. 106663
Author(s):  
Yujie Dun ◽  
Zongyang Da ◽  
Shuai Yang ◽  
Yao Xue ◽  
Xueming Qian
Keyword(s):  
Super Resolution ◽  
Single Image ◽  
Residual Network ◽  
Image Super Resolution ◽  
Single Image Super Resolution

scholarly journals Real-Time Single Image Depth Perception in the Wild with Handheld Devices

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 15
Author(s):  
Filippo Aleotti ◽  
Giulio Zaccaroni ◽  
Luca Bartolomei ◽  
Matteo Poggi ◽  
Fabio Tosi ◽  
...  
Keyword(s):  
Real Time ◽  
Depth Perception ◽  
Depth Estimation ◽  
Autonomous Driving ◽  
Estimation Methods ◽  
Handheld Devices ◽  
Single Image ◽  
Handheld Device ◽  
Time Performance ◽  
In The Wild

Depth perception is paramount for tackling real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image would represent the most versatile solution since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit the practical deployment of monocular depth estimation methods on such devices: (i) the low reliability when deployed in the wild and (ii) the resources needed to achieve real-time performance, often not compatible with low-power embedded systems. Therefore, in this paper, we deeply investigate all these issues, showing how they are both addressable by adopting appropriate network design and training strategies. Moreover, we also outline how to map the resulting networks on handheld devices to achieve real-time performance. Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications. Indeed, to further support this evidence, we report experimental results concerning real-time, depth-aware augmented reality and image blurring with smartphones in the wild.

Sign in / Sign up

Export Citation Format

Share Document