Real-time depth map processor for offset aperture based single camera system

2018 ◽  
Author(s):  
Hyeji Kim ◽  
Jinyeon Lim ◽  
Yeongmin Lee ◽  
Woojin Yun ◽  
Young-Gyu Kim ◽  
...  
Keyword(s):  
Real Time ◽  
Depth Map ◽  
Single Camera ◽  
Camera System

scholarly journals Real-time Estimation of Road Surfaces using Fast Monocular Depth Estimation and Normal Vector Clustering

2021 ◽  
Vol 5 (3) ◽  
pp. 206
Author(s):  
Chuho Yi ◽  
Jungwon Cho
Keyword(s):  
Real Time ◽  
Autonomous Vehicle ◽  
Depth Map ◽  
Time Estimation ◽  
Weather Conditions ◽  
Depth Estimation ◽  
Normal Vector ◽  
Single Camera ◽  
Distance Information ◽  
Lower Accuracy

Estimating a road surface or planes for applying AR(Augmented Reality) or an autonomous vehicle using a camera requires significant computation. Vision sensors have lower accuracy in distance measurement than other types of sensor, and have the difficulty that additional algorithms for estimating data must be included. However, using a camera has the advantage of being able to extract various information such as weather conditions, sign information, and road markings that are difficult to measure with other sensors. Various methods differing in sensor type and configuration have been applied. Many of the existing studies had generally researched by performing the depth estimation after the feature extraction. However, recent studies have suggested using deep learning to skip multiple processes and use a single DNN(Deep Neural Network). Also, a method using a limited single camera instead of a method using a plurality of sensors has been proposed. This paper presents a single-camera method that performs quickly and efficiently by employing a DNN to extract distance information using a single camera, and proposes a modified method for using a depth map to obtain real-time surface characteristics. First, a DNN is used to estimate the depth map, and then for quick operation, normal vector that can connect similar planes to depth is calculated, and a clustering method that can be connected is provided. An experiment is used to show the validity of our method, and to evaluate the calculation time.

scholarly journals A Residual Network and FPGA Based Real-Time Depth Map Enhancement System

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 546
Author(s):  
Zhenni Li ◽  
Haoyi Sun ◽  
Yuliang Gao ◽  
Jiao Wang
Keyword(s):  
Real Time ◽  
Super Resolution ◽  
Depth Map ◽  
Acquisition System ◽  
Depth Image ◽  
Fpga Design ◽  
Depth Sensing ◽  
Residual Network ◽  
Real Time Processing ◽  
Depth Maps

Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 fps@ (1280 × 960 + 328 × 248 × 3).

scholarly journals Stochastic-Depth Ambient Occlusion

2021 ◽  
Vol 4 (1) ◽  
pp. 1-15
Author(s):  
Jop Vermeer ◽  
Leonardo Scandolo ◽  
Elmar Eisemann
Keyword(s):  
Real Time ◽  
High Performance ◽  
Depth Map ◽  
Visual Quality ◽  
Ambient Light ◽  
Missing Information ◽  
Time Performance ◽  
Ambient Occlusion ◽  
Depth Buffer ◽  
Screen Space

Ambient occlusion (AO) is a popular rendering technique that enhances depth perception and realism by darkening locations that are less exposed to ambient light (e.g., corners and creases). In real-time applications, screen-space variants, relying on the depth buffer, are used due to their high performance and good visual quality. However, these only take visible surfaces into account, resulting in inconsistencies, especially during motion. Stochastic-Depth Ambient Occlusion is a novel AO algorithm that accounts for occluded geometry by relying on a stochastic depth map, capturing multiple scene layers per pixel at random. Hereby, we efficiently gather missing information in order to improve upon the accuracy and spatial stability of conventional screen-space approximations, while maintaining real-time performance. Our approach integrates well into existing rendering pipelines and improves the robustness of many different AO techniques, including multi-view solutions.

scholarly journals Robust hand gesture recognition using multiple shape-oriented visual cues

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Samy Bakheet ◽  
Ayoub Al-Hamadi
Keyword(s):  
Real Time ◽  
Gesture Recognition ◽  
Pose Estimation ◽  
Depth Map ◽  
Hand Gesture Recognition ◽  
Support Vector ◽  
Hand Gesture ◽  
Hand Pose Estimation ◽  
Time Operation ◽  
Hand Pose

AbstractRobust vision-based hand pose estimation is highly sought but still remains a challenging task, due to its inherent difficulty partially caused by self-occlusion among hand fingers. In this paper, an innovative framework for real-time static hand gesture recognition is introduced, based on an optimized shape representation build from multiple shape cues. The framework incorporates a specific module for hand pose estimation based on depth map data, where the hand silhouette is first extracted from the extremely detailed and accurate depth map captured by a time-of-flight (ToF) depth sensor. A hybrid multi-modal descriptor that integrates multiple affine-invariant boundary-based and region-based features is created from the hand silhouette to obtain a reliable and representative description of individual gestures. Finally, an ensemble of one-vs.-all support vector machines (SVMs) is independently trained on each of these learned feature representations to perform gesture classification. When evaluated on a publicly available dataset incorporating a relatively large and diverse collection of egocentric hand gestures, the approach yields encouraging results that agree very favorably with those reported in the literature, while maintaining real-time operation.

scholarly journals Joint upsampling and noise reduction for real-time depth map enhancement

2014 ◽  
Author(s):  
Kazuki Matsumoto ◽  
Chiyoung Song ◽  
Francois de Sorbier ◽  
Hideo Saito
Keyword(s):  
Noise Reduction ◽  
Real Time ◽  
Depth Map

Real-Time System for Counting the Number of Passing People Using a Single Camera

2003 ◽  
pp. 466-473 ◽  
Author(s):  
Jae-Won Kim ◽  
Kang-Sun Choi ◽  
Byeong-Doo Choi ◽  
Jae-Yong Lee ◽  
Sung-Jea Ko
Keyword(s):  
Real Time ◽  
Single Camera ◽  
Time System ◽  
Real Time System

Single Camera System for Multi-Parametric Fluorescence Imaging of the Heart

2011 ◽  
Vol 27 (Supplement) ◽  
pp. OP05_3
Author(s):  
Takeshi Yamanaka ◽  
Tatsuhiko Arafune ◽  
Nitaro Shibata ◽  
Haruo Honjo ◽  
Kaichiro Kamiya ◽  
...  
Keyword(s):  
Fluorescence Imaging ◽  
Single Camera ◽  
Camera System

scholarly journals Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Electronics ◽  
2020 ◽  
Vol 9 (3) ◽  
pp. 451 ◽  
Author(s):  
Limin Guan ◽  
Yi Chen ◽  
Guiping Wang ◽  
Xu Lei
Keyword(s):  
Real Time ◽  
Color Image ◽  
Evidence Theory ◽  
Depth Map ◽  
Vehicle Detection ◽  
Detection Accuracy ◽  
Distance Information ◽  
Detection Model ◽  
Detection Mode ◽  
Single Sensor

Vehicle detection is essential for driverless systems. However, the current single sensor detection mode is no longer sufficient in complex and changing traffic environments. Therefore, this paper combines camera and light detection and ranging (LiDAR) to build a vehicle-detection framework that has the characteristics of multi adaptability, high real-time capacity, and robustness. First, a multi-adaptive high-precision depth-completion method was proposed to convert the 2D LiDAR sparse depth map into a dense depth map, so that the two sensors are aligned with each other at the data level. Then, the You Only Look Once Version 3 (YOLOv3) real-time object detection model was used to detect the color image and the dense depth map. Finally, a decision-level fusion method based on bounding box fusion and improved Dempster–Shafer (D–S) evidence theory was proposed to merge the two results of the previous step and obtain the final vehicle position and distance information, which not only improves the detection accuracy but also improves the robustness of the whole framework. We evaluated our method using the KITTI dataset and the Waymo Open Dataset, and the results show the effectiveness of the proposed depth completion method and multi-sensor fusion strategy.

Sign in / Sign up

Export Citation Format

Share Document