Real-Time Dense Stereo Matching With ELAS on FPGA-Accelerated Embedded Devices

<p><strong>Abstract.</strong> This paper introduces a system for real-time generation of digital surface models (DSM) based on an optical multi-camera system flown on board of a manned airplane or helicopter. The system consists of high end consumer cameras, GNSS/IMU system, and on-board computers for real-time data processing. Usually, generation of digital surface models from aerial imagery is done in an off-line process, leading to delayed availability of height data. The proposed system processes data in real time on board of the aircraft and downlinks the generated DSM to a ground station. This paper evaluates the GNSS/IMU on-line solution quality and its impact on dense stereo matching. The proposed real time sliding window based bundle adjustment significantly improves image orientations and DSM quality, allowing generation of detailed digital surface models with a resolution of 2*GSD. Experiments using two flight patterns are conducted over the city of Landsberg and the resulting DSMs are evaluated against a LiDAR generated reference point cloud. The online bundle adjustment is shown to minimize the effect of systematic GNSS/IMU offsets while adding only a limited delay.</p>

Download Full-text

Using Real-time Stereo Matching for Human Gesture Detection and Tracking

TECHART Journal of Arts and Imaging Science ◽

10.15323/jais.2014.02.1.1.60 ◽

2014 ◽

Author(s):

Sungil Kang

Keyword(s):

Real Time ◽

Stereo Matching ◽

Detection And Tracking

Download Full-text

Dense stereo matching by region growing based on disparity space sampling

Journal of Computer Applications ◽

10.3724/sp.j.1087.2009.02690 ◽

2009 ◽

Vol 29 (10) ◽

pp. 2690-2692

Author(s):

Bao-hai YANG ◽

Xiao-li LIU ◽

Dai-feng ZHA

Keyword(s):

Stereo Matching ◽

Region Growing ◽

Dense Stereo Matching ◽

Disparity Space

Download Full-text

Real-Time Instance Segmentation of Traffic Videos for Embedded Devices

Sensors ◽

10.3390/s21010275 ◽

2021 ◽

Vol 21 (1) ◽

pp. 275

Author(s):

Ruben Panero Martinez ◽

Ionut Schiopu ◽

Bruno Cornelis ◽

Adrian Munteanu

Keyword(s):

Real Time ◽

Network Architecture ◽

Training Procedure ◽

Segmentation Method ◽

Embedded Devices ◽

Network Training ◽

Assignment Algorithm ◽

Ablation Study ◽

Reduced Rate ◽

Instance Segmentation

The paper proposes a novel instance segmentation method for traffic videos devised for deployment on real-time embedded devices. A novel neural network architecture is proposed using a multi-resolution feature extraction backbone and improved network designs for the object detection and instance segmentation branches. A novel post-processing method is introduced to ensure a reduced rate of false detection by evaluating the quality of the output masks. An improved network training procedure is proposed based on a novel label assignment algorithm. An ablation study on speed-vs.-performance trade-off further modifies the two branches and replaces the conventional ResNet-based performance-oriented backbone with a lightweight speed-oriented design. The proposed architectural variations achieve real-time performance when deployed on embedded devices. The experimental results demonstrate that the proposed instance segmentation method for traffic videos outperforms the you only look at coefficients algorithm, the state-of-the-art real-time instance segmentation method. The proposed architecture achieves qualitative results with 31.57 average precision on the COCO dataset, while its speed-oriented variations achieve speeds of up to 66.25 frames per second on the Jetson AGX Xavier module.

Download Full-text