scholarly journals Real-Time FPGA Accelerated Stereo Matching for Temporal Statistical Pattern Projector Systems

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6435
Author(s):  
Zan Brus ◽  
Marko Kos ◽  
Matic Erker ◽  
Iztok Kramberger
Keyword(s):  
Real Time ◽  
Stereo Matching ◽  
Input Image ◽  
Stereo Image ◽  
Image Pair ◽  
Performance Per Watt ◽  
Statistical Pattern ◽  
And Performance ◽  
Image Pairs ◽  
High Level

The presented paper describes a hardware-accelerated field programmable gate array (FPGA)–based solution capable of real-time stereo matching for temporal statistical pattern projector systems. Modern 3D measurement systems have seen an increased use of temporal statistical pattern projectors as their active illumination source. The use of temporal statistical patterns in stereo vision systems includes the advantage of not requiring information about pattern characteristics, enabling a simplified projector design. Stereo-matching algorithms used in such systems rely on the locally unique temporal changes in brightness to establish a pixel correspondence between the stereo image pair. Finding the temporal correspondence between individual pixels in temporal image pairs is computationally expensive, requiring GPU-based solutions to achieve real-time calculation. By leveraging a high-level synthesis approach, matching cost simplification, and FPGA-specific design optimizations, an energy-efficient, high throughput stereo-matching solution was developed. The design is capable of calculating disparity images on a 1024 × 1024(@291 FPS) input image pair stream at 8.1 W on an embedded FPGA platform (ZC706). Several different design configurations were tested, evaluating device utilization, throughput, power consumption, and performance-per-watt. The average performance-per-watt of the FPGA solution was two times higher than in a GPU-based solution.

scholarly journals Efficient Stereo Matching with Decoupled Dissimilarity Measure Using Successive Weighted Summation

2014 ◽  
Vol 2014 ◽  
pp. 1-9
Author(s):  
Cheng-Tao Zhu ◽  
Yau-Zen Chang ◽  
Huai-Ming Wang ◽  
Kai He ◽  
Shih-Tseng Lee ◽  
...  
Keyword(s):  
Stereo Matching ◽  
Dissimilarity Measure ◽  
Stereo Image ◽  
Fitting Procedure ◽  
Comparable Performance ◽  
Novel Method ◽  
Disparity Maps ◽  
Image Pairs ◽  
Benchmark Evaluation ◽  
Complexity Algorithm

Developing matching algorithms from stereo image pairs to obtain correct disparity maps for 3D reconstruction has been the focus of intensive research. A constant computational complexity algorithm to calculate dissimilarity aggregation in assessing disparity based on separable successive weighted summation (SWS) among horizontal and vertical directions was proposed but still not satisfactory. This paper presents a novel method which enables decoupled dissimilarity measure in the aggregation, further improving the accuracy and robustness of stereo correspondence. The aggregated cost is also used to refine disparities based on a local curve-fitting procedure. According to our experimental results on Middlebury benchmark evaluation, the proposed approach has comparable performance when compared with the selected state-of-the-art algorithms and has the lowest mismatch rate. Besides, the refinement procedure is shown to be capable of preserving object boundaries and depth discontinuities while smoothing out disparity maps.

scholarly journals A Contribution for Virtual Prototyping of Mechatronic Systems Based on Real-Time Distributed High Level Architecture

2011 ◽  
Vol 12 (1) ◽  
Author(s):  
H. J. Hadj-Amor ◽  
T. Soriano
Keyword(s):  
Real Time ◽  
Geometric Model ◽  
Virtual Prototyping ◽  
Rapid Evolution ◽  
Behavior Model ◽  
Hybrid Automata ◽  
Mechatronic Systems ◽  
Work Related ◽  
And Performance ◽  
High Level

Mechatronics is the integration of different sciences and techniques of mechanical engineering, automatic control, electronics, and informatics. The rapid evolution of the market competitors requires the reduction of development time of a product while increasing the quality and performance. It is, therefore, necessary to increase the efficiency of the design process. To meet this need, simulation and, especially, virtual prototyping have become a key technology. It is difficult to find simulation tools are able to analyze multidependent systems of different areas. However, an environment that allows a simulation integrating multidisciplinary mechatronic systems is necessary. This paper describes a method of design and simulation of mechatronic systems. First, we identify the behavior model and its associated 3D geometric model. The behavior model is seen as a dynamic hybrid system of two coupled hybrid automata (operative part and control part). Then, we present OpenMASK and OpenModelica simulators, the IEEE1516 standard HLA and work related to this distributed architecture for simulation. In a top-down approach, we present our method and experiments to integrate HLA functionalities in these simulators and to distribute the modeling elements of mechatronic systems. Also, we propose extensions to integrate real-time for interactive simulations. Finally, we apply our approach on a representative example of a mechatronic system.

Real-Time Primary Image Processing

2013 ◽  
pp. 33-53
Author(s):  
Radu Dobrescu ◽  
Dan Popescu
Keyword(s):  
Image Processing ◽  
Real Time ◽  
Median Filter ◽  
Processing System ◽  
Input Image ◽  
Sensing Element ◽  
Software Configuration ◽  
Software Algorithms ◽  
Primary Image ◽  
High Level

Image processing operations have been classified into three main levels, namely low (primary), intermediate, and high. In order to combine speed and flexibility, an optimum hardware/software configuration is required. For multitask primary processing, a pipeline configuration is proposed. This structure, which is an interface between the sensing element (camera) and the main processing system, achieves real time video signal preprocessing, during the image acquisition time. In order to form the working neighborhoods, the input image signal is delayed (two lines and three pixels). Thus, locally 3×3 type processing modules are created. A successive comparison median filter and a logical filter for edge detection are implemented for a pipeline configuration. On the other hand, for low level, intermediate, and high level operations, software algorithms on parallel platforms are proposed. Finally, a case study of lines detection using directional filter discusses the performance dependency on number of processors.

High-End Storage

2012 ◽  
pp. 132-152
Author(s):  
Isabel Schwerdtfeger
Keyword(s):  
Real Time ◽  
Data Storage ◽  
High Performance ◽  
Storage System ◽  
Real Time Processing ◽  
Storage Solutions ◽  
Future Demands ◽  
And Performance ◽  
Extreme Scale ◽  
High Level

This chapter discusses the challenges high-end storage solutions will have with future demands. Due to heavy end-user demands for real-time processing of data access, this need must be addressed by high-end storage solutions. But what type of high-end storage solutions address this need and are suitable to ensure high performance write and retrieval of data in real-time from high- end storage infrastructures, including read and write access from digital archives? For this reason, this chapter reviews a few disk and tape solutions as well as combined disk- and tape storage solutions. The review on the different storage solutions does not focus on compliance of data storage management, but on available commercial high-end systems, addressing scalability and performance requirements both for online storage and archives. High level requirements aid in identifying high-end storage system features and support Extreme Scale infrastructures for the amount of data that high-end storage systems will need to manage in future.

scholarly journals Robust Stereo Visual-Inertial Odometry Using Nonlinear Optimization

Sensors ◽  
2019 ◽  
Vol 19 (17) ◽  
pp. 3747 ◽  
Author(s):  
Ma ◽  
Bai ◽  
Wang ◽  
Fang
Keyword(s):  
Nonlinear Optimization ◽  
Stereo Matching ◽  
Measurement Model ◽  
Stereo Image ◽  
Measurement Unit ◽  
Experimental Comparison ◽  
The Public ◽  
Stereo Cameras ◽  
Image Pairs ◽  
The Cost

The fusion of visual and inertial odometry has matured greatly due to the complementarity of the two sensors. However, the use of high-quality sensors and powerful processors in some applications is difficult due to size and cost limitations, and there are also many challenges in terms of robustness of the algorithm and computational efficiency. In this work, we present VIO-Stereo, a stereo visual-inertial odometry (VIO), which jointly combines the measurements of the stereo cameras and an inexpensive inertial measurement unit (IMU). We use nonlinear optimization to integrate visual measurements with IMU readings in VIO tightly. To decrease the cost of computation, we use the FAST feature detector to improve its efficiency and track features by the KLT sparse optical flow algorithm. We also incorporate accelerometer bias into the measurement model and optimize it together with other variables. Additionally, we perform circular matching between the previous and current stereo image pairs in order to remove outliers in the stereo matching and feature tracking steps, thus reducing the mismatch of feature points and improving the robustness and accuracy of the system. Finally, this work contributes to the experimental comparison of monocular visual-inertial odometry and stereo visual-inertial odometry by evaluating our method using the public EuRoC dataset. Experimental results demonstrate that our method exhibits competitive performance with the most advanced techniques.

A Fast Binocular Vision Stereo Matching Algorithm

2014 ◽  
Vol 556-562 ◽  
pp. 3735-3738
Author(s):  
Hui Zhang ◽  
Ling Tao Zhang ◽  
Yi Ren
Keyword(s):  
Stereo Matching ◽  
Computational Cost ◽  
Stereo Image ◽  
Feature Points ◽  
Image Pair ◽  
Canny Operator ◽  
Canny Edge Detection ◽  
Matching Algorithm ◽  
Canny Edge ◽  
Image Edge

In this paper we present a fast indoor stereo matching algorithm based on canny edge detection and line moments. We first detect image edge by using Canny operator, then find the target objects according line moments, the feature points of the objects’ contours are extracted. Finally, matching the pixel in stereo image pair according the angle vector. The algorithm effectively reduces the computational complexity, computational cost is decreased greatly. The experimental results show that the algorithm is possible and valid.

scholarly journals BICOS—An Algorithm for Fast Real-Time Correspondence Search for Statistical Pattern Projection-Based Active Stereo Sensors

2019 ◽  
Vol 9 (16) ◽  
pp. 3330 ◽  
Author(s):  
Patrick Dietrich ◽  
Stefan Heist ◽  
Martin Landmann ◽  
Peter Kühmstedt ◽  
Gunther Notni
Keyword(s):  
Real Time ◽  
High Speed ◽  
Cross Correlation ◽  
Stereo Matching ◽  
A Priori ◽  
Computer Hardware ◽  
Pattern Projection ◽  
Statistical Pattern ◽  
Image Area ◽  
Correspondence Search

Pattern projection-based 3D measurement systems are widely used for contactless, non-destructive optical 3D shape measurements. In addition, many robot-operated automation tasks require real-time reconstruction of accurate 3D data. In previous works, we have demonstrated 3D scanning based on statistical pattern projection-aided stereo matching between two cameras. One major advantage of this technology is that the actually projected patterns do not have to be known a priori in the reconstruction software. This allows much simpler projector designs and enables high-speed projection. However, to find corresponding pixels between cameras, it is necessary to search the best match amongst all pixels within the geometrically possible image area (that is, within a range on the corresponding epipolar line). The well-established method for this search is to compare each candidate pixel by temporal normalized cross correlation of the brightness value sequences of both pixels. This is computationally expensive and interdicts fast real-time applications on inexpensive computer hardware. We show two variants of our algorithm “Binary Correspondence Search” (BICOS), which solve this task in significantly reduced calculation time. In practice, our algorithm is much faster than traditional, purely cross-correlation-based search while maintaining a similar level of accuracy.

Sign in / Sign up

Export Citation Format

Share Document