Real-time object tracking system based on field-programmable gate array and convolution neural network

Vision-based object tracking has lots of applications in robotics, like surveillance, navigation, motion capturing, and so on. However, the existing object tracking systems still suffer from the challenging problem of high computation consumption in the image processing algorithms. The problem can prevent current systems from being used in many robotic applications which have limitations of payload and power, for example, micro air vehicles. In these applications, the central processing unit- or graphics processing unit-based computers are not good choices due to the high weight and power consumption. To address the problem, this article proposed a real-time object tracking system based on field-programmable gate array, convolution neural network, and visual servo technology. The time-consuming image processing algorithms, such as distortion correction, color space convertor, and Sobel edge, Harris corner features detector, and convolution neural network were redesigned using the programmable gates in field-programmable gate array. Based on the field-programmable gate array-based image processing, an image-based visual servo controller was designed to drive a two degree of freedom manipulator to track the target in real time. Finally, experiments on the proposed system were performed to illustrate the effectiveness of the real-time object tracking system.

Download Full-text

Real-time object tracking in a video stream using Field Programmable Gate Array

2012 Annual IEEE India Conference (INDICON) ◽

10.1109/indcon.2012.6420608 ◽

2012 ◽

Cited By ~ 3

Author(s):

V M Sandeep Rao ◽

Aravind Natarajan ◽

S. Moorthi ◽

M. P. Selvan

Keyword(s):

Object Tracking ◽

Real Time ◽

Field Programmable Gate Array ◽

Video Stream ◽

Field Programmable ◽

Gate Array

Download Full-text

A system on chip-based real-time tracking system for amphibious spherical robots

International Journal of Advanced Robotic Systems ◽

10.1177/1729881417716559 ◽

2017 ◽

Vol 14 (4) ◽

pp. 172988141771655 ◽

Cited By ~ 16

Author(s):

Shuxiang Guo ◽

Shaowu Pan ◽

Xiaoqiong Li ◽

Liwei Shi ◽

Pengyi Zhang ◽

...

Keyword(s):

Real Time ◽

Field Programmable Gate Array ◽

Heterogeneous Computing ◽

Simulated Annealing Algorithm ◽

Tracking System ◽

Resource Consumption ◽

Spherical Robot ◽

Field Programmable ◽

Gate Array ◽

Future Demands

Aiming at vision applications of our amphibious spherical robot, a real-time detection and tracking system adopting Gaussian background model and compressive tracking algorithm was designed and implemented in this article. Considering the narrow load space, the limited power resource and the specialized application scenarios of the robot, a heterogeneous computing architecture combining advanced Reduced Instruction-Set Computer (RISC) machine and field programmable gate array was proposed on the basis of Zynq-7000 system on chip.Under the architecture, main parts of the vision algorithms were implemented as software programs running on the advanced RISC machine-Linux subsystem. And customized image accelerators were deployed on the field programmable gate array subsystem to speed up the time-consuming processes of visual algorithms. Moreover, dynamic reconfiguration was used to switch accelerators online for reducing resource consumption and improving system adaptability. The word length of accelerators was optimized with simulated annealing algorithm to make a compromise between calculation accuracy and resource consumption. Experimental results confirmed the feasibility of the proposed architecture. The single board tracking system was able to provide an image processing rate of up to 89.2 frames per second at the resolution of 320 × 240, which could meet future demands of our robot in biological monitoring and multi-target tracking.

Download Full-text

A Real-Time Deep Learning OFDM Receiver

ACM Transactions on Reconfigurable Technology and Systems ◽

10.1145/3494049 ◽

2022 ◽

Vol 15 (3) ◽

pp. 1-25

Author(s):

Stefan Brennsteiner ◽

Tughrul Arslan ◽

John Thompson ◽

Andrew McCormick

Keyword(s):

Neural Network ◽

Neural Networks ◽

Real Time ◽

Field Programmable Gate Array ◽

Orthogonal Frequency Division Multiplexing ◽

Frequency Division Multiplexing ◽

Frequency Division ◽

Field Programmable ◽

Gate Array ◽

Fully Connected

Machine learning in the physical layer of communication systems holds the potential to improve performance and simplify design methodology. Many algorithms have been proposed; however, the model complexity is often unfeasible for real-time deployment. The real-time processing capability of these systems has not been proven yet. In this work, we propose a novel, less complex, fully connected neural network to perform channel estimation and signal detection in an orthogonal frequency division multiplexing system. The memory requirement, which is often the bottleneck for fully connected neural networks, is reduced by ≈ 27 times by applying known compression techniques in a three-step training process. Extensive experiments were performed for pruning and quantizing the weights of the neural network detector. Additionally, Huffman encoding was used on the weights to further reduce memory requirements. Based on this approach, we propose the first field-programmable gate array based, real-time capable neural network accelerator, specifically designed to accelerate the orthogonal frequency division multiplexing detector workload. The accelerator is synthesized for a Xilinx RFSoC field-programmable gate array, uses small-batch processing to increase throughput, efficiently supports branching neural networks, and implements superscalar Huffman decoders.

Download Full-text

Efficient field‐programmable gate array‐based reconfigurable accelerator for deep convolution neural network

Electronics Letters ◽

10.1049/ell2.12121 ◽

2021 ◽

Author(s):

Xianghong Hu ◽

Taosheng Chen ◽

Hongmin Huang ◽

Zihao Liu ◽

Xueming Li ◽

...

Keyword(s):

Neural Network ◽

Field Programmable Gate Array ◽

Convolution Neural Network ◽

Field Programmable ◽

Deep Convolution Neural Network ◽

Gate Array

Download Full-text

Field programmable Gate Array based Real Time Object Tracking using Partial Least Square Analysis

International Journal of Computer Communication and Informatics ◽

10.34256/ijcci2028 ◽

2020 ◽

Vol 2 (2) ◽

pp. 95-110

Author(s):

Somasundaram D. ◽

Kumaresan N ◽

Vanitha S

Keyword(s):

Object Tracking ◽

Real Time ◽

Field Programmable Gate Array ◽

Partial Least Square ◽

Tracking Algorithm ◽

Least Square ◽

Appearance Model ◽

Field Programmable ◽

Pls Analysis ◽

Gate Array

In this paper, we proposed an object tracking algorithm in real time implementation of moving object tracking system using Field programmable gate array (FPGA). Object tracking is considered as a binary classification problem and one of the approaches to this problem is that to extract appropriate features from the appearance of the object based on partial least square (PLS) analysis method, which is a low dimension reduction technique in the subspace. In this method, the adaptive appearance model integrated with PLS analysis is used for continuous update of the appearance change of the target over time. For robust and efficient tracking, particle filtering is used in between every two consecutive frames of the video. This has implemented using Cadence and Virtuoso software integrated environment with MATLAB. The experimental results are performed on challenging video sequences to show the performance of the proposed tracking algorithm using FPGA in real time.

Download Full-text

Research and implementation of a fabric printing detection system based on a field programmable gate array and deep neural network

Textile Research Journal ◽

10.1177/00405175211048156 ◽

2021 ◽

pp. 004051752110481

Author(s):

Feng Li ◽

Qinggang Xi

Keyword(s):

Neural Network ◽

Field Programmable Gate Array ◽

Detection System ◽

Parallel Optimization ◽

Detection Accuracy ◽

Processing Unit ◽

Backbone Network ◽

The Neural Network ◽

Field Programmable ◽

Gate Array

In this paper, aiming at the problems of difficult positioning, slow speed and low precision of digital printing, a detection system suitable for textile printing positioning is proposed and designed. This detection system innovatively combines a neural network and field programmable gate array (FPGA) to realize rapid and accurate positioning of printing. In the neural network part, this paper selects the backbone network Darknet19 of YOLOv2 as the backbone network, and under the premise of ensuring a certain detection accuracy, the network model is pruned and quantified to make it suitable for deployment on the embedded device FPGA. In addition, before the network training, this paper optimizes the candidate boxes by introducing k-means clustering to customize the analysis of the fabric print dataset to improve the detection accuracy. In the FPGA part, this paper optimizes the architecture on the FPGA side in two parts: data computation and data transmission. In terms of computational optimization, parallel optimization of the neural network is performed by combining FPGA optimization methods, such as pipeline and unroll. In terms of transmission optimization, we use a double-buffered design to ping-pong in the input and output modules to overlap the latency, and then use multi-port transmission to improve the overall bandwidth utilization and reduce the transmission latency caused by on-chip and off-chip interactions. The experimental results show that the detection system combining the neural network and FPGA can effectively position fabric prints and meet the needs of real-time. The design scheme has lower power compared to the graphics processing unit and is faster compared to the central processing unit.

Download Full-text