Robust Data Association Using Fusion of Data-Driven and Engineered Features for Real-Time Pedestrian Tracking in Thermal Images

Object tracking is an essential problem in computer vision that has been extensively researched for decades. Tracking objects in thermal images is particularly difficult because of the lack of color information, low image resolution, or high similarity between objects of the same class. One of the main challenges in multi-object tracking, also referred to as the data association problem, is finding the correct correspondences between measurements and tracks and adapting the object appearance changes over time. We addressed this challenge of data association for thermal images by proposing three contributions. The first contribution consisted of the creation of a data-driven appearance score using five Siamese Networks, which operate on the image detection and on parts of it. Secondly, we engineered an original edge-based descriptor that improves the data association process. Lastly, we proposed a dataset consisting of pedestrian instances that were recorded in different scenarios and are used for training the Siamese Networks. The data-driven part of the data association score offers robustness, while feature engineering offers adaptability to unknown scenarios and their combination leads to a more powerful tracking solution. Our approach had a running time of 25 ms and achieved an average precision of 86.2% on publicly available benchmarks, containing real-world scenarios, as shown in the evaluation section.

Download Full-text

A Two-Stage Data Association Approach for 3D Multi-Object Tracking

Sensors ◽

10.3390/s21092894 ◽

2021 ◽

Vol 21 (9) ◽

pp. 2894

Author(s):

Minh-Quan Dao ◽

Vincent Frémont

Keyword(s):

Object Detection ◽

Object Tracking ◽

Moving Objects ◽

Data Association ◽

Autonomous Driving ◽

Tracking Accuracy ◽

Two Stage ◽

Bipartite Matching ◽

3D Object ◽

3D Object Detection

Multi-Object Tracking (MOT) is an integral part of any autonomous driving pipelines because it produces trajectories of other moving objects in the scene and predicts their future motion. Thanks to the recent advances in 3D object detection enabled by deep learning, track-by-detection has become the dominant paradigm in 3D MOT. In this paradigm, a MOT system is essentially made of an object detector and a data association algorithm which establishes track-to-detection correspondence. While 3D object detection has been actively researched, association algorithms for 3D MOT has settled at bipartite matching formulated as a Linear Assignment Problem (LAP) and solved by the Hungarian algorithm. In this paper, we adapt a two-stage data association method which was successfully applied to image-based tracking to the 3D setting, thus providing an alternative for data association for 3D MOT. Our method outperforms the baseline using one-stage bipartite matching for data association by achieving 0.587 Average Multi-Object Tracking Accuracy (AMOTA) in NuScenes validation set and 0.365 AMOTA (at level 2) in Waymo test set.

Download Full-text

Multi-Scale Anti-Occlusion Correlation Filters Object Tracking Method Based on Complementary Features

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001421550028 ◽

2020 ◽

pp. 2155002

Author(s):

Xiuhua Hu ◽

Yuan Chen ◽

Yan Hui ◽

Yingyu Liang ◽

Guiping Li ◽

...

Keyword(s):

Object Tracking ◽

Scale Model ◽

Correlation Filters ◽

Motion Feature ◽

Tracking Method ◽

Object State ◽

Multi Scale ◽

Tracking Process ◽

Object Appearance ◽

Response Peak

Aiming to tackle the problem of tracking drift easily caused by complex factors during the tracking process, this paper proposes an improved object tracking method under the framework of kernel correlation filter. To achieve discriminative information that is not sensitive to object appearance change, it combines dimensionality-reduced Histogram of Oriented Gradients features and Lab color features, which can be used to exploit the complementary characteristics robustly. Based on the idea of multi-resolution pyramid theory, a multi-scale model of the object is constructed, and the optimal scale for tracking the object is found according to the confidence maps’ response peaks of different sizes. For the case that tracking failure can easily occur when there exists inappropriate updating in the model, it detects occlusion based on whether the occlusion rate of the response peak corresponding to the best object state is less than a set threshold. At the same time, Kalman filter is used to record the motion feature information of the object before occlusion, and predict the state of the object disturbed by occlusion, which can achieve robust tracking of the object affected by occlusion influence. Experimental results show the effectiveness of the proposed method in handling various internal and external interferences under challenging environments.

Download Full-text

Multiple Context Features in Siamese Networks for Visual Object Tracking

Lecture Notes in Computer Science - Computer Vision – ECCV 2018 Workshops ◽

10.1007/978-3-030-11009-3_6 ◽

2019 ◽

pp. 116-131

Author(s):

Henrique Morimitsu

Keyword(s):

Object Tracking ◽

Visual Object ◽

Visual Object Tracking ◽

Multiple Context ◽

Context Features ◽

Siamese Networks

Download Full-text

A data association algorithm for multiple object tracking in video sequences

IEE Seminar on Target Tracking: Algorithms and Applications ◽

10.1049/ic:20060565 ◽

2006 ◽

Cited By ~ 6

Author(s):

M.H. Jaward ◽

L. Mihaylova ◽

N. Canagarajah ◽

D. Bull

Keyword(s):

Object Tracking ◽

Data Association ◽

Multiple Object Tracking ◽

Video Sequences ◽

Multiple Object ◽

Association Algorithm

Download Full-text

A Hierarchical Association Framework for Multi-Object Tracking in Airborne Videos

Remote Sensing ◽

10.3390/rs10091347 ◽

2018 ◽

Vol 10 (9) ◽

pp. 1347 ◽

Cited By ~ 3

Author(s):

Ting Chen ◽

Andrea Pennisi ◽

Zhi Li ◽

Yanning Zhang ◽

Hichem Sahli

Keyword(s):

Object Tracking ◽

Target Tracking ◽

Moving Objects ◽

State Of The Art ◽

Data Association ◽

Multiple Object Tracking ◽

Challenging Problem ◽

Compressive Tracking ◽

Vehicle Motion ◽

Airborne Vehicle

Multi-Object Tracking (MOT) in airborne videos is a challenging problem due to the uncertain airborne vehicle motion, vibrations of the mounted camera, unreliable detections, changes of size, appearance and motion of the moving objects and occlusions caused by the interaction between moving and static objects in the scene. To deal with these problems, this work proposes a four-stage hierarchical association framework for multiple object tracking in airborne video. The proposed framework combines Data Association-based Tracking (DAT) methods and target tracking using a compressive tracking approach, to robustly track objects in complex airborne surveillance scenes. In each association stage, different sets of tracklets and detections are associated to efficiently handle local tracklet generation, local trajectory construction, global drifting tracklet correction and global fragmented tracklet linking. Experiments with challenging airborne videos show significant tracking improvement compared to existing state-of-the-art methods.

Download Full-text