Online Depth Image-Based Object Tracking with Sparse Representation and Object Detection

This paper deals with the development of an Advanced Driver Assistance System (ADAS) for a smart electric wheelchair in order to improve the autonomy of disabled people. Our use case, built from a formal clinical study, is based on the detection, depth estimation, localization and tracking of objects in wheelchair’s indoor environment, namely: door and door handles. The aim of this work is to provide a perception layer to the wheelchair, enabling this way the detection of these keypoints in its immediate surrounding, and constructing of a short lifespan semantic map. Firstly, we present an adaptation of the YOLOv3 object detection algorithm to our use case. Then, we present our depth estimation approach using an Intel RealSense camera. Finally, as a third and last step of our approach, we present our 3D object tracking approach based on the SORT algorithm. In order to validate all the developments, we have carried out different experiments in a controlled indoor environment. Detection, distance estimation and object tracking are experimented using our own dataset, which includes doors and door handles.

Download Full-text

A Two-Stage Data Association Approach for 3D Multi-Object Tracking

Sensors ◽

10.3390/s21092894 ◽

2021 ◽

Vol 21 (9) ◽

pp. 2894

Author(s):

Minh-Quan Dao ◽

Vincent Frémont

Keyword(s):

Object Detection ◽

Object Tracking ◽

Moving Objects ◽

Data Association ◽

Autonomous Driving ◽

Tracking Accuracy ◽

Two Stage ◽

Bipartite Matching ◽

3D Object ◽

3D Object Detection

Multi-Object Tracking (MOT) is an integral part of any autonomous driving pipelines because it produces trajectories of other moving objects in the scene and predicts their future motion. Thanks to the recent advances in 3D object detection enabled by deep learning, track-by-detection has become the dominant paradigm in 3D MOT. In this paradigm, a MOT system is essentially made of an object detector and a data association algorithm which establishes track-to-detection correspondence. While 3D object detection has been actively researched, association algorithms for 3D MOT has settled at bipartite matching formulated as a Linear Assignment Problem (LAP) and solved by the Hungarian algorithm. In this paper, we adapt a two-stage data association method which was successfully applied to image-based tracking to the 3D setting, thus providing an alternative for data association for 3D MOT. Our method outperforms the baseline using one-stage bipartite matching for data association by achieving 0.587 Average Multi-Object Tracking Accuracy (AMOTA) in NuScenes validation set and 0.365 AMOTA (at level 2) in Waymo test set.

Download Full-text

Robust object tracking based on sparse representation and incremental weighted PCA

Multimedia Tools and Applications ◽

10.1007/s11042-015-3164-6 ◽

2016 ◽

Vol 76 (2) ◽

pp. 2039-2057 ◽

Cited By ~ 3

Author(s):

Xiaofen Xing ◽

Fuhao Qiu ◽

Xiangmin Xu ◽

Chunmei Qing ◽

Yinrong Wu

Keyword(s):

Sparse Representation ◽

Object Tracking

Download Full-text

Depth image super-resolution using multi-dictionary sparse representation

2013 IEEE International Conference on Image Processing ◽

10.1109/icip.2013.6738198 ◽

2013 ◽

Cited By ~ 9

Author(s):

H. Zheng ◽

A. Bouzerdoum ◽

S. L. Phung

Keyword(s):

Sparse Representation ◽

Super Resolution ◽

Depth Image ◽

Image Super Resolution

Download Full-text

Robust object tracking based on sparse representation

10.1117/12.863437 ◽

2010 ◽

Cited By ~ 6

Author(s):

Shengping Zhang ◽

Hongxun Yao ◽

Xin Sun ◽

Shaohui Liu

Keyword(s):

Sparse Representation ◽

Object Tracking

Download Full-text

Robust template patches-based object tracking with sparse representation

Infrared and Laser Engineering ◽

10.3788/irla201948.0326003 ◽

2019 ◽

Vol 48 (3) ◽

pp. 326003

Author(s):

卢瑞涛 Lu Ruitao ◽

任世杰 Ren Shijie ◽

申璐榕 Shen Lurong ◽

杨小冈 Yang Xiaogang

Keyword(s):

Sparse Representation ◽

Object Tracking

Download Full-text

Learning a Sparse Representation for Object Detection

Computer Vision — ECCV 2002 - Lecture Notes in Computer Science ◽

10.1007/3-540-47979-1_8 ◽

2002 ◽

pp. 113-127 ◽

Cited By ~ 128

Author(s):

Shivani Agarwal ◽

Dan Roth

Keyword(s):

Object Detection ◽

Sparse Representation

Download Full-text

Deep Learning Based Object Recognition Using Physically-Realistic Synthetic Depth Scenes

Machine Learning and Knowledge Extraction ◽

10.3390/make1030051 ◽

2019 ◽

Vol 1 (3) ◽

pp. 883-903 ◽

Cited By ~ 1

Author(s):

Daulet Baimukashev ◽

Alikhan Zhilisbayev ◽

Askat Kuzdeuov ◽

Artemiy Oleinikov ◽

Denis Fadeyev ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Object Recognition ◽

Data Collection ◽

Object Detection ◽

Depth Image ◽

Cluttered Environment ◽

Depth Data ◽

Depth Images ◽

Image Dataset

Recognizing objects and estimating their poses have a wide range of application in robotics. For instance, to grasp objects, robots need the position and orientation of objects in 3D. The task becomes challenging in a cluttered environment with different types of objects. A popular approach to tackle this problem is to utilize a deep neural network for object recognition. However, deep learning-based object detection in cluttered environments requires a substantial amount of data. Collection of these data requires time and extensive human labor for manual labeling. In this study, our objective was the development and validation of a deep object recognition framework using a synthetic depth image dataset. We synthetically generated a depth image dataset of 22 objects randomly placed in a 0.5 m × 0.5 m × 0.1 m box, and automatically labeled all objects with an occlusion rate below 70%. Faster Region Convolutional Neural Network (R-CNN) architecture was adopted for training using a dataset of 800,000 synthetic depth images, and its performance was tested on a real-world depth image dataset consisting of 2000 samples. Deep object recognizer has 40.96% detection accuracy on the real depth images and 93.5% on the synthetic depth images. Training the deep learning model with noise-added synthetic images improves the recognition accuracy for real images to 46.3%. The object detection framework can be trained on synthetically generated depth data, and then employed for object recognition on the real depth data in a cluttered environment. Synthetic depth data-based deep object detection has the potential to substantially decrease the time and human effort required for the extensive data collection and labeling.

Download Full-text