Target Recovery for Robust Deep Learning-Based Person Following in Mobile Robots: Online Trajectory Prediction

The ability to predict a person’s trajectory and recover a target person in the event the target moves out of the field of view of the robot’s camera is an important requirement for mobile robots designed to follow a specific person in the workspace. This paper describes an extended work of an online learning framework for trajectory prediction and recovery, integrated with a deep learning-based person-following system. The proposed framework first detects and tracks persons in real time using the single-shot multibox detector deep neural network. It then estimates the real-world positions of the persons by using a point cloud and identifies the target person to be followed by extracting the clothes color using the hue-saturation-value model. The framework allows the robot to learn online the target trajectory prediction according to the historical path of the target person. The global and local path planners create robot trajectories that follow the target while avoiding static and dynamic obstacles, all of which are elaborately designed in the state machine control. We conducted intensive experiments in a realistic environment with multiple people and sharp corners behind which the target person may quickly disappear. The experimental results demonstrated the effectiveness and practicability of the proposed framework in the given environment.

Download Full-text

Deep-Learning-Based Indoor Human Following of Mobile Robot Using Color Feature

Sensors ◽

10.3390/s20092699 ◽

2020 ◽

Vol 20 (9) ◽

pp. 2699 ◽

Cited By ~ 2

Author(s):

Redhwan Algabri ◽

Mun-Taek Choi

Keyword(s):

Deep Learning ◽

Obstacle Avoidance ◽

Learning Algorithm ◽

Human Robot Interaction ◽

Target Person ◽

Single Shot ◽

Color Feature ◽

Mapping Algorithm ◽

Deep Learning Algorithm ◽

Illumination Changes

Human following is one of the fundamental functions in human–robot interaction for mobile robots. This paper shows a novel framework with state-machine control in which the robot tracks the target person in occlusion and illumination changes, as well as navigates with obstacle avoidance while following the target to the destination. People are detected and tracked using a deep learning algorithm, called Single Shot MultiBox Detector, and the target person is identified by extracting the color feature using the hue-saturation-value histogram. The robot follows the target safely to the destination using a simultaneous localization and mapping algorithm with the LIDAR sensor for obstacle avoidance. We performed intensive experiments on our human following approach in an indoor environment with multiple people and moderate illumination changes. Experimental results indicated that the robot followed the target well to the destination, showing the effectiveness and practicability of our proposed system in the given environment.

Download Full-text

A convergence proof of a fusion algorithm of global and local path-planning for multiple mobile robots

Proceedings of the 2001 IEEE International Symposium on Assembly and Task Planning (ISATP2001). Assembly and Disassembly in the Twenty-first Century. (Cat. No.01TH8560) ◽

10.1109/isatp.2001.929001 ◽

2002 ◽

Cited By ~ 1

Author(s):

K. Kinugawa ◽

H. Noborio

Keyword(s):

Path Planning ◽

Mobile Robots ◽

Fusion Algorithm ◽

Multiple Mobile Robots ◽

Convergence Proof ◽

Local Path Planning ◽

Local Path ◽

Global And Local

Download Full-text

Automated detection of colorectal tumors based on artificial intelligence

BMC Medical Informatics and Decision Making ◽

10.1186/s12911-020-01314-8 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Kwang-Sig Lee ◽

Sang-Hyuk Son ◽

Sang-Hyun Park ◽

Eun Sun Kim

Keyword(s):

Artificial Intelligence ◽

Deep Learning ◽

Automated Detection ◽

Single Shot ◽

Learning Framework ◽

Average Accuracy ◽

Decision Supporting ◽

Decision Supporting System ◽

Validation Set ◽

The Stability

Abstract Background This study developed a diagnostic tool to automatically detect normal, unclear and tumor images from colonoscopy videos using artificial intelligence. Methods For the creation of training and validation sets, 47,555 images in the jpg format were extracted from colonoscopy videos for 24 patients in Korea University Anam Hospital. A gastroenterologist with the clinical experience of 15 years divided the 47,555 images into three classes of Normal (25,895), Unclear (2038) and Tumor (19,622). A single shot detector, a deep learning framework designed for object detection, was trained using the 47,255 images and validated with two sets of 300 images—each validation set included 150 images (50 normal, 50 unclear and 50 tumor cases). Half of the 47,255 images were used for building the model and the other half were used for testing the model. The learning rate of the model was 0.0001 during 250 epochs (training cycles). Results The average accuracy, precision, recall, and F1 score over the category were 0.9067, 0.9744, 0.9067 and 0.9393, respectively. These performance measures had no change with respect to the intersection-over-union threshold (0.45, 0.50, and 0.55). This finding suggests the stability of the model. Conclusion Automated detection of normal, unclear and tumor images from colonoscopy videos is possible by using a deep learning framework. This is expected to provide an invaluable decision supporting system for clinical experts.

Download Full-text

Real time deep learning framework to monitor social distancing using improved single shot detector based on overhead position

Earth Science Informatics ◽

10.1007/s12145-021-00758-4 ◽

2022 ◽

Author(s):

Bharathi Gopal ◽

Anandharaj Ganesan

Keyword(s):

Deep Learning ◽

Real Time ◽

Single Shot ◽

Social Distancing ◽

Learning Framework

Download Full-text

MesoGRU: Deep Learning Framework for Mesoscale Eddy Trajectory Prediction

IEEE Geoscience and Remote Sensing Letters ◽

10.1109/lgrs.2021.3087835 ◽

2021 ◽

pp. 1-5

Author(s):

Xinning Wang ◽

Xuegong Wang ◽

Miao Yu ◽

Chong Li ◽

Dalei Song ◽

...

Keyword(s):

Deep Learning ◽

Mesoscale Eddy ◽

Trajectory Prediction ◽

Learning Framework

Download Full-text

A Deep Learning Framework for Signal Detection and Modulation Classification

Sensors ◽

10.3390/s19184042 ◽

2019 ◽

Vol 19 (18) ◽

pp. 4042 ◽

Cited By ~ 6

Author(s):

Xiong Zha ◽

Hua Peng ◽

Xin Qin ◽

Guang Li ◽

Sihan Yang

Keyword(s):

Deep Learning ◽

Signal Detection ◽

Communication Systems ◽

Center Frequency ◽

Great Success ◽

Single Shot ◽

Modulation Classification ◽

Modulation Recognition ◽

Modulation Format ◽

Learning Framework

Deep learning (DL) is a powerful technique which has achieved great success in many applications. However, its usage in communication systems has not been well explored. This paper investigates algorithms for multi-signals detection and modulation classification, which are significant in many communication systems. In this work, a DL framework for multi-signals detection and modulation recognition is proposed. Compared to some existing methods, the signal modulation format, center frequency, and start-stop time can be obtained from the proposed scheme. Furthermore, two types of networks are built: (1) Single shot multibox detector (SSD) networks for signal detection and (2) multi-inputs convolutional neural networks (CNNs) for modulation recognition. Additionally, the importance of signal representation to different tasks is investigated. Experimental results demonstrate that the DL framework is capable of detecting and recognizing signals. And compared to the traditional methods and other deep network techniques, the current built DL framework can achieve better performance.

Download Full-text