Depth and Thermal Image Fusion for Human Detection with Occlusion Handling Under Poor Illumination from Mobile Robot

Author(s):  
Saipol Hadi Hasim ◽  
Rosbi Mamat ◽  
Usman Ullah Sheikh ◽  
Shamsuddin Hj. Mohd. Amin
2016 ◽  
Vol 78 (6-13) ◽  
Author(s):  
Saipol Hadi Hasim ◽  
Rosbi Mamat ◽  
Usman Ullah Sheikh ◽  
Shamsuddin Mohd Amin

In this paper, a robust surveillance system to enable robots to detect humans in indoor environments is proposed. The proposed method is based on fusing information from thermal and depth images which allows the detection of human even under occlusion. The proposed method consists of three stages; pre-processing, ROI generation and object classification. A new dataset was developed to evaluate the performance of the proposed method. The experimental results show that the proposed method is able to detect multiple humans under occlusions and illumination variations.  


2016 ◽  
Vol 20 (suppl. 5) ◽  
pp. 1553-1559 ◽  
Author(s):  
Ivan Ciric ◽  
Zarko Cojbasic ◽  
Danijela Ristic-Durrant ◽  
Vlastimir Nikolic ◽  
Milica Ciric ◽  
...  

This paper presents the results of the authors in thermal vision based mobile robot control. The most important segment of the high level control loop of mobile robot platform is an intelligent real-time algorithm for human detection and tracking. Temperature variations across same objects, air flow with different temperature gradients, reflections, person overlap while crossing each other, and many other non-linearities, uncertainty and noise, put challenges in thermal image processing and therefore the need of computationally intelligent algorithms for obtaining the efficient performance from human motion tracking system. The main goal was to enable mobile robot platform or any technical system to recognize the person in indoor environment, localize it and track it with accuracy high enough to allow adequate human-machine interaction. The developed computationally intelligent algorithms enables robust and reliable human detection and tracking based on neural network classifier and autoregressive neural network for time series prediction. Intelligent algorithm used for thermal image segmentation gives accurate inputs for classification.


Sensors ◽  
2013 ◽  
Vol 13 (9) ◽  
pp. 11603-11635 ◽  
Author(s):  
Efstathios Fotiadis ◽  
Mario Garzón ◽  
Antonio Barrientos
Keyword(s):  

Author(s):  
Weidong Wang ◽  
Chengjin Du ◽  
Zhijiang Du

Purpose This paper aims to present a prototype of medical transportation robot whose positioning accuracy can reach millimeter-level in terms of patient transportation. By using this kind of mobile robot, a fully automatic image diagnosis process among independent CT/PET devices and the image fusion can be achieved. Design/methodology/approach Following a short introduction, a large-load 4WD-4WS (four-wheel driving and four-wheel steering) mobile robot for carrying patient among multiple medical imaging equipments is developed. At the same time, a specially designed bedplate with self-locking function is also introduced. For further improving the positioning accuracy, the authors proposed a calibration method based on Gaussian process regression (GPR) to process the measuring data of the sensors. The performance of this robot is verified by the calibration experiment and Image fusion experiment. Finally, concluding comments are drawn. Findings By calibrating the robot’s positioning system through the proposed GPR method, one can obtain the accuracy of the robot’s offset distance and deflection angle, which are 0.50 mm and +0.21°, respectively. Independent repeated trials were then set up to verify this result. Subsequent phantom experiment shows the accuracy of image fusion can be accurate within 0.57 mm in the front-rear direction and 0.83 in the left-right direction, respectively, while the clinical experiment shows that the proposed robot can practically realize the transportation of patient and image fusion between multiple imaging diagnosis devices. Practical implications The proposed robot offers an economical image fusion solution for medical institutions whose imaging diagnosis system basically comprises independent MRI, CT and PET devices. Also, a fully automatic diagnosis process can be achieved so that the patient’s suffering of getting in and out of the bed and the doctor’s radiation dose can be obviated. Social implications The general bedplate presented in Section 2 that can be mounted on the CT and PET devices and the self-locking mechanism has realized the catching and releasing motion of the patient on different medical devices. They also provide a detailed method regarding patient handling and orientation maintenance, which was hardly mentioned in previous research. By establishing the positioning system between the robot and different medical equipment, a fully automatic diagnosis process can be achieved so that the patient’s suffering of getting in and out of the bed and the doctor’s radiation dose can be obviated. Originality/value The GPR-based method proposed in this paper offers a novel method for enhancing the positioning accuracy of the industrial AGV while the transportation robot proposed in this paper also offers a solution for modern imaging fusion diagnosis, which are basically predicated on the conjoint analysis between different kinds of medical devices.


Sign in / Sign up

Export Citation Format

Share Document