Deep learning techniques‐based perfection of multi‐sensor fusion oriented human‐robot interaction system for identification of dense organisms

A novel posture motion-based spatiotemporal fused graph convolutional network (PM-STGCN) is presented for skeleton-based action recognition. Existing methods on skeleton-based action recognition focus on independently calculating the joint information in single frame and motion information of joints between adjacent frames from the human body skeleton structure and then combine the classification results. However, that does not take into consideration of the complicated temporal and spatial relationship of the human body action sequence, so they are not very efficient in distinguishing similar actions. In this work, we enhance the ability of distinguishing similar actions by focusing on spatiotemporal fusion and adaptive feature extraction for high discrimination information. Firstly, the local posture motion-based attention (LPM-TAM) module is proposed for the purpose of suppressing the skeleton sequence data with a low amount of motion in the temporal domain, and the representation of motion posture features is concentrated. Besides, the local posture motion-based channel attention module (LPM-CAM) is introduced to make use of the strongly discriminative representation between different action classes of similarity. Finally, the posture motion-based spatiotemporal fusion (PM-STF) module is constructed which fuses the spatiotemporal skeleton data by filtering out the low-information sequence and enhances the posture motion features adaptively with high discrimination. Extensive experiments have been conducted, and the results demonstrate that the proposed model is superior to the commonly used action recognition methods. The designed human-robot interaction system based on action recognition has competitive performance compared with the speech interaction system.

Download Full-text

Interface Design of a Physical Human–Robot Interaction System for Human Impedance Adaptive Skill Transfer

IEEE Transactions on Automation Science and Engineering ◽

10.1109/tase.2017.2743000 ◽

2018 ◽

Vol 15 (1) ◽

pp. 329-340 ◽

Cited By ~ 75

Author(s):

Chenguang Yang ◽

Chao Zeng ◽

Peidong Liang ◽

Zhijun Li ◽

Ruifeng Li ◽

...

Keyword(s):

Interface Design ◽

Human Robot Interaction ◽

Skill Transfer ◽

Robot Interaction ◽

Interaction System ◽

Adaptive Skill

Download Full-text

Design and verification of a human–robot interaction system for upper limb exoskeleton rehabilitation

Medical Engineering & Physics ◽

10.1016/j.medengphy.2020.01.016 ◽

2020 ◽

Vol 79 ◽

pp. 19-25

Author(s):

Wang Wendong ◽

Li Hanhao ◽

Xiao Menghan ◽

Chu Yang ◽

Yuan Xiaoqing ◽

...

Keyword(s):

Upper Limb ◽

Human Robot Interaction ◽

Robot Interaction ◽

Upper Limb Exoskeleton ◽

Interaction System

Download Full-text

Human-Robot Interaction System with Quantum-Inspired Bidirectional Associative Memory

2013 Second International Conference on Robot, Vision and Signal Processing ◽

10.1109/rvsp.2013.23 ◽

2013 ◽

Author(s):

Naoki Masuyama ◽

Chu Kiong Loo ◽

Naoyuki Kubota

Keyword(s):

Associative Memory ◽

Human Robot Interaction ◽

Bidirectional Associative Memory ◽

Robot Interaction ◽

Interaction System

Download Full-text