Multi-modal Data Fusion Method for Human Behavior Recognition Based on Two IA-Net and CHMM

The multi-modal data fusion method based on IA-net and CHMM technical proposed is designed to solve the problem that the incompleteness of target behavior information in complex family environment leads to the low accuracy of human behavior recognition.The two improved neural networks(STA-ResNet50、STA-GoogleNet)are combined with LSTM to form two IA-Nets respectively to extract RGB and skeleton modal behavior features in video. The two modal feature sequences are input CHMM to construct the probability fusion model of multi-modal behavior recognition.The experimental results show that the human behavior recognition model proposed in this paper has higher accuracy than the previous fusion methods on HMDB51 and UCF101 datasets. New contributions: attention mechanism is introduced to improve the efficiency of video target feature extraction and utilization. A skeleton based feature extraction framework is proposed, which can be used for human behavior recognition in complex environment. In the field of human behavior recognition, probability theory and neural network are cleverly combined and applied, which provides a new method for multi-modal information fusion.

Download Full-text

Wearable Sensor Data Based Human Behavior Recognition: a Method of Data Feature Extraction

Journal of Computer-Aided Design & Computer Graphics ◽

10.3724/sp.j.1089.2021.18690 ◽

2021 ◽

Vol 33 (8) ◽

pp. 1246-1253

Author(s):

Yibo Guo ◽

Wenhua Meng ◽

Yiming Fan ◽

Lishuo Hou ◽

Yue Yuan ◽

...

Keyword(s):

Feature Extraction ◽

Human Behavior ◽

Sensor Data ◽

Wearable Sensor ◽

Behavior Recognition ◽

Human Behavior Recognition

Download Full-text

3D Human behavior recognition based on binocular vision and face–hand feature

Optik ◽

10.1016/j.ijleo.2015.08.103 ◽

2015 ◽

Vol 126 (23) ◽

pp. 4712-4717 ◽

Cited By ~ 3

Author(s):

Qing Ye ◽

Junfeng Dong ◽

Yongmei Zhang

Keyword(s):

Human Behavior ◽

Binocular Vision ◽

Behavior Recognition ◽

Human Behavior Recognition

Download Full-text

Human behavior recognition for cooking support robot

RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759) ◽

10.1109/roman.2004.1374787 ◽

2005 ◽

Cited By ~ 4

Author(s):

T. Fukuda ◽

Y. Nakauchi ◽

K. Noguchi ◽

T. Matsubara

Keyword(s):

Human Behavior ◽

Behavior Recognition ◽

Human Behavior Recognition

Download Full-text

Based on spatio-temporal graph convolution networks with residual connection for intelligence behavior recognition

International Journal of Electrical Engineering Education ◽

10.1177/0020720921996600 ◽

2021 ◽

pp. 002072092199660

Author(s):

Yinong Zhang ◽

Shanshan Guan ◽

Cheng Xu ◽

Hongzhe Liu

Keyword(s):

Human Behavior ◽

Behavior Recognition ◽

Convolutional Network ◽

Skeleton Model ◽

Basic Technology ◽

Temporal Model ◽

Temporal Graph ◽

Spatio Temporal ◽

Human Behavior Recognition ◽

Behavior Characteristics

In the era of intelligent education, human behavior recognition based on computer vision is an important branch of pattern recognition. Human behavior recognition is a basic technology in the fields of intelligent monitoring and human-computer interaction in education. The dynamic changes of human skeleton provide important information for the recognition of educational behavior. Traditional methods usually use manual information to label or traverse rules only, resulting in limited representation capabilities and poor generalization performance of the model. In this paper, a kind of dynamic skeleton model with residual is adopted—a spatio-temporal graph convolutional network based on residual connections, which not only overcomes the limitations of previous methods, but also can learn the spatio-temporal model from the skeleton data. In the big bone NTU-RGB + D dataset, the network model not only improved the representation ability of human behavior characteristics, but also improved the generalization ability, and achieved better recognition effect than the existing model. In addition, this paper also compares the results of behavior recognition on subsets of different joint points, and finds that spatial structure division have better effects.

Download Full-text