Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network

Emotion recognition has drawn consistent attention from researchers recently. Although gesture modality plays an important role in expressing emotion, it is seldom considered in the field of emotion recognition. A key reason is the scarcity of labeled data containing 3D skeleton data. Some studies in action recognition have applied graph-based neural networks to explicitly model the spatial connection between joints. However, this method has not been considered in the field of gesture-based emotion recognition, so far. In this work, we applied a pose estimation based method to extract 3D skeleton coordinates for IEMOCAP database. We propose a self-attention enhanced spatial temporal graph convolutional network for skeleton-based emotion recognition, in which the spatial convolutional part models the skeletal structure of the body as a static graph, and the self-attention part dynamically constructs more connections between the joints and provides supplementary information. Our experiment demonstrates that the proposed model significantly outperforms other models and that the features of the extracted skeleton data improve the performance of multimodal emotion recognition.

Download Full-text

Multi-Channel Temporal Graph Convolutional Network for Stock Return Prediction

2020 IEEE 18th International Conference on Industrial Informatics (INDIN) ◽

10.1109/indin45582.2020.9442196 ◽

2020 ◽

Author(s):

Jifeng Sun ◽

Jianwu Lin ◽

Yi Zhou

Keyword(s):

Stock Return ◽

Convolutional Network ◽

Stock Return Prediction ◽

Temporal Graph

Download Full-text

Adieu recurrence? End-to-end speech emotion recognition using a context stacking dilated convolutional network

2020 28th European Signal Processing Conference (EUSIPCO) ◽

10.23919/eusipco47968.2020.9287667 ◽

2021 ◽

Author(s):

Duowei Tang ◽

Peter Kuppens ◽

Luc Geurts ◽

Toon van Waterschoot

Keyword(s):

Emotion Recognition ◽

Speech Emotion Recognition ◽

Convolutional Network ◽

End To End

Download Full-text

Cyber—Physical Attack Detection in Water Distribution Systems with Temporal Graph Convolutional Neural Networks

Water ◽

10.3390/w13091247 ◽

2021 ◽

Vol 13 (9) ◽

pp. 1247

Author(s):

Lydia Tsiami ◽

Christos Makropoulos

Keyword(s):

Neural Networks ◽

Distribution System ◽

Distribution Systems ◽

Water Distribution ◽

Attack Detection ◽

Convolutional Network ◽

Irreversible Damage ◽

Physical Attack ◽

Temporal Graph ◽

Graph Neural Networks

Prompt detection of cyber–physical attacks (CPAs) on a water distribution system (WDS) is critical to avoid irreversible damage to the network infrastructure and disruption of water services. However, the complex interdependencies of the water network’s components make CPA detection challenging. To better capture the spatiotemporal dimensions of these interdependencies, we represented the WDS as a mathematical graph and approached the problem by utilizing graph neural networks. We presented an online, one-stage, prediction-based algorithm that implements the temporal graph convolutional network and makes use of the Mahalanobis distance. The algorithm exhibited strong detection performance and was capable of localizing the targeted network components for several benchmark attacks. We suggested that an important property of the proposed algorithm was its explainability, which allowed the extraction of useful information about how the model works and as such it is a step towards the creation of trustworthy AI algorithms for water applications. Additional insights into metrics commonly used to rank algorithm performance were also presented and discussed.

Download Full-text

A Channel-fused Dense Convolutional Network for EEG-based Emotion Recognition

IEEE Transactions on Cognitive and Developmental Systems ◽

10.1109/tcds.2020.2976112 ◽

2020 ◽

pp. 1-1 ◽

Cited By ~ 3

Author(s):

Zhongke Gao ◽

Xinmin Wang ◽

Yuxuan Yang ◽

Yanli Li ◽

Kai Ma ◽

...

Keyword(s):

Emotion Recognition ◽

Convolutional Network

Download Full-text

Based on spatio-temporal graph convolution networks with residual connection for intelligence behavior recognition

International Journal of Electrical Engineering Education ◽

10.1177/0020720921996600 ◽

2021 ◽

pp. 002072092199660

Author(s):

Yinong Zhang ◽

Shanshan Guan ◽

Cheng Xu ◽

Hongzhe Liu

Keyword(s):

Human Behavior ◽

Behavior Recognition ◽

Convolutional Network ◽

Skeleton Model ◽

Basic Technology ◽

Temporal Model ◽

Temporal Graph ◽

Spatio Temporal ◽

Human Behavior Recognition ◽

Behavior Characteristics

In the era of intelligent education, human behavior recognition based on computer vision is an important branch of pattern recognition. Human behavior recognition is a basic technology in the fields of intelligent monitoring and human-computer interaction in education. The dynamic changes of human skeleton provide important information for the recognition of educational behavior. Traditional methods usually use manual information to label or traverse rules only, resulting in limited representation capabilities and poor generalization performance of the model. In this paper, a kind of dynamic skeleton model with residual is adopted—a spatio-temporal graph convolutional network based on residual connections, which not only overcomes the limitations of previous methods, but also can learn the spatio-temporal model from the skeleton data. In the big bone NTU-RGB + D dataset, the network model not only improved the representation ability of human behavior characteristics, but also improved the generalization ability, and achieved better recognition effect than the existing model. In addition, this paper also compares the results of behavior recognition on subsets of different joint points, and finds that spatial structure division have better effects.

Download Full-text