Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamics

Human motion prediction aims at generating future frames of human motion based on an observed sequence of skeletons. Recent methods employ the latest hidden states of a recurrent neural network (RNN) to encode the historical skeletons, which can only address short-term prediction. In this work, we propose a motion context modeling by summarizing the historical human motion with respect to the current prediction. A modified highway unit (MHU) is proposed for efficiently eliminating motionless joints and estimating next pose given the motion context. Furthermore, we enhance the motion dynamic by minimizing the gram matrix loss for long-term motion prediction. Experimental results show that the proposed model can promisingly forecast the human future movements, which yields superior performances over related state-of-the-art approaches. Moreover, specifying the motion context with the activity labels enables our model to perform human motion transfer.

Download Full-text

On Retrospecting Human Dynamics with Attention

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/100 ◽

2019 ◽

Cited By ~ 1

Author(s):

Minjing Dong ◽

Chang Xu

Keyword(s):

Human Motion ◽

Motion Prediction ◽

Sufficient Information ◽

Short Term ◽

Human Dynamics ◽

Time Horizons ◽

Term Prediction ◽

Error Accumulation ◽

Short Term Prediction

Deep recurrent neural networks have achieved impressive success in forecasting human motion with a sequence to sequence architecture. However, forecasting in longer time horizons often leads to implausible human poses or converges to mean poses, because of error accumulation and difficulties in keeping track of longerterm information. To address these challenges, we propose to retrospect human dynamics with attention. A retrospection module is designed upon RNN to regularly retrospect past frames and correct mistakes in time. This significantly improves the memory of RNN and provides sufficient information for the decoder networks to generate longer term prediction. Moreover, we present a spatial attention module to explore and exploit cooperation among joints in performing a particular motion. Residual connections are also included to guarantee the performance of short term prediction. We evaluate the proposed algorithm on the largest and most challenging Human 3.6M dataset in the field. Experimental results demonstrate the necessity of investigating motion prediction in a self audit manner and the effectiveness of the proposed algorithm in both short term and long term predictions.

Download Full-text

Guest Editorial: Introduction to the Special Issue on Long-Term Human Motion Prediction

IEEE Robotics and Automation Letters ◽

10.1109/lra.2021.3077964 ◽

2021 ◽

Vol 6 (3) ◽

pp. 5613-5617

Author(s):

Luigi Palmieri ◽

Rudenko Andrey ◽

Jim Mainprice ◽

Marc Hanheide ◽

Alexandre Alahi ◽

...

Keyword(s):

Guest Editorial ◽

Human Motion ◽

Editorial Introduction ◽

Motion Prediction ◽

Special Issue ◽

Human Motion Prediction

Download Full-text

EAN: Error Attenuation Network for Long-term Human Motion Prediction

2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI) ◽

10.1109/cchi.2019.8901951 ◽

2019 ◽

Author(s):

Jie Xu ◽

Xuguang Lan ◽

Jin Li ◽

Xingyu Chen ◽

Nanning Zheng

Keyword(s):

Human Motion ◽

Motion Prediction ◽

Human Motion Prediction

Download Full-text

Few-shot Human Motion Prediction via Learning Novel Motion Dynamics

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/118 ◽

2020 ◽

Author(s):

Chuanqi Zang ◽

Mingtao Pei ◽

Yu Kong

Keyword(s):

Temporal Structure ◽

Human Motion ◽

Training Data ◽

Motion Prediction ◽

Motion Data ◽

Human Motion Prediction ◽

Novel Approach ◽

Spatio Temporal ◽

Motion Dynamics

Human motion prediction is a task where we anticipate future motion based on past observation. Previous approaches rely on the access to large datasets of skeleton data, and thus are difficult to be generalized to novel motion dynamics with limited training data. In our work, we propose a novel approach named Motion Prediction Network (MoPredNet) for few-short human motion prediction. MoPredNet can be adapted to predicting new motion dynamics using limited data, and it elegantly captures long-term dependency in motion dynamics. Specifically, MoPredNet dynamically selects the most informative poses in the streaming motion data as masked poses. In addition, MoPredNet improves its encoding capability of motion dynamics by adaptively learning spatio-temporal structure from the observed poses and masked poses. We also propose to adapt MoPredNet to novel motion dynamics based on accumulated motion experiences and limited novel motion dynamics data. Experimental results show that our method achieves better performance over state-of-the-art methods in motion prediction.

Download Full-text