scholarly journals Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables

Sensors ◽  
2021 ◽  
Vol 21 (24) ◽  
pp. 8337
Author(s):  
Hyeokhyen Kwon ◽  
Gregory D. Abowd ◽  
Thomas Plötz
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Large Scale ◽  
State Of The Art ◽  
Human Activity Recognition ◽  
The State ◽  
Model Parameters ◽  
Data Systems ◽  
Performance Improvements ◽  
Measurement Units

Supervised training of human activity recognition (HAR) systems based on body-worn inertial measurement units (IMUs) is often constrained by the typically rather small amounts of labeled sample data. Systems like IMUTube have been introduced that employ cross-modality transfer approaches to convert videos of activities of interest into virtual IMU data. We demonstrate for the first time how such large-scale virtual IMU datasets can be used to train HAR systems that are substantially more complex than the state-of-the-art. Complexity is thereby represented by the number of model parameters that can be trained robustly. Our models contain components that are dedicated to capture the essentials of IMU data as they are of relevance for activity recognition, which increased the number of trainable parameters by a factor of 1100 compared to state-of-the-art model architectures. We evaluate the new model architecture on the challenging task of analyzing free-weight gym exercises, specifically on classifying 13 dumbbell execises. We have collected around 41 h of virtual IMU data using IMUTube from exercise videos available from YouTube. The proposed model is trained with the large amount of virtual IMU data and calibrated with a mere 36 min of real IMU data. The trained model was evaluated on a real IMU dataset and we demonstrate the substantial performance improvements of 20% absolute F1 score compared to the state-of-the-art convolutional models in HAR.

scholarly journals Rate-Invariant Modeling in Lie Algebra for Activity Recognition

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1888
Author(s):  
Malek Boujebli ◽  
Hassen Drira ◽  
Makram Mestiri ◽  
Imed Riadh Farah
Keyword(s):  
Computer Vision ◽  
Lie Algebra ◽  
Activity Recognition ◽  
Human Activity ◽  
State Of The Art ◽  
Human Activity Recognition ◽  
High Complexity ◽  
Human Motions ◽  
Inter Frame ◽  
Automatic Systems

Human activity recognition is one of the most challenging and active areas of research in the computer vision domain. However, designing automatic systems that are robust to significant variability due to object combinations and the high complexity of human motions are more challenging. In this paper, we propose to model the inter-frame rigid evolution of skeleton parts as the trajectory in the Lie group SE(3)×…×SE(3). The motion of the object is similarly modeled as an additional trajectory in the same manifold. The classification is performed based on a rate-invariant comparison of the resulting trajectories mapped to a vector space, the Lie algebra. Experimental results on three action and activity datasets show that the proposed method outperforms various state-of-the-art human activity recognition approaches.

scholarly journals Data Augmentation vs. Domain Adaptation—A Case Study in Human Activity Recognition

Technologies ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 55
Author(s):  
Evaggelos Spyrou ◽  
Eirini Mathe ◽  
Georgios Pikramenos ◽  
Konstantinos Kechagias ◽  
Phivos Mylonas
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Data Augmentation ◽  
Domain Adaptation ◽  
Recognition Task ◽  
Human Activity Recognition ◽  
Video Data ◽  
Training Dataset ◽  
Data Systems ◽  
Big Data Systems

Recent advances in big data systems and databases have made it possible to gather raw unlabeled data at unprecedented rates. However, labeling such data constitutes a costly and timely process. This is especially true for video data, and in particular for human activity recognition (HAR) tasks. For this reason, methods for reducing the need of labeled data for HAR applications have drawn significant attention from the research community. In particular, two popular approaches developed to address the above issue are data augmentation and domain adaptation. The former attempts to leverage problem-specific, hand-crafted data synthesizers to augment the training dataset with artificial labeled data instances. The latter attempts to extract knowledge from distinct but related supervised learning tasks for which labeled data is more abundant than the problem at hand. Both methods have been extensively studied and used successfully on various tasks, but a comprehensive comparison of the two has not been carried out in the context of video data HAR. In this work, we fill this gap by providing ample experimental results comparing data augmentation and domain adaptation techniques on a cross-viewpoint, human activity recognition task from pose information.

scholarly journals A Review on Scaling Mobile Sensing Platforms for Human Activity Recognition: Challenges and Recommendations for Future Research

IoT ◽  
2020 ◽  
Vol 1 (2) ◽  
pp. 451-473
Author(s):  
Liliana I. Carvalho ◽  
Rute C. Sofia
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Large Scale ◽  
Human Activity Recognition ◽  
Mobile Sensing ◽  
Future Research ◽  
Raising Awareness ◽  
Sensing Platforms ◽  
And Behavior ◽  
Personal Devices

Mobile sensing has been gaining ground due to the increasing capabilities of mobile and personal devices that are carried around by citizens, giving access to a large variety of data and services based on the way humans interact. Mobile sensing brings several advantages in terms of the richness of available data, particularly for human activity recognition. Nevertheless, the infrastructure required to support large-scale mobile sensing requires an interoperable design, which is still hard to achieve today. This review paper contributes to raising awareness of challenges faced today by mobile sensing platforms that perform learning and behavior inference with respect to human routines: how current solutions perform activity recognition, which classification models they consider, and which types of behavior inferences can be seamlessly provided. The paper provides a set of guidelines that contribute to a better functional design of mobile sensing infrastructures, keeping scalability as well as interoperability in mind.

A Survey on Deep Learning for Human Activity Recognition

2022 ◽  
Vol 54 (8) ◽  
pp. 1-34
Author(s):  
Fuqiang Gu ◽  
Mu-Huan Chung ◽  
Mark Chignell ◽  
Shahrokh Valaee ◽  
Baoding Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Smart Home ◽  
State Of The Art ◽  
Human Activity Recognition ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Comprehensive Survey ◽  
Conventional Machine

Human activity recognition is a key to a lot of applications such as healthcare and smart home. In this study, we provide a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning. Although there are many surveys on HAR, they focused mainly on the taxonomy of HAR and reviewed the state-of-the-art HAR systems implemented with conventional machine learning methods. Recently, several works have also been done on reviewing studies that use deep models for HAR, whereas these works cover few deep models and their variants. There is still a need for a comprehensive and in-depth survey on HAR with recently developed deep learning methods.

Human Activity Recognition Algorithm in Video Sequences Based on Integration of Magnitude and Orientation Information of Optical Flow

2021 ◽  
pp. 2250009
Author(s):  
Arati Kushwaha ◽  
Ashish Khare ◽  
Manish Khare
Keyword(s):  
Optical Flow ◽  
Activity Recognition ◽  
Human Activity ◽  
State Of The Art ◽  
Human Activity Recognition ◽  
Experimental Result ◽  
Video Sequences ◽  
Histogram Of Oriented Gradients ◽  
Art Methods ◽  
Orientation Information

Human activity recognition from video sequences has emerged recently as pivotal research area due to its importance in a large number of applications such as real-time surveillance monitoring, healthcare, smart homes, security, behavior analysis, and many more. However, lots of challenges also exist such as intra-class variations, object occlusion, varying illumination condition, complex background, camera motion, etc. In this work, we introduce a novel feature descriptor based on the integration of magnitude and orientation information of optical flow and histogram of oriented gradients which gives an efficient and robust feature vector for the recognition of human activities for real-world environment. In the proposed approach first we computed magnitude and orientation of the optical flow separately then a local-oriented histogram of magnitude and orientation of motion flow vectors are computed using histogram of oriented gradients followed by linear combination feature fusion strategy. The resultant features are then processed by a multiclass Support Vector Machine (SVM) classifier for activity recognition. The experimental results are performed over different publically available benchmark video datasets such as UT interaction, CASIA, and HMDB51 datasets. The effectiveness of the proposed approach is evaluated in terms of six different performance parameters such as accuracy, precision, recall, specificity, [Formula: see text]-measure, and Matthew’s correlation coefficient (MCC). To show the significance of the proposed method, it is compared with the other state-of-the-art methods. The experimental result shows that the proposed method performs well in comparison to other state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document