Towards Activity Recognition through Multidimensional Mobile Data Fusion with a Smartphone and Deep Learning

The field of activity recognition has evolved relatively early and has attracted countless researchers. With the continuous development of science and technology, people’s research on human activity recognition is also deepening and becoming richer. Nowadays, whether it is medicine, education, sports, or smart home, various fields have developed a strong interest in activity recognition, and a series of research results have also been put into people’s real production and life. Nowadays, smart phones have become quite popular, and the technology is becoming more and more mature, and various sensors have emerged at the historic moment, so the related research on activity recognition based on mobile phone sensors has its necessity and possibility. This article will use an Android smartphone to collect the data of six basic behaviors of human, which are walking, running, standing, sitting, going upstairs, and going downstairs, through its acceleration sensor, and use the classic model of deep learning CNN (convolutional neural network) to fuse those multidimensional mobile data, using TensorFlow for model training and test evaluation. The generated model is finally transplanted to an Android phone to complete the mobile-end activity recognition system.

Download Full-text

Joint Learning of Temporal Models to Handle Imbalanced Data for Human Activity Recognition

Applied Sciences ◽

10.3390/app10155293 ◽

2020 ◽

Vol 10 (15) ◽

pp. 5293 ◽

Cited By ~ 1

Author(s):

Rebeen Ali Hamad ◽

Longzhi Yang ◽

Wai Lok Woo ◽

Bo Wei

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Smart Home ◽

Short Term Memory ◽

Recognition System ◽

Human Activity Recognition ◽

Sensor Data ◽

Temporal Models ◽

Wide Range

Human activity recognition has become essential to a wide range of applications, such as smart home monitoring, health-care, surveillance. However, it is challenging to deliver a sufficiently robust human activity recognition system from raw sensor data with noise in a smart environment setting. Moreover, imbalanced human activity datasets with less frequent activities create extra challenges for accurate activity recognition. Deep learning algorithms have achieved promising results on balanced datasets, but their performance on imbalanced datasets without explicit algorithm design cannot be promised. Therefore, we aim to realise an activity recognition system using multi-modal sensors to address the issue of class imbalance in deep learning and improve recognition accuracy. This paper proposes a joint diverse temporal learning framework using Long Short Term Memory and one-dimensional Convolutional Neural Network models to improve human activity recognition, especially for less represented activities. We extensively evaluate the proposed method for Activities of Daily Living recognition using binary sensors dataset. A comparative study on five smart home datasets demonstrate that our proposed approach outperforms the existing individual temporal models and their hybridization. Furthermore, this is particularly the case for minority classes in addition to reasonable improvement on the majority classes of human activities.

Download Full-text

Human Activity Recognition with Deep Learning: Overview, Challenges & Possibilities

10.20944/preprints202102.0349.v1 ◽

2021 ◽

Author(s):

Pranjal Kumar

Keyword(s):

Deep Learning ◽

Internet Of Things ◽

Activity Recognition ◽

Human Activity ◽

Recognition System ◽

The Internet ◽

Learning Approaches ◽

The Public ◽

Sensory Data ◽

The Internet Of Things

The growing use of sensor tools and the Internet of Things requires sensors to understand the applications. There are major difficulties in realistic situations, though, that can impact the efficiency of the recognition system. Recently, as the utility of deep learning in many fields has been shown, various deep approaches were researched to tackle the challenges of detection and recognition. We present in this review a sample of specialized deep learning approaches for the identification of sensor-based human behaviour. Next, we present the multi-modal sensory data and include information for the public databases which can be used in different challenge tasks for study. A new taxonomy is then suggested, to organize deep approaches according to challenges. Deep problems and approaches connected to problems are summarized and evaluated to provide an analysis of the ongoing advancement in science. By the conclusion of this research, we are answering unanswered issues and providing perspectives into the future.

Download Full-text

Classifying Imbalanced Multi-modal Sensor Data for Human Activity Recognition in a Smart Home using Deep Learning

2020 International Joint Conference on Neural Networks (IJCNN) ◽

10.1109/ijcnn48605.2020.9207697 ◽

2020 ◽

Author(s):

Ali A. Alani ◽

Georgina Cosma ◽

Aboozar Taherkhani

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Smart Home ◽

Human Activity Recognition ◽

Sensor Data

Download Full-text

Architecture for Organizing Context-Aware Data in Smart Home for Activity Recognition System

Distributed, Ambient, and Pervasive Interactions - Lecture Notes in Computer Science ◽

10.1007/978-3-642-39351-8_20 ◽

2013 ◽

pp. 173-182 ◽

Cited By ~ 1

Author(s):

Konlakorn Wongpatikaseree ◽

Junsoo Kim ◽

Yoshiki Makino ◽

Azman Osman Lim ◽

Yasuo Tan

Keyword(s):

Activity Recognition ◽

Smart Home ◽

Recognition System ◽

Context Aware

Download Full-text

A Survey on Deep Learning for Human Activity Recognition

ACM Computing Surveys ◽

10.1145/3472290 ◽

2022 ◽

Vol 54 (8) ◽

pp. 1-34

Author(s):

Fuqiang Gu ◽

Mu-Huan Chung ◽

Mark Chignell ◽

Shahrokh Valaee ◽

Baoding Zhou ◽

...

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Smart Home ◽

State Of The Art ◽

Human Activity Recognition ◽

Learning Methods ◽

Machine Learning Methods ◽

Comprehensive Survey ◽

Conventional Machine

Human activity recognition is a key to a lot of applications such as healthcare and smart home. In this study, we provide a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning. Although there are many surveys on HAR, they focused mainly on the taxonomy of HAR and reviewed the state-of-the-art HAR systems implemented with conventional machine learning methods. Recently, several works have also been done on reviewing studies that use deep models for HAR, whereas these works cover few deep models and their variants. There is still a need for a comprehensive and in-depth survey on HAR with recently developed deep learning methods.

Download Full-text

HuAc: Human Activity Recognition Using Crowdsourced WiFi Signals and Skeleton Data

Wireless Communications and Mobile Computing ◽

10.1155/2018/6163475 ◽

2018 ◽

Vol 2018 ◽

pp. 1-15 ◽

Cited By ~ 10

Author(s):

Linlin Guo ◽

Lei Wang ◽

Jialin Liu ◽

Wei Zhou ◽

Bingxian Lu

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Smart Home ◽

Spatial Relationship ◽

Recognition System ◽

Human Activity Recognition ◽

Security Monitoring ◽

Average Accuracy ◽

Weak Light ◽

Relationship Of

The joint of WiFi-based and vision-based human activity recognition has attracted increasing attention in the human-computer interaction, smart home, and security monitoring fields. We propose HuAc, the combination of WiFi-based and Kinect-based activity recognition system, to sense human activity in an indoor environment with occlusion, weak light, and different perspectives. We first construct a WiFi-based activity recognition dataset named WiAR to provide a benchmark for WiFi-based activity recognition. Then, we design a mechanism of subcarrier selection according to the sensitivity of subcarriers to human activities. Moreover, we optimize the spatial relationship of adjacent skeleton joints and draw out a corresponding relationship between CSI and skeleton-based activity recognition. Finally, we explore the fusion information of CSI and crowdsourced skeleton joints to achieve the robustness of human activity recognition. We implemented HuAc using commercial WiFi devices and evaluated it in three kinds of scenarios. Our results show that HuAc achieves an average accuracy of greater than 93% using WiAR dataset.

Download Full-text