Continuous Human Activity Recognition in Logistics from Inertial Sensor Data using Temporal Convolutions in CNN

With the fast increase in the demand for location-based services and the proliferation of smartphones, the topic of indoor localization is attracting great interest. In indoor environments, users’ performed activities carry useful semantic information. These activities can then be used by indoor localization systems to confirm users’ current relative locations in a building. In this paper, we propose a deep-learning model based on a Convolutional Long Short-Term Memory (ConvLSTM) network to classify human activities within the indoor localization scenario using smartphone inertial sensor data. Results show that the proposed human activity recognition (HAR) model accurately identifies nine types of activities: not moving, walking, running, going up in an elevator, going down in an elevator, walking upstairs, walking downstairs, or going up and down a ramp. Moreover, predicted human activities were integrated within an existing indoor positioning system and evaluated in a multi-story building across several testing routes, with an average positioning error of 2.4 m. The results show that the inclusion of human activity information can reduce the overall localization error of the system and actively contribute to the better identification of floor transitions within a building. The conducted experiments demonstrated promising results and verified the effectiveness of using human activity-related information for indoor localization.

Download Full-text

A Method of Human Activity Recognition in Transitional Period

Information ◽

10.3390/info11090416 ◽

2020 ◽

Vol 11 (9) ◽

pp. 416 ◽

Cited By ~ 1

Author(s):

Lei Chen ◽

Shurui Fan ◽

Vikram Kumar ◽

Yating Jia

Keyword(s):

Support Vector Machine ◽

Activity Recognition ◽

Human Activity ◽

Inertial Sensor ◽

Human Activity Recognition ◽

Transitional Period ◽

Sensor Data ◽

Support Vector ◽

Feature Subset ◽

Posture Classification

Human activity recognition (HAR) has been increasingly used in medical care, behavior analysis, and entertainment industry to improve the experience of users. Most of the existing works use fixed models to identify various activities. However, they do not adapt well to the dynamic nature of human activities. We investigated the activity recognition with postural transition awareness. The inertial sensor data was processed by filters and we used both time domain and frequency domain of the signals to extract the feature set. For the corresponding posture classification, three feature selection algorithms were considered to select 585 features to obtain the optimal feature subset for the posture classification. And We adopted three classifiers (support vector machine, decision tree, and random forest) for comparative analysis. After experiments, the support vector machine gave better classification results than other two methods. By using the support vector machine, we could achieve up to 98% accuracy in the Multi-class classification. Finally, the results were verified by probability estimation.

Download Full-text

IMU-Based Movement Trajectory Heatmaps for Human Activity Recognition

Sensors ◽

10.3390/s20247179 ◽

2020 ◽

Vol 20 (24) ◽

pp. 7179

Author(s):

Orhan Konak ◽

Pit Wegner ◽

Bert Arnrich

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Data Augmentation ◽

Short Term Memory ◽

Inertial Sensor ◽

Human Activity Recognition ◽

Training Data ◽

Sensor Data ◽

Movement Trajectory ◽

Movement Trajectories

Recent trends in ubiquitous computing have led to a proliferation of studies that focus on human activity recognition (HAR) utilizing inertial sensor data that consist of acceleration, orientation and angular velocity. However, the performances of such approaches are limited by the amount of annotated training data, especially in fields where annotating data is highly time-consuming and requires specialized professionals, such as in healthcare. In image classification, this limitation has been mitigated by powerful oversampling techniques such as data augmentation. Using this technique, this work evaluates to what extent transforming inertial sensor data into movement trajectories and into 2D heatmap images can be advantageous for HAR when data are scarce. A convolutional long short-term memory (ConvLSTM) network that incorporates spatiotemporal correlations was used to classify the heatmap images. Evaluation was carried out on Deep Inertial Poser (DIP), a known dataset composed of inertial sensor data. The results obtained suggest that for datasets with large numbers of subjects, using state-of-the-art methods remains the best alternative. However, a performance advantage was achieved for small datasets, which is usually the case in healthcare. Moreover, movement trajectories provide a visual representation of human activities, which can help researchers to better interpret and analyze motion patterns.

Download Full-text

Studi Literatur Human Activity Recognition (HAR) Menggunakan Sensor Inersia

Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) ◽

10.29207/resti.v5i6.3665 ◽

2021 ◽

Vol 5 (6) ◽

pp. 1193-1206

Author(s):

Humaira Nur Pradani ◽

Faizal Mahananto

Keyword(s):

Data Acquisition ◽

Activity Recognition ◽

Human Activity ◽

Inertial Sensors ◽

Inertial Sensor ◽

Computation Time ◽

Human Activity Recognition ◽

Sensor Data ◽

Future Research ◽

Literature Study

Human activity recognition (HAR) is one of the topics that is being widely researched because of its diverse implementation in various fields such as health, construction, and UI / UX. As MEMS (Micro Electro Mechanical Systems) evolves, HAR data acquisition can be done more easily and efficiently using inertial sensors. Inertial sensor data processing for HAR requires a series of processes and a variety of techniques. This literature study aims to summarize the various approaches that have been used in existing research in building the HAR model. Published articles are collected from ScienceDirect, IEEE Xplore, and MDPI over the past five years (2017-2021). From the 38 studies identified, information extracted are the overview of the areas of HAR implementation, data acquisition, public datasets, pre-process methods, feature extraction approaches, feature selection methods, classification models, training scenarios, model performance, and research challenges in this topic. The analysis showed that there is still room to improve the performance of the HAR model. Therefore, future research on the topic of HAR using inertial sensors can focus on extracting and selecting more optimal features, considering the robustness level of the model, increasing the complexity of classified activities, and balancing accuracy with computation time.

Download Full-text

Human Behavior Recognition Model Based on Feature and Classifier Selection

Sensors ◽

10.3390/s21237791 ◽

2021 ◽

Vol 21 (23) ◽

pp. 7791

Author(s):

Ge Gao ◽

Zhixin Li ◽

Zhan Huan ◽

Ying Chen ◽

Jiuzhen Liang ◽

...

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Inertial Sensor ◽

Rapid Development ◽

Human Activity Recognition ◽

Sensor Data ◽

Support Vector ◽

Recognition Model ◽

Model Based ◽

Classifier Selection

With the rapid development of the computer and sensor field, inertial sensor data have been widely used in human activity recognition. At present, most relevant studies divide human activities into basic actions and transitional actions, in which basic actions are classified by unified features, while transitional actions usually use context information to determine the category. For the existing single method that cannot well realize human activity recognition, this paper proposes a human activity classification and recognition model based on smartphone inertial sensor data. The model fully considers the feature differences of different properties of actions, uses a fixed sliding window to segment the human activity data of inertial sensors with different attributes and, finally, extracts the features and recognizes them on different classifiers. The experimental results show that dynamic and transitional actions could obtain the best recognition performance on support vector machines, while static actions could obtain better classification effects on ensemble classifiers; as for feature selection, the frequency-domain feature used in dynamic action had a high recognition rate, up to 99.35%. When time-domain features were used for static and transitional actions, higher recognition rates were obtained, 98.40% and 91.98%, respectively.

Download Full-text

Similarity Embedding Networks for Robust Human Activity Recognition

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3448021 ◽

2021 ◽

Vol 15 (6) ◽

pp. 1-17

Author(s):

Chenglin Li ◽

Carrie Lu Tong ◽

Di Niu ◽

Bei Jiang ◽

Xiao Zuo ◽

...

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Real Space ◽

Human Activity Recognition ◽

Sensor Data ◽

Activity Data ◽

Extensive Evaluation ◽

Sensor Signals ◽

Public Datasets

Deep learning models for human activity recognition (HAR) based on sensor data have been heavily studied recently. However, the generalization ability of deep models on complex real-world HAR data is limited by the availability of high-quality labeled activity data, which are hard to obtain. In this article, we design a similarity embedding neural network that maps input sensor signals onto real vectors through carefully designed convolutional and Long Short-Term Memory (LSTM) layers. The embedding network is trained with a pairwise similarity loss, encouraging the clustering of samples from the same class in the embedded real space, and can be effectively trained on a small dataset and even on a noisy dataset with mislabeled samples. Based on the learned embeddings, we further propose both nonparametric and parametric approaches for activity recognition. Extensive evaluation based on two public datasets has shown that the proposed similarity embedding network significantly outperforms state-of-the-art deep models on HAR classification tasks, is robust to mislabeled samples in the training set, and can also be used to effectively denoise a noisy dataset.

Download Full-text

Non-Linear Chaotic Features-Based Human Activity Recognition

Electronics ◽

10.3390/electronics10020111 ◽

2021 ◽

Vol 10 (2) ◽

pp. 111

Author(s):

Pengjia Tu ◽

Junhuai Li ◽

Huaijun Wang ◽

Ting Cao ◽

Kan Wang

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Human Activity Recognition ◽

Human Motion ◽

Sensor Data ◽

Largest Lyapunov Exponent ◽

Optimal Delay ◽

Motion Time ◽

Non Linear ◽

Optimal Delay Time

Human activity recognition (HAR) has vital applications in human–computer interaction, somatosensory games, and motion monitoring, etc. On the basis of the human motion accelerate sensor data, through a nonlinear analysis of the human motion time series, a novel method for HAR that is based on non-linear chaotic features is proposed in this paper. First, the C-C method and G-P algorithm are used to, respectively, compute the optimal delay time and embedding dimension. Additionally, a Reconstructed Phase Space (RPS) is formed while using time-delay embedding for the human accelerometer motion sensor data. Subsequently, a two-dimensional chaotic feature matrix is constructed, where the chaotic feature is composed of the correlation dimension and largest Lyapunov exponent (LLE) of attractor trajectory in the RPS. Next, the classification algorithms are used in order to classify and recognize the two different activity classes, i.e., basic and transitional activities. The experimental results show that the chaotic feature has a higher accuracy than traditional time and frequency domain features.

Download Full-text

A software architecture for generic human activity recognition from smartphone sensor data

2017 IEEE International Workshop on Measurement and Networking (M&N) ◽

10.1109/iwmn.2017.8078368 ◽

2017 ◽

Cited By ~ 2

Author(s):

Alberto Testoni ◽

Marco Di Felice

Keyword(s):

Software Architecture ◽

Activity Recognition ◽

Human Activity ◽

Human Activity Recognition ◽

Sensor Data ◽

Smartphone Sensor

Download Full-text

RestHAR: Residual Feature Learning Transformer for Human Activity Recognition from Multi-sensor Data

10.1109/iscmi53840.2021.9654816 ◽

2021 ◽

Author(s):

Debaditya Shome

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Feature Learning ◽

Human Activity Recognition ◽

Sensor Data

Download Full-text

Iss2Image: A Novel Signal-Encoding Technique for CNN-Based Human Activity Recognition

Sensors ◽

10.3390/s18113910 ◽

2018 ◽

Vol 18 (11) ◽

pp. 3910 ◽

Cited By ~ 11

Author(s):

Taeho Hur ◽

Jaehun Bang ◽

Thien Huynh-The ◽

Jongwon Lee ◽

Jee-In Kim ◽

...

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Expert Knowledge ◽

Inertial Sensor ◽

Human Activity Recognition ◽

Learning Technology ◽

Sensor Signal ◽

Signal Encoding ◽

The Right

The most significant barrier to success in human activity recognition is extracting and selecting the right features. In traditional methods, the features are chosen by humans, which requires the user to have expert knowledge or to do a large amount of empirical study. Newly developed deep learning technology can automatically extract and select features. Among the various deep learning methods, convolutional neural networks (CNNs) have the advantages of local dependency and scale invariance and are suitable for temporal data such as accelerometer (ACC) signals. In this paper, we propose an efficient human activity recognition method, namely Iss2Image (Inertial sensor signal to Image), a novel encoding technique for transforming an inertial sensor signal into an image with minimum distortion and a CNN model for image-based activity classification. Iss2Image converts real number values from the X, Y, and Z axes into three color channels to precisely infer correlations among successive sensor signal values in three different dimensions. We experimentally evaluated our method using several well-known datasets and our own dataset collected from a smartphone and smartwatch. The proposed method shows higher accuracy than other state-of-the-art approaches on the tested datasets.

Download Full-text