Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors

Human activity recognition (HAR) has become a popular topic in research because of its wide application. With the development of deep learning, new ideas have appeared to address HAR problems. Here, a deep network architecture using residual bidirectional long short-term memory (LSTM) is proposed. The advantages of the new network include that a bidirectional connection can concatenate the positive time direction (forward state) and the negative time direction (backward state). Second, residual connections between stacked cells act as shortcut for gradients, effectively avoiding the gradient vanishing problem. Generally, the proposed network shows improvements on both the temporal (using bidirectional cells) and the spatial (residual connections stacked) dimensions, aiming to enhance the recognition rate. When testing with the Opportunity dataset and the public domain UCI dataset, the accuracy is significantly improved compared with previous results.

Download Full-text

Wearable Sensor-Based Human Activity Recognition Using Hybrid Deep Learning Techniques

Security and Communication Networks ◽

10.1155/2020/2132138 ◽

2020 ◽

Vol 2020 ◽

pp. 1-12 ◽

Cited By ~ 1

Author(s):

Huaijun Wang ◽

Jing Zhao ◽

Junhuai Li ◽

Ling Tian ◽

Pengjia Tu ◽

...

Keyword(s):

Health Care ◽

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Specific Activity ◽

Recognition Rate ◽

Human Activity Recognition ◽

Wearable Sensor ◽

Identification Rate

Human activity recognition (HAR) can be exploited to great benefits in many applications, including elder care, health care, rehabilitation, entertainment, and monitoring. Many existing techniques, such as deep learning, have been developed for specific activity recognition, but little for the recognition of the transitions between activities. This work proposes a deep learning based scheme that can recognize both specific activities and the transitions between two different activities of short duration and low frequency for health care applications. In this work, we first build a deep convolutional neural network (CNN) for extracting features from the data collected by sensors. Then, the long short-term memory (LTSM) network is used to capture long-term dependencies between two actions to further improve the HAR identification rate. By combing CNN and LSTM, a wearable sensor based model is proposed that can accurately recognize activities and their transitions. The experimental results show that the proposed approach can help improve the recognition rate up to 95.87% and the recognition rate for transitions higher than 80%, which are better than those of most existing similar models over the open HAPT dataset.

Download Full-text

Dilated causal convolution with multi-head self attention for sensor human activity recognition

Neural Computing and Applications ◽

10.1007/s00521-021-06007-5 ◽

2021 ◽

Author(s):

Rebeen Ali Hamad ◽

Masashi Kimura ◽

Longzhi Yang ◽

Wai Lok Woo ◽

Bo Wei

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Activity Recognition ◽

Human Activity ◽

Network Architecture ◽

Short Term Memory ◽

Human Activity Recognition ◽

Sensor Data ◽

One Dimensional ◽

Temporal Models

AbstractSystems of sensor human activity recognition are becoming increasingly popular in diverse fields such as healthcare and security. Yet, developing such systems poses inherent challenges due to the variations and complexity of human behaviors during the performance of physical activities. Recurrent neural networks, particularly long short-term memory have achieved promising results on numerous sequential learning problems, including sensor human activity recognition. However, parallelization is inhibited in recurrent networks due to sequential operation and computation that lead to slow training, occupying more memory and hard convergence. One-dimensional convolutional neural network processes input temporal sequential batches independently that lead to effectively executed operations in parallel. Despite that, a one-dimensional Convolutional Neural Network is not sensitive to the order of the time steps which is crucial for accurate and robust systems of sensor human activity recognition. To address this problem, we propose a network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps. The proposed method is evaluated for human activities using smart home binary sensors data and wearable sensor data. Results of conducted extensive experiments on eight public and benchmark HAR data sets show that the proposed network outperforms the state-of-the-art models based on recurrent settings and temporal models.

Download Full-text

A deep neural network model for multi-view human activity recognition

PLoS ONE ◽

10.1371/journal.pone.0262181 ◽

2022 ◽

Vol 17 (1) ◽

pp. e0262181

Author(s):

Prasetia Utama Putra ◽

Keisuke Shima ◽

Koji Shimatani

Keyword(s):

Neural Networks ◽

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Recognition Rate ◽

Human Activity Recognition ◽

Multiple Cameras ◽

Single View ◽

Online Classification ◽

Proposed Model

Multiple cameras are used to resolve occlusion problem that often occur in single-view human activity recognition. Based on the success of learning representation with deep neural networks (DNNs), recent works have proposed DNNs models to estimate human activity from multi-view inputs. However, currently available datasets are inadequate in training DNNs model to obtain high accuracy rate. Against such an issue, this study presents a DNNs model, trained by employing transfer learning and shared-weight techniques, to classify human activity from multiple cameras. The model comprised pre-trained convolutional neural networks (CNNs), attention layers, long short-term memory networks with residual learning (LSTMRes), and Softmax layers. The experimental results suggested that the proposed model could achieve a promising performance on challenging MVHAR datasets: IXMAS (97.27%) and i3DPost (96.87%). A competitive recognition rate was also observed in online classification.

Download Full-text

Biometric User Identification Based on Human Activity Recognition Using Wearable Sensors: An Experiment Using Deep Learning Models

Electronics ◽

10.3390/electronics10030308 ◽

2021 ◽

Vol 10 (3) ◽

pp. 308

Author(s):

Sakorn Mekruksavanich ◽

Anuchit Jitpattanakul

Keyword(s):

Deep Learning ◽

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Wearable Sensors ◽

The Elderly ◽

Human Activity Recognition ◽

Learning Models ◽

User Identification ◽

Sensory Data

Currently, a significant amount of interest is focused on research in the field of Human Activity Recognition (HAR) as a result of the wide variety of its practical uses in real-world applications, such as biometric user identification, health monitoring of the elderly, and surveillance by authorities. The widespread use of wearable sensor devices and the Internet of Things (IoT) has led the topic of HAR to become a significant subject in areas of mobile and ubiquitous computing. In recent years, the most widely-used inference and problem-solving approach in the HAR system has been deep learning. Nevertheless, major challenges exist with regard to the application of HAR for problems in biometric user identification in which various human behaviors can be regarded as types of biometric qualities and used for identifying people. In this research study, a novel framework for multi-class wearable user identification, with a basis in the recognition of human behavior through the use of deep learning models, is presented. In order to obtain advanced information regarding users during the performance of various activities, sensory data from tri-axial gyroscopes and tri-axial accelerometers of the wearable devices are applied. Additionally, a set of experiments were shown to validate this work, and the proposed framework’s effectiveness was demonstrated. The results for the two basic models, namely, the Convolutional Neural Network (CNN) and the Long Short-Term Memory (LSTM) deep learning, showed that the highest accuracy for all users was 91.77% and 92.43%, respectively. With regard to the biometric user identification, these are both acceptable levels.

Download Full-text

Similarity Embedding Networks for Robust Human Activity Recognition

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3448021 ◽

2021 ◽

Vol 15 (6) ◽

pp. 1-17

Author(s):

Chenglin Li ◽

Carrie Lu Tong ◽

Di Niu ◽

Bei Jiang ◽

Xiao Zuo ◽

...

Keyword(s):

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Real Space ◽

Human Activity Recognition ◽

Sensor Data ◽

Activity Data ◽

Extensive Evaluation ◽

Sensor Signals ◽

Public Datasets

Deep learning models for human activity recognition (HAR) based on sensor data have been heavily studied recently. However, the generalization ability of deep models on complex real-world HAR data is limited by the availability of high-quality labeled activity data, which are hard to obtain. In this article, we design a similarity embedding neural network that maps input sensor signals onto real vectors through carefully designed convolutional and Long Short-Term Memory (LSTM) layers. The embedding network is trained with a pairwise similarity loss, encouraging the clustering of samples from the same class in the embedded real space, and can be effectively trained on a small dataset and even on a noisy dataset with mislabeled samples. Based on the learned embeddings, we further propose both nonparametric and parametric approaches for activity recognition. Extensive evaluation based on two public datasets has shown that the proposed similarity embedding network significantly outperforms state-of-the-art deep models on HAR classification tasks, is robust to mislabeled samples in the training set, and can also be used to effectively denoise a noisy dataset.

Download Full-text

Improving Human Activity Recognition Performance by Data Fusion and Feature Engineering

Sensors ◽

10.3390/s21030692 ◽

2021 ◽

Vol 21 (3) ◽

pp. 692

Author(s):

Jingcheng Chen ◽

Yining Sun ◽

Shaoming Sun

Keyword(s):

Feature Selection ◽

Activity Recognition ◽

Human Activity ◽

Joint Angle ◽

Recognition Performance ◽

Wearable Sensors ◽

Estimation Algorithm ◽

Recognition System ◽

Human Activity Recognition ◽

Time Frequency

Human activity recognition (HAR) is essential in many health-related fields. A variety of technologies based on different sensors have been developed for HAR. Among them, fusion from heterogeneous wearable sensors has been developed as it is portable, non-interventional and accurate for HAR. To be applied in real-time use with limited resources, the activity recognition system must be compact and reliable. This requirement can be achieved by feature selection (FS). By eliminating irrelevant and redundant features, the system burden is reduced with good classification performance (CP). This manuscript proposes a two-stage genetic algorithm-based feature selection algorithm with a fixed activation number (GFSFAN), which is implemented on the datasets with a variety of time, frequency and time-frequency domain features extracted from the collected raw time series of nine activities of daily living (ADL). Six classifiers are used to evaluate the effects of selected feature subsets from different FS algorithms on HAR performance. The results indicate that GFSFAN can achieve good CP with a small size. A sensor-to-segment coordinate calibration algorithm and lower-limb joint angle estimation algorithm are introduced. Experiments on the effect of the calibration and the introduction of joint angle on HAR shows that both of them can improve the CP.

Download Full-text