scholarly journals Exploiting Smart Meter Power Consumption Measurements for Human Activity Recognition (HAR) with a Motif-Detection-Based Non-Intrusive Load Monitoring (NILM) Approach

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 8036
Author(s):  
Sebastian Wilhelm ◽  
Jakob Kasbauer
Keyword(s):  
Power Consumption ◽  
Activity Recognition ◽  
Human Activity ◽  
Human Activity Recognition ◽  
Smart Meter ◽  
Activity Data ◽  
Motif Detection ◽  
Novel Approach ◽  
Load Monitoring ◽  
Individual Household

Numerous approaches exist for disaggregating power consumption data, referred to as non-intrusive load monitoring (NILM). Whereas NILM is primarily used for energy monitoring, we intend to disaggregate a household’s power consumption to detect human activity in the residence. Therefore, this paper presents a novel approach for NILM, which uses pattern recognition on the raw power waveform of the smart meter measurements to recognize individual household appliance actions. The presented NILM approach is capable of (near) real-time appliance action detection in a streaming setting, using edge computing. It is unique in our approach that we quantify the disaggregating uncertainty using continuous pattern correlation instead of binary device activity states. Further, we outline using the disaggregated appliance activity data for human activity recognition (HAR). To evaluate our approach, we use a dataset collected from actual households. We show that the developed NILM approach works, and the disaggregation quality depends on the pattern selection and the appliance type. In summary, we demonstrate that it is possible to detect human activity within the residence using a motif-detection-based NILM approach applied to smart meter measurements.

scholarly journals Similarity Embedding Networks for Robust Human Activity Recognition

2021 ◽  
Vol 15 (6) ◽  
pp. 1-17
Author(s):  
Chenglin Li ◽  
Carrie Lu Tong ◽  
Di Niu ◽  
Bei Jiang ◽  
Xiao Zuo ◽  
...  
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Real Space ◽  
Human Activity Recognition ◽  
Sensor Data ◽  
Activity Data ◽  
Extensive Evaluation ◽  
Sensor Signals ◽  
Public Datasets

Deep learning models for human activity recognition (HAR) based on sensor data have been heavily studied recently. However, the generalization ability of deep models on complex real-world HAR data is limited by the availability of high-quality labeled activity data, which are hard to obtain. In this article, we design a similarity embedding neural network that maps input sensor signals onto real vectors through carefully designed convolutional and Long Short-Term Memory (LSTM) layers. The embedding network is trained with a pairwise similarity loss, encouraging the clustering of samples from the same class in the embedded real space, and can be effectively trained on a small dataset and even on a noisy dataset with mislabeled samples. Based on the learned embeddings, we further propose both nonparametric and parametric approaches for activity recognition. Extensive evaluation based on two public datasets has shown that the proposed similarity embedding network significantly outperforms state-of-the-art deep models on HAR classification tasks, is robust to mislabeled samples in the training set, and can also be used to effectively denoise a noisy dataset.

scholarly journals On-Device Deep Learning Inference for Efficient Activity Data Collection

Sensors ◽  
2019 ◽  
Vol 19 (15) ◽  
pp. 3434 ◽  
Author(s):  
Nattaya Mairittha ◽  
Tittaya Mairittha ◽  
Sozo Inoue
Keyword(s):  
Deep Learning ◽  
Data Collection ◽  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Human Activity Recognition ◽  
Classification Model ◽  
Activity Data ◽  
Ground Truth Data ◽  
Recognition Systems

Labeling activity data is a central part of the design and evaluation of human activity recognition systems. The performance of the systems greatly depends on the quantity and “quality” of annotations; therefore, it is inevitable to rely on users and to keep them motivated to provide activity labels. While mobile and embedded devices are increasingly using deep learning models to infer user context, we propose to exploit on-device deep learning inference using a long short-term memory (LSTM)-based method to alleviate the labeling effort and ground truth data collection in activity recognition systems using smartphone sensors. The novel idea behind this is that estimated activities are used as feedback for motivating users to collect accurate activity labels. To enable us to perform evaluations, we conduct the experiments with two conditional methods. We compare the proposed method showing estimated activities using on-device deep learning inference with the traditional method showing sentences without estimated activities through smartphone notifications. By evaluating with the dataset gathered, the results show our proposed method has improvements in both data quality (i.e., the performance of a classification model) and data quantity (i.e., the number of data collected) that reflect our method could improve activity data collection, which can enhance human activity recognition systems. We discuss the results, limitations, challenges, and implications for on-device deep learning inference that support activity data collection. Also, we publish the preliminary dataset collected to the research community for activity recognition.

scholarly journals Combining Supervised and Unsupervised Learning Algorithms for Human Activity Recognition

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6309
Author(s):  
Elena-Alexandra Budisteanu ◽  
Irina Georgiana Mocanu
Keyword(s):  
Unsupervised Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Data Augmentation ◽  
Learning Algorithms ◽  
Human Activity Recognition ◽  
Convolutional Networks ◽  
Supervised And Unsupervised Learning ◽  
Novel Approach ◽  
Unsupervised Deep Learning

Human activity recognition is an extensively researched topic in the last decade. Recent methods employ supervised and unsupervised deep learning techniques in which spatial and temporal dependency is modeled. This paper proposes a novel approach for human activity recognition using skeleton data. The method combines supervised and unsupervised learning algorithms in order to provide qualitative results and performance in real time. The proposed method involves a two-stage framework: the first stage applies an unsupervised clustering technique to group up activities based on their similarity, while the second stage classifies data assigned to each group using graph convolutional networks. Different clustering techniques and data augmentation strategies are explored for improving the training process. The results were compared against the state of the art methods and the proposed model achieved 90.22% Top-1 accuracy performance for NTU-RGB+D dataset (the performance was increased by approximately 9% compared with the baseline graph convolutional method). Moreover, inference time and total number of parameters stay within the same magnitude order. Extending the initial set of activities with additional classes is fast and robust, since there is no required retraining of the entire architecture but only to retrain the cluster to which the activity is assigned.

scholarly journals Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1716 ◽  
Author(s):  
Seungeun Chung ◽  
Jiyoun Lim ◽  
Kyoung Ju Noh ◽  
Gague Kim ◽  
Hyuntae Jeong
Keyword(s):  
Deep Learning ◽  
Data Acquisition ◽  
Sensor Fusion ◽  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Sampling Rate ◽  
Human Activity Recognition ◽  
Sensor Data ◽  
Activity Data

In this paper, we perform a systematic study about the on-body sensor positioning and data acquisition details for Human Activity Recognition (HAR) systems. We build a testbed that consists of eight body-worn Inertial Measurement Units (IMU) sensors and an Android mobile device for activity data collection. We develop a Long Short-Term Memory (LSTM) network framework to support training of a deep learning model on human activity data, which is acquired in both real-world and controlled environments. From the experiment results, we identify that activity data with sampling rate as low as 10 Hz from four sensors at both sides of wrists, right ankle, and waist is sufficient in recognizing Activities of Daily Living (ADLs) including eating and driving activity. We adopt a two-level ensemble model to combine class-probabilities of multiple sensor modalities, and demonstrate that a classifier-level sensor fusion technique can improve the classification performance. By analyzing the accuracy of each sensor on different types of activity, we elaborate custom weights for multimodal sensor fusion that reflect the characteristic of individual activities.

scholarly journals A Fast and Robust Deep Convolutional Neural Networks for Complex Human Activity Recognition Using Smartphone

Sensors ◽  
2019 ◽  
Vol 19 (17) ◽  
pp. 3731 ◽  
Author(s):  
Wen Qi ◽  
Hang Su ◽  
Chenguang Yang ◽  
Giancarlo Ferrigno ◽  
Elena De Momi ◽  
...  
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Human Activity Recognition ◽  
Streaming Data ◽  
Computational Method ◽  
Measurement Unit ◽  
Deep Convolutional Neural Networks ◽  
Activity Data ◽  
Intelligent Sensors ◽  
Signal Processing Algorithms

As a significant role in healthcare and sports applications, human activity recognition (HAR) techniques are capable of monitoring humans’ daily behavior. It has spurred the demand for intelligent sensors and has been giving rise to the explosive growth of wearable and mobile devices. They provide the most availability of human activity data (big data). Powerful algorithms are required to analyze these heterogeneous and high-dimension streaming data efficiently. This paper proposes a novel fast and robust deep convolutional neural network structure (FR-DCNN) for human activity recognition (HAR) using a smartphone. It enhances the effectiveness and extends the information of the collected raw data from the inertial measurement unit (IMU) sensors by integrating a series of signal processing algorithms and a signal selection module. It enables a fast computational method for building the DCNN classifier by adding a data compression module. Experimental results on the sampled 12 complex activities dataset show that the proposed FR-DCNN model is the best method for fast computation and high accuracy recognition. The FR-DCNN model only needs 0.0029 s to predict activity in an online way with 95.27% accuracy. Meanwhile, it only takes 88 s (average) to establish the DCNN classifier on the compressed dataset with less precision loss 94.18%.

scholarly journals A Comparative Study of Feature Selection Approaches for Human Activity Recognition Using Multimodal Sensory Data

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2368
Author(s):  
Fatima Amjad ◽  
Muhammad Hassan Khan ◽  
Muhammad Adeel Nisar ◽  
Muhammad Shahid Farid ◽  
Marcin Grzegorzek
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Human Body ◽  
Human Activities ◽  
Wearable Sensors ◽  
Human Activity Recognition ◽  
Motion Sensors ◽  
Activity Data ◽  
Sensory Data ◽  
Atomic Actions

Human activity recognition (HAR) aims to recognize the actions of the human body through a series of observations and environmental conditions. The analysis of human activities has drawn the attention of the research community in the last two decades due to its widespread applications, diverse nature of activities, and recording infrastructure. Lately, one of the most challenging applications in this framework is to recognize the human body actions using unobtrusive wearable motion sensors. Since the human activities of daily life (e.g., cooking, eating) comprises several repetitive and circumstantial short sequences of actions (e.g., moving arm), it is quite difficult to directly use the sensory data for recognition because the multiple sequences of the same activity data may have large diversity. However, a similarity can be observed in the temporal occurrence of the atomic actions. Therefore, this paper presents a two-level hierarchical method to recognize human activities using a set of wearable sensors. In the first step, the atomic activities are detected from the original sensory data, and their recognition scores are obtained. Secondly, the composite activities are recognized using the scores of atomic actions. We propose two different methods of feature extraction from atomic scores to recognize the composite activities, and they include handcrafted features and the features obtained using the subspace pooling technique. The proposed method is evaluated on the large publicly available CogAge dataset, which contains the instances of both atomic and composite activities. The data is recorded using three unobtrusive wearable devices: smartphone, smartwatch, and smart glasses. We also investigated the performance evaluation of different classification algorithms to recognize the composite activities. The proposed method achieved 79% and 62.8% average recognition accuracies using the handcrafted features and the features obtained using subspace pooling technique, respectively. The recognition results of the proposed technique and their comparison with the existing state-of-the-art techniques confirm its effectiveness.

scholarly journals Human Activity Recognition Based on Residual Network and BiLSTM

Sensors ◽  
2022 ◽  
Vol 22 (2) ◽  
pp. 635
Author(s):  
Yong Li ◽  
Luping Wang
Keyword(s):  
Deep Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Inertial Sensors ◽  
Human Activity Recognition ◽  
Activity Data ◽  
Multidimensional Signals ◽  
Proposed Model ◽  
Public Datasets ◽  
Residual Block

Due to the wide application of human activity recognition (HAR) in sports and health, a large number of HAR models based on deep learning have been proposed. However, many existing models ignore the effective extraction of spatial and temporal features of human activity data. This paper proposes a deep learning model based on residual block and bi-directional LSTM (BiLSTM). The model first extracts spatial features of multidimensional signals of MEMS inertial sensors automatically using the residual block, and then obtains the forward and backward dependencies of feature sequence using BiLSTM. Finally, the obtained features are fed into the Softmax layer to complete the human activity recognition. The optimal parameters of the model are obtained by experiments. A homemade dataset containing six common human activities of sitting, standing, walking, running, going upstairs and going downstairs is developed. The proposed model is evaluated on our dataset and two public datasets, WISDM and PAMAP2. The experimental results show that the proposed model achieves the accuracy of 96.95%, 97.32% and 97.15% on our dataset, WISDM and PAMAP2, respectively. Compared with some existing models, the proposed model has better performance and fewer parameters.

scholarly journals Virtual Sensors for Optimal Integration of Human Activity Data

Sensors ◽  
2019 ◽  
Vol 19 (9) ◽  
pp. 2017 ◽  
Author(s):  
Antonio A. Aguileta ◽  
Ramon F. Brena ◽  
Oscar Mayora ◽  
Erik Molino-Minero-Re ◽  
Luis A. Trejo
Keyword(s):  
Machine Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Human Activity Recognition ◽  
Combination Method ◽  
Learning Methods ◽  
Activity Data ◽  
Virtual Sensors ◽  
Machine Learning Methods ◽  
Source Of Information

Sensors are becoming more and more ubiquitous as their price and availability continue to improve, and as they are the source of information for many important tasks. However, the use of sensors has to deal with noise and failures. The lack of reliability in the sensors has led to many forms of redundancy, but simple solutions are not always the best, and the precise way in which several sensors are combined has a big impact on the overall result. In this paper, we discuss how to deal with the combination of information coming from different sensors, acting thus as “virtual sensors”, in the context of human activity recognition, in a systematic way, aiming for optimality. To achieve this goal, we construct meta-datasets containing the “signatures” of individual datasets, and apply machine-learning methods in order to distinguish when each possible combination method could be actually the best. We present specific results based on experimentation, supporting our claims of optimality.

scholarly journals Exploration of Human Activity Recognition Using a Single Sensor for Stroke Survivors and Able-Bodied People

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 799
Author(s):  
Long Meng ◽  
Anjing Zhang ◽  
Chen Chen ◽  
Xingwei Wang ◽  
Xinyu Jiang ◽  
...  
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Standard Error ◽  
Human Activity Recognition ◽  
Support Vector ◽  
Stroke Survivors ◽  
Extensor Carpi Ulnaris ◽  
Novel Approach ◽  
Sensor Position ◽  
Single Sensor

Commonly used sensors like accelerometers, gyroscopes, surface electromyography sensors, etc., which provide a convenient and practical solution for human activity recognition (HAR), have gained extensive attention. However, which kind of sensor can provide adequate information in achieving a satisfactory performance, or whether the position of a single sensor would play a significant effect on the performance in HAR are sparsely studied. In this paper, a comparative study to fully investigate the performance of the aforementioned sensors for classifying four activities (walking, tooth brushing, face washing, drinking) is explored. Sensors are spatially distributed over the human body, and subjects are categorized into three groups (able-bodied people, stroke survivors, and the union of both). Performances of using accelerometer, gyroscope, sEMG, and their combination in each group are evaluated by adopting the Support Vector Machine classifier with the Leave-One-Subject-Out Cross-Validation technique, and the optimal sensor position for each kind of sensor is presented based on the accuracy. Experimental results show that using the accelerometer could obtain the best performance in each group. The highest accuracy of HAR involving stroke survivors was 95.84 ± 1.75% (mean ± standard error), achieved by the accelerometer attached to the extensor carpi ulnaris. Furthermore, taking the practical application of HAR into consideration, a novel approach to distinguish various activities of stroke survivors based on a pre-trained HAR model built on healthy subjects is proposed, the highest accuracy of which is 77.89 ± 4.81% (mean ± standard error) with the accelerometer attached to the extensor carpi ulnaris.

Sign in / Sign up

Export Citation Format

Share Document