Fusion of smartphone sensor data for classification of daily user activities

New mobile applications need to estimate user activities by using sensor data provided by smart wearable devices and deliver context-aware solutions to users living in smart environments. We propose a novel hybrid data fusion method to estimate three types of daily user activities (being in a meeting, walking, and driving with a motorized vehicle) using the accelerometer and gyroscope data acquired from a smart watch using a mobile phone. The approach is based on the matrix time series method for feature fusion, and the modified Better-than-the-Best Fusion (BB-Fus) method with a stochastic gradient descent algorithm for construction of optimal decision trees for classification. For the estimation of user activities, we adopted a statistical pattern recognition approach and used the k-Nearest Neighbor (kNN) and Support Vector Machine (SVM) classifiers. We acquired and used our own dataset of 354 min of data from 20 subjects for this study. We report a classification performance of 98.32 % for SVM and 97.42 % for kNN.

SVSL: A Human Activity Recognition Method Using Soft-Voting and Self-Learning

Many smart city and society applications such as smart health (elderly care, medical applications), smart surveillance, sports, and robotics require the recognition of user activities, an important class of problems known as human activity recognition (HAR). Several issues have hindered progress in HAR research, particularly due to the emergence of fog and edge computing, which brings many new opportunities (a low latency, dynamic and real-time decision making, etc.) but comes with its challenges. This paper focuses on addressing two important research gaps in HAR research: (i) improving the HAR prediction accuracy and (ii) managing the frequent changes in the environment and data related to user activities. To address this, we propose an HAR method based on Soft-Voting and Self-Learning (SVSL). SVSL uses two strategies. First, to enhance accuracy, it combines the capabilities of Deep Learning (DL), Generalized Linear Model (GLM), Random Forest (RF), and AdaBoost classifiers using soft-voting. Second, to classify the most challenging data instances, the SVSL method is equipped with a self-training mechanism that generates training data and retrains itself. We investigate the performance of our proposed SVSL method using two publicly available datasets on six human activities related to lying, sitting, and walking positions. The first dataset consists of 562 features and the second dataset consists of five features. The data are collected using the accelerometer and gyroscope smartphone sensors. The results show that the proposed method provides 6.26%, 1.75%, 1.51%, and 4.40% better prediction accuracy (average over the two datasets) compared to GLM, DL, RF, and AdaBoost, respectively. We also analyze and compare the class-wise performance of the SVSL methods with that of DL, GLM, RF, and AdaBoost.

Emojis as Contextual Indicants in Location-Based Social Media Posts

The presented study aims to investigate the relationship between the use of emojis in location-based social media and the location of the corresponding post in terms of perceived objects and conducted activities connected to this place. The basis for this is not a purely frequency-based assessment, but a specifically introduced measure called typicality. To evaluate the typicality measure and examine the assumption that emojis are contextual indicants, a dataset of worldwide geotagged posts from Instagram relating to sunset and sunrise events is used, converted to a privacy-aware version based on a Hyperloglog approach. Results suggest that emojis can often provide more nuanced information about user activities and the surrounding environment than is possible with hashtags. Thus, emojis may be suitable for identifying less obvious characteristics and the sense of a place. Emojis are already explored in research, but mainly for sentiment analysis, for semantic studies or as part of emoji prediction. In contrast, this work provides novel insights into the user’s spatial or activity context by applying the typicality measure and therefore considers emojis contextual indicants.

Signature Identification and User Activity Analysis on Whatsapp Web through Network Data

WhatsApp messenger is a popular instant messaging application that employs end-to-end encryption for communication. WhatsApp Web is the browser-based implementation of WhatsApp messenger. Users of WhatsApp communicate securely using SSL protocol. Encryption and use of common port for communication by multiple applications poses challenge in traffic classification for application identification. It is highly needed to analyze the network traffic for the purpose of QoS, Intrusion Detection and application specific traffic classification. In this paper, we have done traffic analysis on the network packets captured through data transfer in whatsapp web. In the result, we have explored the user activities such as message texting, contact sharing, voice message, location sharing, media transfer and status viewing. Packet level traffic analysis of user activities reveal patterns in the encrypted SSL communication. This pattern is identified across SSL packet lengths for WhatsApp media transfer and voice message communication. Other important features WhatsApp is the ability to view the status of the message being sent. We have identified the read and unread message status in these data packets by exposing signatures in the network layer. These signatures are identified with the help of the SSL lengths in the TLS header information of WhatsApp Web network traffic traces. Various other information on WhatsApp traffic presented in our study is relevant to the version of WhatsApp Web v0.3.2386.

Using Dirichlet Marked Hawkes Processes for Insider Threat Detection

Malicious insiders cause significant loss to organizations. Due to an extremely small number of malicious activities from insiders, insider threat is hard to detect. In this paper, we present a Dirichlet Marked Hawkes Process (DMHP) to detect malicious activities from insiders in real-time. DMHP combines the Dirichlet process and marked Hawkes processes to model the sequence of user activities. Dirichlet process is capable of detecting unbounded user modes (patterns) of infinite user activities, while for each detected user mode, one set of marked Hawkes processes is adopted to model user activities from time and activity type (e.g., WWW visit or send email) information so that different user modes are modeled by different sets of marked Hawkes processes. To achieve real-time malicious insider activity detection, the likelihood of the most recent activity calculated by DMHP is adopted as a score to measure the maliciousness of the activity. Since the majority of user activities are benign, those activities with low likelihoods are labeled as malicious activities. Experimental results on two datasets show the effectiveness of DMHP.

A egis +

The introduction of modern Smart Home Systems (SHSs) is redefining the way we perform everyday activities. Today, myriad SHS applications and the devices they control are widely available to users. Specifically, users can easily download and install the apps from vendor-specific app markets, or develop their own, to effectively implement their SHS solutions. However, despite their benefits, app-based SHSs unfold diverse security risks. Several attacks have already been reported to SHSs and current security solutions only consider smart home devices and apps individually to detect malicious actions, rather than the context of the SHS as a whole. Thus, the current security solutions applied to SHSs cannot capture user activities and sensor-device-user interactions in a holistic fashion. To address these limitations, in this article, we introduce A egis +, a novel context-aware platform-independent security framework to detect malicious behavior in an SHS. Specifically, A egis + observes the states of the connected smart home entities (sensors and devices) for different user activities and usage patterns in an SHS and builds a contextual model to differentiate between malicious and benign behavior. We evaluated the efficacy and performance of A egis + in multiple smart home settings (i.e., single bedroom, double bedroom, duplex) and platforms (i.e., Samsung SmartThings, Amazon Alexa) where real users perform day-to-day activities using real SHS devices. We also measured the performance of A egis + against five different malicious behaviors. Our detailed evaluation shows that A egis + can detect malicious behavior in SHS with high accuracy (over 95%) and secure the SHS regardless of the smart home layout and platforms, device configurations, installed apps, controller devices, and enforced user policies. Finally, A egis + yields minimum overhead to the SHS, ensuring effective deployability in real-life smart environments.