Expanding the Horizons of Wireless Sensing

All interactions of objects, humans, and machines with the physical world are via contact forces. For instance, objects placed on a table exert their gravitational forces, and the contact interactions via our hands/feet are guided by the sense of contact force felt by our skin. Thus, the ability to sense the contact forces can allow us to measure all these ubiquitous interactions, enabling a myriad of applications. Furthermore, force sensors are a critical requirement for safer surgeries, which require measuring complex contact forces experienced as a surgical instrument interacts with the surrounding tissues during the surgical procedure. However, with currently available discrete point-force sensors, which require a battery to sense the forces and communicate the readings wirelessly, these ubiquitous sensing and surgical sensing applications are not practical. This motivates the development of new force sensors that can sense, and communicate wirelessly without consuming significant power to enable a battery-free design. In this magazine article, we present WiForce, a low-power wireless force sensor utilizing a joint sensing-communication paradigm. That is, instead of having separate sensing and communication blocks, WiForce directly transduces the force measurements onto variations in wireless signals reflecting WiForce from the sensor. This novel trans-duction mechanism also allows WiForce to generalize easily to a length continuum, where we can detect as well as localize forces acting on the continuum. We fabricate and test our sensor prototype in different scenarios, including testing beneath a tissue phantom, and obtain sub-N sensing and sub-mm localizing accuracies (0.34 N and 0.6 mm, respectively).

SonicFace

Accurate recognition of facial expressions and emotional gestures is promising to understand the audience's feedback and engagement on the entertainment content. Existing methods are primarily based on various cameras or wearable sensors, which either raise privacy concerns or demand extra devices. To this aim, we propose a novel ubiquitous sensing system based on the commodity microphone array --- SonicFace, which provides an accessible, unobtrusive, contact-free, and privacy-preserving solution to monitor the user's emotional expressions continuously without playing hearable sound. SonicFace utilizes a pair of speaker and microphone array to recognize various fine-grained facial expressions and emotional hand gestures by emitted ultrasound and received echoes. Based on a set of experimental evaluations, the accuracy of recognizing 6 common facial expressions and 4 emotional gestures can reach around 80%. Besides, the extensive system evaluations with distinct configurations and an extended real-life case study have demonstrated the robustness and generalizability of the proposed SonicFace system.

Privacy-Enhanced MQTT Protocol for Massive IoT

The growing expectations for ubiquitous sensing have led to the integration of countless embedded sensors, actuators, and RFIDs in our surroundings. Combined with rapid developments in high-speed wireless networks, these resource-constrained devices are paving the road for the Internet-of-Things paradigm, a computing model aiming to bring together millions of heterogeneous and pervasive elements. However, it is commonly accepted that the Privacy consideration remains one of its main challenges, a notion that does not only encompasses malicious individuals but can also be extended to honest-but-curious third-parties. In this paper, we study the design of a privacy-enhanced communication protocol for lightweight IoT devices. Applying the proposed approach to MQTT, a highly popular lightweight publish/subscribe communication protocol prevents no valuable information from being extracted from the messages flowing through the broker. In addition, it also prevents partners re-identification. Starting from a privacy-ideal, but unpractical, exact transposition of the Oblivious Transfer (OT) technology to MQTT, this paper follows an iterative process where each previous model’s drawbacks are appropriately mitigated all the while trying to preserve acceptable privacy levels. Our work provides resistance to statistical analysis attacks and dynamically supports new client participation. Additionally the whole proposal is based on the existence of a non-communicating 3rd party during pre-development. This particular contribution reaches a proof-of-concept stage through implementation, and achieves its goals thanks to OT’s indistinguishability property as well as hash-based topic obfuscations.

Ubi-SleepNet

Sleep is a fundamental physiological process that is essential for sustaining a healthy body and mind. The gold standard for clinical sleep monitoring is polysomnography(PSG), based on which sleep can be categorized into five stages, including wake/rapid eye movement sleep (REM sleep)/Non-REM sleep 1 (N1)/Non-REM sleep 2 (N2)/Non-REM sleep 3 (N3). However, PSG is expensive, burdensome and not suitable for daily use. For long-term sleep monitoring, ubiquitous sensing may be a solution. Most recently, cardiac and movement sensing has become popular in classifying three-stage sleep, since both modalities can be easily acquired from research-grade or consumer-grade devices (e.g., Apple Watch). However, how best to fuse the data for greatest accuracy remains an open question. In this work, we comprehensively studied deep learning (DL)-based advanced fusion techniques consisting of three fusion strategies alongside three fusion methods for three-stage sleep classification based on two publicly available datasets. Experimental results demonstrate important evidences that three-stage sleep can be reliably classified by fusing cardiac/movement sensing modalities, which may potentially become a practical tool to conduct large-scale sleep stage assessment studies or long-term self-tracking on sleep. To accelerate the progression of sleep research in the ubiquitous/wearable computing community, we made this project open source, and the code can be found at: https://github.com/bzhai/Ubi-SleepNet.

User Experiences with a Short Message Service-Based Illness Self-Management Intervention: A Mixed Methods Study (Preprint)

BACKGROUND Limited access to mental health care services due to provider shortages, geographic limitations, and cost has driven the area of mobile health (mHealth) care to address these access gaps. Reports from the Cohen Veterans Network and National Council for Behavioral Health show that in states where mental health care is more accessible, there is still 38% of people who are not receiving the care they need. MHealth strategies help to provide care to individuals experiencing these barriers at lower cost and greater convenience, making mHealth a great resource to bridge the gap. OBJECTIVE We present a mixed-methods study to evaluate user experiences with the mental mHealth service, Cope Notes. Specifically, we aimed to investigate the following research questions: 1. How do Cope Notes users perceive the service as it relates to stigma, impact of the intervention, and perceived usefulness? 2. How do Cope Notes users rate the Cope Notes service and messaging along various dimensions of acceptability? 3. What is the relationship between Cope Notes message ratings and user personality and coping strategies? 4. What are user perspectives of ubiquitous sensing technologies, including integration of ubiquitous sensing for the improvement in timeliness of the intervention and quality of tailored content? METHODS We performed qualitative interviews with Cope Notes users (n=14) who have used the service for at least 30 days to evaluate their experience and usefulness of the service. These interviews were coded by two raters, and interrater reliability was calculated with SPSS at 61.8%. Additionally, participants completed quantitative measures, including a user experiences survey, personality inventory (Big Five-10), and coping assessment (Brief COPE). RESULTS We derived seven main overarching themes from our qualitative interviews: Likes/Perceived Benefits, Dislikes/Limitations, Suggested Changes, Stigma/Help Seeking, Perceptions of Ubiquitous Sensing, Cultural Sensitivity, and Alternative mHealth Resources. Exploratory analyses between acceptability ratings of Cope Notes and personality factors from BF-10 yielded statistically significant positive relationships between seeing oneself as someone who is generally trusting and various acceptability items, the most significant being item 7 (“I fully understood the sentiment behind Cope Notes Messages”) with (rs(10) = 0.82, P = .001). We also found statistically significant relationships between the acceptability items and Brief COPE items, with the strongest positive correlation between participants strongly endorsing coping by accepting the reality that an event has happened and acceptability item 7 (rs(8) = 0.86, P = .001). CONCLUSIONS Our study found that Cope Notes subscribers appreciate the service for reframing and refocusing their mental wellness with statistically significant correlations between personality and the acceptability of the service. We found that some users prefer a more personalized experience with neutral to positive reactions to a potential companion app that continuously monitors user behavior via smartphone sensor readings to provide just-in-time interventions when users need it the most.

WiStress

Stress plays a critical role in our lives, impacting our productivity and our long-term physiological and psychological well-being. This has motivated the development of stress monitoring solutions to better understand stress, its impact on productivity and teamwork, and help users adapt their habits toward more sustainable stress levels. However, today's stress monitoring solutions remain obtrusive, requiring active user participation (e.g., self-reporting), interfering with people's daily activities, and often adding more burden to users looking to reduce their stress. In this paper, we introduce WiStress, the first system that can passively monitor a user's stress levels by relying on wireless signals. WiStress does not require users to actively provide input or to wear any devices on their bodies. It operates by transmitting ultra-low-power wireless signals and measuring their reflections off the user's body. WiStress introduces two key innovations. First, it presents the first machine learning network that can accurately and robustly extract heartbeat intervals (IBI's) from wireless reflections without constraints on a user's daily activities. Second, it introduces a stress classification framework that combines the extracted heartbeats with other wirelessly captured stress-related features in order to infer a subject's stress level. We built a prototype of WiStress and tested it on 22 different subjects across different environments in both stress-induced and free-living conditions. Our results demonstrate that WiStress has high accuracy (84%-95%) in inferring a person's stress level in a fully-automated way, paving the way for ubiquitous sensing systems that can monitor stress and provide feedback to improve productivity, health, and well-being.

IoT based Smart Helmet for Industrial Workers

Safety is very important in every workplace, but very often we hear about accidents in factories industries causing loss of life. The labours and workers working in any factory, industries, construction site or mine is vulnerable to accidents and therefore they should be with safety guards properly. In most of the accidents, number of deaths or severe injuries is maximized because the labours and worker are not wearing safety equipmentor wearing low grade safety equipment. Working environment hazards include radiation leakage, fall due to suffocation, poisoning gas leakage and gas explosion. Hence air quality and hazardous event detection is very important factorin industry. In order to achieve those safety measures, the proposed system provides wireless sensors network for monitoring real time situation of working environment from monitoring station. Keywords: Industries; Helmet; Cloud Computing; ThingSpeak Internet of Things; Sensors; Ubiquitous Sensing

The Use of Transfer Learning for Activity Recognition in Instances of Heterogeneous Sensing

Transfer learning is a growing field that can address the variability of activity recognition problems by reusing the knowledge from previous experiences to recognise activities from different conditions, resulting in the leveraging of resources such as training and labelling efforts. Although integrating ubiquitous sensing technology and transfer learning seem promising, there are some research opportunities that, if addressed, could accelerate the development of activity recognition. This paper presents TL-FmRADLs; a framework that converges the feature fusion strategy with a teacher/learner approach over the active learning technique to automatise the self-training process of the learner models. Evaluation TL-FmRADLs is conducted over InSync; an open access dataset introduced for the first time in this paper. Results show promising effects towards mitigating the insufficiency of labelled data available by enabling the learner model to outperform the teacher’s performance.

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now with the advent of wearable, mobile, and ubiquitous technologies that we can aim at sensing and recognizing emotions, continuously and in the wild. This Special Issue aims at bringing together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and recognition of human emotions.

Nonverbal Social Sensing: What Social Sensing Can and Cannot Do for the Study of Nonverbal Behavior From Video

The study of nonverbal behavior (NVB), and in particular kinesics (i.e., face and body motions), is typically seen as cost-intensive. However, the development of new technologies (e.g., ubiquitous sensing, computer vision, and algorithms) and approaches to study social behavior [i.e., social signal processing (SSP)] makes it possible to train algorithms to automatically code NVB, from action/motion units to inferences. Nonverbal social sensing refers to the use of these technologies and approaches for the study of kinesics based on video recordings. Nonverbal social sensing appears as an inspiring and encouraging approach to study NVB at reduced costs, making it a more attractive research field. However, does this promise hold? After presenting what nonverbal social sensing is and can do, we discussed the key challenges that researchers face when using nonverbal social sensing on video data. Although nonverbal social sensing is a promising tool, researchers need to be aware of the fact that algorithms might be as biased as humans when extracting NVB or that the automated NVB coding might remain context-dependent. We provided study examples to discuss these challenges and point to potential solutions.