PulseEdit: Editing Physiological Signal in Facial Videos for Privacy Protection

Recent studies have shown that physiological signals can be remotely captured from human faces using a portable color camera under ambient light. This technology, namely remote photoplethysmography (rPPG), can be used to collect users' physiological status who are sitting in front of a camera, which may raise physiological privacy issues. To avoid the privacy abuse of the rPPG technology, this paper develops PulseEdit, a novel and efficient algorithm that can edit the physiological signals in facial videos without affecting visual appearance to protect the user's physiological signal from disclosure. PulseEdit can either remove the trace of the physiological signal in a video or transform the video to contain a target physiological signal chosen by a user. Experimental results show that PulseEdit can effectively edit physiological signals in facial videos and prevent heart rate measurement based on rPPG. It is possible to utilize PulseEdit in adversarial scenarios against some rPPG-based visual security algorithms. We present analyses on the performance of PulseEdit against rPPG-based liveness detection and rPPG-based deepfake detection, and demonstrate its ability to circumvent these visual security algorithms.

Evaluation of Remote Photoplethysmography Measurement Conditions toward Telemedicine Applications

Camera-based remote photoplethysmography (rPPG) is a low-cost and casual non-contact heart rate measurement method suitable for telemedicine. Several factors affect the accuracy of measuring the heart rate and heart rate variability (HRV) using rPPG despite HRV being an important indicator for healthcare monitoring. This study aimed to investigate the appropriate setup for precise HRV measurements using rPPG while considering the effects of possible factors including illumination, direction of the light, frame rate of the camera, and body motion. In the lighting conditions experiment, the smallest mean absolute R–R interval (RRI) error was obtained when light greater than 500 lux was cast from the front (among the following conditions—illuminance: 100, 300, 500, and 700 lux; directions: front, top, and front and top). In addition, the RRI and HRV were measured with sufficient accuracy at frame rates above 30 fps. The accuracy of the HRV measurement was greatly reduced when the body motion was not constrained; thus, it is necessary to limit the body motion, especially the head motion, in an actual telemedicine situation. The results of this study can act as guidelines for setting up the shooting environment and camera settings for rPPG use in telemedicine.

Assessment of ROI Selection for Facial Video-Based rPPG

In general, facial image-based remote photoplethysmography (rPPG) methods use color-based and patch-based region-of-interest (ROI) selection methods to estimate the blood volume pulse (BVP) and beats per minute (BPM). Anatomically, the thickness of the skin is not uniform in all areas of the face, so the same diffuse reflection information cannot be obtained in each area. In recent years, various studies have presented experimental results for their ROIs but did not provide a valid rationale for the proposed regions. In this paper, to see the effect of skin thickness on the accuracy of the rPPG algorithm, we conducted an experiment on 39 anatomically divided facial regions. Experiments were performed with seven algorithms (CHROM, GREEN, ICA, PBV, POS, SSR, and LGI) using the UBFC-rPPG and LGI-PPGI datasets considering 29 selected regions and two adjusted regions out of 39 anatomically classified regions. We proposed a BVP similarity evaluation metric to find a region with high accuracy. We conducted additional experiments on the TOP-5 regions and BOT-5 regions and presented the validity of the proposed ROIs. The TOP-5 regions showed relatively high accuracy compared to the previous algorithm’s ROI, suggesting that the anatomical characteristics of the ROI should be considered when developing a facial image-based rPPG algorithm.

Photoplethysmography for demarcation of cutaneous squamous cell carcinoma

A video processing algorithm designed to identify cancer suspicious skin areas is presented here. It is based on video recordings of squamous cell carcinoma in the skin. Squamous cell carcinoma is a common malignancy, normally treated by surgical removal. The surgeon should always balance sufficient tissue removal against unnecessary mutilation, and therefore methods for distinction of cancer boundaries are wanted. Squamous cell carcinoma has angiogenesis and increased blood supply. Remote photoplethysmography is an evolving technique for analysis of signal variations in video recordings in order to extract vital signs such as pulsation. We hypothesize that the remote photoplethysmography signal inside the area of a squamous cell carcinoma is significantly different from the surrounding healthy skin. Based on high speed video recordings of 13 patients with squamous cell carcinoma, we have examined temporal signal differences in cancer areas versus healthy skin areas. A significant difference in temporal signal changes between cancer areas and healthy areas was found. Our video processing algorithm showed promising results encouraging further investigation to clarify how detailed distinctions can be made.

Mobile Remote Photoplethysmography: A Real-Time Realization Perspective

Remote photoplethysmography (rPPG) sensors have attracted a significant amount of attention as they enable the remote monitoring of instantaneous heart rates (HRs) and thus do not require any additional devices to be worn on fingers or wrists. In this study, we mounted rPPG sensors on a robot for active and autonomous instantaneous HR (R-AAIH) estimation. Subsequently, we proposed the algorithm providing accurate instantaneous HRs, which can be performed in real time with vision and robot manipulation algorithms. By simplifying the extraction of facial skin images using saturation (S) values in the HSV color space, and selecting pixels based on the most frequent S value on the face image, we achieved reliable HR assessment. The results of the proposed algorithm using the R-AAIH were evaluated by rigorous comparison with the results of existing algorithms on the UBFC-RPPG dataset (n = 42). Our algorithm exhibited an average absolute error (AAE) of 0.71 beats per minute (bpm). The developed algorithm is simple and the processing time is less than 1 s (275 ms for an 8-s window). The algorithm was further validated on our own dataset (BAMI-RPPG dataset [n = 14]) with an AAE of 0.82 bpm.