scholarly journals Fusion Method to Estimate Heart Rate from Facial Videos Based on RPPG and RBCG

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6764
Author(s):  
Hyunwoo Lee ◽  
Ayoung Cho ◽  
Mincheol Whang
Keyword(s):  
Remote Sensing ◽  
Heart Rate ◽  
Principal Component ◽  
Component Analysis ◽  
Motion Artifacts ◽  
Color Variation ◽  
Fusion Method ◽  
Ensemble Averaging ◽  
Skin Contact ◽  
Main Challenge

Remote sensing of vital signs has been developed to improve the measurement environment by using a camera without a skin-contact sensor. The camera-based method is based on two concepts, namely color and motion. The color-based method, remote photoplethysmography (RPPG), measures the color variation of the face generated by reflectance of blood, whereas the motion-based method, remote ballistocardiography (RBCG), measures the subtle motion of the head generated by heartbeat. The main challenge of remote sensing is overcoming the noise of illumination variance and motion artifacts. The studies on remote sensing have focused on the blind source separation (BSS) method for RGB colors or motions of multiple facial points to overcome the noise. However, they have still been limited in their real-world applications. This study hypothesized that BSS-based combining of colors and the motions can improve the accuracy and feasibility of remote sensing in daily life. Thus, this study proposed a fusion method to estimate heart rate based on RPPG and RBCG by the BSS methods such as ensemble averaging (EA), principal component analysis (PCA), and independent component analysis (ICA). The proposed method was verified by comparing it with previous RPPG and RBCG from three datasets according to illumination variance and motion artifacts. The three main contributions of this study are as follows: (1) the proposed method based on RPPG and RBCG improved the remote sensing with the benefits of each measurement; (2) the proposed method was demonstrated by comparing it with previous methods; and (3) the proposed method was tested in various measurement conditions for more practical applications.

scholarly journals Vision-Based Measurement of Heart Rate from Ballistocardiographic Head Movements Using Unsupervised Clustering

Sensors ◽  
2019 ◽  
Vol 19 (15) ◽  
pp. 3263 ◽  
Author(s):  
Lee ◽  
Cho ◽  
Lee ◽  
Whang
Keyword(s):  
Heart Rate ◽  
Facial Expressions ◽  
Principal Component ◽  
Peak Detection ◽  
Motion Artifacts ◽  
Unsupervised Clustering ◽  
Head Movements ◽  
Rate Estimation ◽  
Skin Contact ◽  
Heart Rate Estimation

Heart rate has been measured comfortably using a camera without the skin-contact by the development of vision-based measurement. Despite the potential of the vision-based measurement, it has still presented limited ability due to the noise of illumination variance and motion artifacts. Remote ballistocardiography (BCG) was used to estimate heart rate from the ballistocardiographic head movements generated by the flow of blood through the carotid arteries. It was robust to illumination variance but still limited in the motion artifacts such as facial expressions and voluntary head motions. Recent studies on remote BCG focus on the improvement of signal extraction by minimizing the motion artifacts. They simply estimated the heart rate from the cardiac signal using peak detection and fast fourier transform (FFT). However, the heart rate estimation based on peak detection and FFT depend on the robust signal estimation. Thus, if the cardiac signal is contaminated with some noise, the heart rate cannot be estimated accurately. This study aimed to develop a novel method to improve heart rate estimation from ballistocardiographic head movements using the unsupervised clustering. First, the ballistocardiographic head movements were measured from facial video by detecting facial points using the good-feature-to-track (GFTT) algorithm and by tracking using the Kanade–Lucas–Tomasi (KLT) tracker. Second, the cardiac signal was extracted from the ballistocardiographic head movements by bandpass filter and principal component analysis (PCA). The relative power density (RPD) was extracted from its power spectrum between 0.75 Hz and 2.5 Hz. Third, the unsupervised clustering was performed to construct a model to estimate the heart rate from the RPD using the dataset consisting of the RPD and the heart rate measured from electrocardiogram (ECG). Finally, the heart rate was estimated from the RPD using the model. The proposed method was verified by comparing it with previous methods using the peak detection and the FFT. As a result, the proposed method estimated a more accurate heart rate than previous methods in three experiments by levels of the motion artifacts consisting of facial expressions and voluntary head motions. The four main contributions are as follows: (1) the unsupervised clustering improved the heart rate estimation by overcoming the motion artifacts (i.e., facial expressions and voluntary head motions); (2) the proposed method was verified by comparing with the previous methods using the peak detection and the FFT; (3) the proposed method can be combined with existing vision-based measurement and can improve their performance; (4) the proposed method was tested by three experiments considering the realistic environment including the motion artifacts, thus, it increases the possibility of the non-contact measurement in daily life.

scholarly journals Identification of catchment functional units by time series of thermal remote sensing images

2014 ◽  
Vol 18 (12) ◽  
pp. 5345-5359 ◽  
Author(s):  
B. Müller ◽  
M. Bernhardt ◽  
K. Schulz
Keyword(s):  
Remote Sensing ◽  
Principal Component Analysis ◽  
Time Series ◽  
Land Surface ◽  
Temporal Dynamics ◽  
Principal Component ◽  
Component Analysis ◽  
Land Surface Models ◽  
Surface Models ◽  
Functional Units

Abstract. The identification of catchment functional behavior with regards to water and energy balance is an important step during the parameterization of land surface models. An approach based on time series of thermal infrared (TIR) data from remote sensing is developed and investigated to identify land surface functioning as is represented in the temporal dynamics of land surface temperature (LST). For the mesoscale Attert catchment in midwestern Luxembourg, a time series of 28 TIR images from ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) was extracted and analyzed, applying a novel process chain. First, the application of mathematical–statistical pattern analysis techniques demonstrated a strong degree of pattern persistency in the data. Dominant LST patterns over a period of 12 years were then extracted by a principal component analysis. Component values of the two most dominant components could be related for each land surface pixel to land use data and geology, respectively. The application of a data condensation technique ("binary words") extracting distinct differences in the LST dynamics allowed the separation into landscape units that show similar behavior under radiation-driven conditions. It is further outlined that both information component values from principal component analysis (PCA), as well as the functional units from the binary words classification, will highly improve the conceptualization and parameterization of land surface models and the planning of observational networks within a catchment.

Sign in / Sign up

Export Citation Format

Share Document