Motion and Illumination Resistant Facial Video based Heart Rate Estimation Method using Levenberg-Marquardt Algorithm Optimized Undercomplete Independent Component Analysis

<div>Heart Rate (HR) estimation is of utmost need due to its applicability in diverse fields. Conventional methods for HR estimation require skin contact and are not suitable for scenarios such as sensitive skin or prolonged unobtrusive HR monitoring. Therefore remote photoplethysmography (rPPG) methods have been an active area of research. These methods utilize the facial videos acquired using a camera followed by extracting the Blood Volume Pulse (BVP) signal for heart rate calculation. The existing rPPG methods either used a single color channel or weighted color differences, which has limitations dealing with motion and illumination artifacts. This study considers BVP extraction as an undercomplete problem and proposed a method U-LMA. First, a non-linear Cumulative Density Function (CDF) approximated by a hyperbolic tangent (tanh) was used to deal with the non-linearity associated with rigid and non-rigid motions and illumination variations. Then, the entropy of the proposed non-linear CDF was optimized using a customized LMA for BVP signal extraction, followed by maximum peak estimation for HR calculation. The performance of the proposed method was tested under three scenarios: constrained, motion, and illumination variations scenarios. High Pearson correlation coefficient values and smaller lower-upper statistical limits of bland-altman plots , justified the good performance of the U-LMA. Comparative analysis of U-LMA with undercomplete ICA with negentropy (U-neg) and other state-of-the-art methods demonstrated its best performance of U-LMA by achieving the lowest error and highest correlation values (0.01 significance level) . Additionally, higher accuracy satisfying the clinically accepted error differences also justified its clinical relevance.</div>

Motion and Illumination Resistant Facial Video based Heart Rate Estimation Method using Levenberg-Marquardt Algorithm Optimized Undercomplete Independent Component Analysis

<div>Heart Rate (HR) estimation is of utmost need due to its applicability in diverse fields. Conventional methods for HR estimation require skin contact and are not suitable for scenarios such as sensitive skin or prolonged unobtrusive HR monitoring. Therefore remote photoplethysmography (rPPG) methods have been an active area of research. These methods utilize the facial videos acquired using a camera followed by extracting the Blood Volume Pulse (BVP) signal for heart rate calculation. The existing rPPG methods either used a single color channel or weighted color differences, which has limitations dealing with motion and illumination artifacts. This study considers BVP extraction as an undercomplete problem and proposed a method U-LMA. First, a non-linear Cumulative Density Function (CDF) approximated by a hyperbolic tangent (tanh) was used to deal with the non-linearity associated with rigid and non-rigid motions and illumination variations. Then, the entropy of the proposed non-linear CDF was optimized using a customized LMA for BVP signal extraction, followed by maximum peak estimation for HR calculation. The performance of the proposed method was tested under three scenarios: constrained, motion, and illumination variations scenarios. High Pearson correlation coefficient values and smaller lower-upper statistical limits of bland-altman plots , justified the good performance of the U-LMA. Comparative analysis of U-LMA with undercomplete ICA with negentropy (U-neg) and other state-of-the-art methods demonstrated its best performance of U-LMA by achieving the lowest error and highest correlation values (0.01 significance level) . Additionally, higher accuracy satisfying the clinically accepted error differences also justified its clinical relevance.</div>

GRAPH NEURAL NETWORK BASED MULTI-FEATURE FUSION FOR BUILDING CHANGE DETECTION

Building Change Detection (BCD) via multi-temporal remote sensing images is essential for various applications such as urban monitoring, urban planning, and disaster assessment. However, most building change detection approaches only extract features from different kinds of remote sensing images for change index determination, which can not determine the insignificant changes of small buildings. Given co-registered multi-temporal remote sensing images, the illumination variations and misregistration errors always lead to inaccurate change detection results. This study investigates the applicability of multi-feature fusion from both directly extract 2D features from remote sensing images and 3D features extracted by the dense image matching (DIM) generated 3D point cloud for accurate building change index generation. This paper introduces a graph neural network (GNN) based end-to-end learning framework for building change detection. The proposed framework includes feature extraction, feature fusion, and change index prediction. It starts with a pre-trained VGG-16 network as a backend and uses U-net architecture with five layers for feature map extraction. The extracted 2D features and 3D features are utilized as input into GNN based feature fusion parts. In the GNN parts, we introduce a flexible context aggregation mechanism based on attention to address the illumination variations and misregistration errors, enabling the framework to reason about the image-based texture information and depth information introduced by DIM generated 3D point cloud jointly. After that, the GNN generated affinity matrix is utilized for change index determination through a Hungarian algorithm. The experiment conducted on a dataset that covered Setagaya-Ku, Tokyo area, shows that the proposed method generated change map achieved the precision of 0.762 and the F1-score of 0.68 at pixel-level. Compared to traditional image-based change detection methods, our approach learns prior over geometrical structure information from the real 3D world, which robust to the misregistration errors. Compared to CNN based methods, the proposed method learns to fuse 2D and 3D features together to represent more comprehensive information for building change index determination. The experimental comparison results demonstrated that the proposed approach outperforms the traditional methods and CNN based methods.

An Improved Algorithm Robust to Illumination Variations for Reconstructing Point Cloud Models from Images

Reconstructing 3D point cloud models from image sequences tends to be impacted by illumination variations and textureless cases in images, resulting in missing parts or uneven distribution of retrieved points. To improve the reconstructing completeness, this work proposes an enhanced similarity metric which is robust to illumination variations among images during the dense diffusions to push the seed-and-expand reconstructing scheme to a further extent. This metric integrates the zero-mean normalized cross-correlation coefficient of illumination and that of texture information which respectively weakens the influence of illumination variations and textureless cases. Incorporated with disparity gradient and confidence constraints, the candidate image features are diffused to their neighborhoods for dense 3D points recovering. We illustrate the two-phase results of multiple datasets and evaluate the robustness of proposed algorithm to illumination variations. Experiments show that ours recovers 10.0% more points, on average, than comparing methods in illumination varying scenarios and achieves better completeness with comparative accuracy.