Deep Learning-based Face Super-resolution: A Survey

Face super-resolution (FSR), also known as face hallucination, which is aimed at enhancing the resolution of low-resolution (LR) face images to generate high-resolution face images, is a domain-specific image super-resolution problem. Recently, FSR has received considerable attention and witnessed dazzling advances with the development of deep learning techniques. To date, few summaries of the studies on the deep learning-based FSR are available. In this survey, we present a comprehensive review of deep learning-based FSR methods in a systematic manner. First, we summarize the problem formulation of FSR and introduce popular assessment metrics and loss functions. Second, we elaborate on the facial characteristics and popular datasets used in FSR. Third, we roughly categorize existing methods according to the utilization of facial characteristics. In each category, we start with a general description of design principles, present an overview of representative approaches, and then discuss the pros and cons among them. Fourth, we evaluate the performance of some state-of-the-art methods. Fifth, joint FSR and other tasks, and FSR-related applications are roughly introduced. Finally, we envision the prospects of further technological advancement in this field.

EXPRESS: Distinct abilities associated with matching same identity faces vs. discriminating different faces: Evidence from individual differences in prosopagnosics and controls

Previous face matching studies provide evidence that matching same identity faces (match trials) and discriminating different face identities (non-match trials) rely on distinct processes. For example, instructional studies geared towards improving face matching in applied settings have often found selective improvements in match or non-match trials only. Additionally, a small study found that developmental prosopagnosics (DPs) have specific deficits in making match but not non-match judgments. In the current study, we sought to replicate this finding in DPs and examine how individual differences across DPs and controls in match vs. non-match performance relate to featural vs. holistic processing abilities. 43 DPs and 27 controls matched face images shown from similar front views or with varied lighting or viewpoint. Participants also performed tasks measuring featural (eyes/mouth) and holistic processing (part-whole task). We found that DPs showed worse overall matching performance than controls and that their relative match vs. non-match deficit depended on image variation condition, indicating that DPs do not consistently show match- or non-match-specific deficits. When examining the association between holistic and featural processing abilities and match vs. non-match trials in the entire group of DPs and controls, we found a very clear dissociation: Match trials significantly correlated with eye processing ability (r=.48) but not holistic processing (r=.11), whereas non-match trials significantly correlated with holistic processing (r=.32) but not eye processing (r=.03). This suggests that matching same identity faces relies more on eye processing while discriminating different faces relies more on holistic processing.

Fluorescein Leakage and Optical Coherence Tomography Angiography Features of Microaneurysms in Diabetic Retinopathy

Background and Objective. To correlate optical coherence tomography angiography (OCTA) characteristics of diabetic microaneurysms (MAs) with leakage status on fluorescein angiography (FA). Patients and Methods. 167 MAs from 39 diabetic eyes were analyzed using OCTA and FA simultaneously. The characteristics of MAs on OCTA en face, OCT en face, and OCT B-scan with flow overlay were evaluated and correlated with fluorescein leakage status. Results. Thirty-six, fifty-two, and seventy-nine MAs showed no, mild, and severe leakage on FA, respectively. Most MAs (61.7%) were centered in the inner nuclear layer. Cystoid spaces were observed adjacent to 60 (35.9%) MAs. MAs with severe leakage had a statistically higher flow proportion compared to MAs with no or mild leakage (both P < 0.001 ). Only 112 MAs (67.1%) were visualized in the OCTA en face images, while 165 MAs (98.8%) could be visualized in the OCT en face images. The location of MAs did not associate significantly with FA leakage status. The presence of nearby cystoid spaces and higher flow proportion by OCT B-scan with flow overlay correlated significantly with FA leakage status. Conclusion. The flow proportion of MAs observed on OCT B-scans with flow overlay might be a potential biomarker to identify leaking MAs. A combination of OCT B-scan, OCT en face, and OCTA en face images increased the detection rate of diabetic MAs in a noninvasive way.

Facial Privacy Preservation using FGSM and Universal Perturbation attacks

<p>Recent research has established the possibility of deducing soft-biometric attributes such as age, gender and race from an individual’s face image with high accuracy. Many techniques have been proposed to ensure user privacy, such as visible distortions to the images, manipulation of the original image with new face attributes, face swapping etc. Though these techniques achieve the goal of user privacy by fooling face recognition models, they don’t help the user when they want to upload original images without visible distortions or manipulation. The objective of this work is to implement techniques to ensure the privacy of user’s sensitive or personal data in face images by creating minimum pixel level distortions using white-box and black-box perturbation algorithms to fool AI models while maintaining the integrity of the image, so as to appear the same to a human eye.</p><div><br></div>

Facial Privacy Preservation using FGSM and Universal Perturbation attacks

<p>Recent research has established the possibility of deducing soft-biometric attributes such as age, gender and race from an individual’s face image with high accuracy. Many techniques have been proposed to ensure user privacy, such as visible distortions to the images, manipulation of the original image with new face attributes, face swapping etc. Though these techniques achieve the goal of user privacy by fooling face recognition models, they don’t help the user when they want to upload original images without visible distortions or manipulation. The objective of this work is to implement techniques to ensure the privacy of user’s sensitive or personal data in face images by creating minimum pixel level distortions using white-box and black-box perturbation algorithms to fool AI models while maintaining the integrity of the image, so as to appear the same to a human eye.</p><div><br></div>

Hyperrealistic neural decoding for reconstructing faces from fMRI activations via the GAN latent space

Neural decoding can be conceptualized as the problem of mapping brain responses back to sensory stimuli via a feature space. We introduce (i) a novel experimental paradigm that uses well-controlled yet highly naturalistic stimuli with a priori known feature representations and (ii) an implementation thereof for HYPerrealistic reconstruction of PERception (HYPER) of faces from brain recordings. To this end, we embrace the use of generative adversarial networks (GANs) at the earliest step of our neural decoding pipeline by acquiring fMRI data as participants perceive face images synthesized by the generator network of a GAN. We show that the latent vectors used for generation effectively capture the same defining stimulus properties as the fMRI measurements. As such, these latents (conditioned on the GAN) are used as the in-between feature representations underlying the perceived images that can be predicted in neural decoding for (re-)generation of the originally perceived stimuli, leading to the most accurate reconstructions of perception to date.

Research on Fast Face Retrieval Optimization Algorithm Based on Fuzzy Clustering

Aiming at the problem of long retrieval time for massive face image databases under a given threshold, a fast retrieval algorithm for massive face images based on fuzzy clustering is proposed. The algorithm builds a deep convolutional neural network model. The model can be used to extract features from face photos to obtain a high-dimensional vector to represent the high-level semantic features of face photos. On this basis, the fuzzy clustering algorithm is used to perform fuzzy clustering on the feature vectors of the face database to construct a retrieval pedigree map. When the threshold is passed in for database retrieval of the target face photos, the pedigree map can be quickly retrieved. Experiments on the LFW face dataset and self-collected face dataset show that the model is better than the commonly used K-means model in face recognition accuracy, clustering effect, and retrieval speed and has certain commercial value.

Optical coherence tomography angiography for the detection of macular neovascularization—comparison of en face versus cross-sectional view

Purpose To evaluate sensitivity and specificity of swept source-optical coherence tomography angiography (SS-OCTA) en face images versus cross-sectional OCTA versus a combination of both for the detection of macular neovascularization (MNV). Design Prospective cohort study. Participants Consecutive patients with various chorioretinal diseases and subretinal hyperreflective material (SHRM) and/or pigment epithelial detachment (PED) on OCT possibly corresponding to MNV in at least one eye. Methods 102 eyes of 63 patients with fluorescein angiography (FA), OCT and SS-OCTA performed on the same day were included. FA images, the outer retina to choriocapillaris (ORCC) OCTA en face slab, a manually modified en face slab (‘custom slab’), cross-sectional OCTA and a combination of OCTA en face and cross-section were evaluated for presence of MNV. Main outcome measures Sensitivity and specificity for MNV detection, as well as the concordance was calculated using FA as the reference. Results OCTA en face imaging alone yielded a sensitivity of 46.3% (automated)/78.1% (custom) and specificity of 93.4% (automated)/88.5% (custom) for MNV detection. Cross-sectional OCTA (combination with en face) resulted in a sensitivity of 85.4% (82.9%) and specificity of 82.0% (85.3%). Concordance to FA was moderate for automated en face OCTA (κ = 0.43), and substantial for custom en face OCTA (κ = 0.67), cross-sectional OCTA (κ = 0.66) and the combination (κ = 0.68). Conclusion Segmentation errors result in decreased sensitivity for MNV detection on automatically generated OCTA en face images. Cross-sectional OCTA allows detection of MNV without manual modification of segmentation lines and should be used for evaluation of MNV on OCTA.

Smart solution for reducing COVID-19 risk using internet of things

The article exposes a smart device designed for mitigating the coronavirus disease (COVID-19) risk using the internet of things. A portable smart alerting device is designed for ensuring safety in public places which can alert people when the guidelines given by the government were not followed and alert health authorities when any abnormalities found. By doing so, the spread of this fatal disease can be stopped. The modules of the proposed system include the face mask detection module, social distance alerting module, crowd detection and analysis module, health screening module and health assessment module. The proposed system can be placed in any public entrances to monitor people without human intervention. Firstly, the human face images are captured for face mask check, then the crowd analysis of the particular entrance where the person is entering is performed, thereafter health screening of the person is done and the values were imported to the health assessment module to check for any abnormalities. Finally, after all the conditions were met the door is opened automatically. The smart device can be installed and effectively used in many scenarios such as malls, stores, crowded places and campuses to avoid the risk of spread of the coronavirus.