scholarly journals NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting

2021 ◽  
Author(s):  
TIANCHENG CHEN
Keyword(s):  
State Of The Art ◽  
Light Transport ◽  
View Synthesis ◽  
Computer Algorithms ◽  
3D Space ◽  
Lighting Conditions ◽  
Camera View ◽  
The Face ◽  
Environmental Lighting ◽  
Synthetic Models

Human portraits exhibit various appearances when observed from different views under different lighting conditions. We can easily imagine how the face will look like in another setup, but computer algorithms still fail on this problem given limited observations. To this end, we present a system for portrait view synthesis and relighting: given multiple portraits, we use a neural network to predict the light-transport field in 3D space, and from the predicted Neural Light-transport Field (NeLF)produce a portrait from a new camera view under a new environmental lighting. Our system is trained on a large number of synthetic models, and can generalize to different synthetic and real portraits under various lighting conditions. Our method achieves simultaneous view synthesis and relighting given multi-view portraits as the input, and achieves state-of-the-art results.

scholarly journals Neural Light Transport for Relighting and View Synthesis

2021 ◽  
Vol 40 (1) ◽  
pp. 1-17
Author(s):  
Xiuming Zhang ◽  
Sean Fanello ◽  
Yun-Ta Tsai ◽  
Tiancheng Sun ◽  
Tianfan Xue ◽  
...  
Keyword(s):  
Neural Representation ◽  
Light Transport ◽  
View Synthesis ◽  
Unified Framework ◽  
Subsurface Scattering ◽  
Lighting Condition ◽  
Qualitative And Quantitative ◽  
Lighting Conditions ◽  
Complete Knowledge ◽  
Physically Based

The light transport (LT) of a scene describes how it appears under different lighting conditions from different viewing directions, and complete knowledge of a scene’s LT enables the synthesis of novel views under arbitrary lighting. In this article, we focus on image-based LT acquisition, primarily for human bodies within a light stage setup. We propose a semi-parametric approach for learning a neural representation of the LT that is embedded in a texture atlas of known but possibly rough geometry. We model all non-diffuse and global LT as residuals added to a physically based diffuse base rendering. In particular, we show how to fuse previously seen observations of illuminants and views to synthesize a new image of the same scene under a desired lighting condition from a chosen viewpoint. This strategy allows the network to learn complex material effects (such as subsurface scattering) and global illumination (such as diffuse interreflection), while guaranteeing the physical correctness of the diffuse LT (such as hard shadows). With this learned LT, one can relight the scene photorealistically with a directional light or an HDRI map, synthesize novel views with view-dependent effects, or do both simultaneously, all in a unified framework using a set of sparse observations. Qualitative and quantitative experiments demonstrate that our Neural Light Transport (NLT) outperforms state-of-the-art solutions for relighting and view synthesis, without requiring separate treatments for both problems that prior work requires. The code and data are available at http://nlt.csail.mit.edu.

Research on Improved Gamma Transform Face Image Preprocessing Fusion Algorithm under Complex Lighting Conditions

2020 ◽  
Vol 13 ◽  
Author(s):  
Xiaolin Tang ◽  
Xiaogang Wang ◽  
Jin Hou ◽  
Huafeng Wu ◽  
Ping He
Keyword(s):  
Edge Detection ◽  
Face Image ◽  
Light Sources ◽  
Fusion Algorithm ◽  
Edge Information ◽  
Image Preprocessing ◽  
Lighting Conditions ◽  
Weighted Fusion ◽  
The Face ◽  
Sobel Edge Detection

Introduction: Under complex illumination conditions such as poor light sources and light changes rapidly, there are two disadvantages of current gamma transform in preprocessing face image: one is that the parameters of transformation need to be set based on experience; the other is the details of the transformed image are not obvious enough. Objective: Improve the current gamma transform. Methods: This paper proposes a weighted fusion algorithm of adaptive gamma transform and edge feature extraction. First, this paper proposes an adaptive gamma transform algorithm for face image preprocessing, that is, the parameter of transformation generated by calculation according to the specific gray value of the input face image. Secondly, this paper uses Sobel edge detection operator to extract the edge information of the transformed image to get the edge detection image. Finally, this paper uses the adaptively transformed image and the edge detection image to obtain the final processing result through a weighted fusion algorithm. Results: The contrast of the face image after preprocessing is appropriate, and the details of the image are obvious. Conclusion: The method proposed in this paper can enhance the face image while retaining more face details, without human-computer interaction, and has lower computational complexity degree.

scholarly journals Bottom-up and Layerwise Domain Adaptation for Pedestrian Detection in Thermal Images

2021 ◽  
Vol 17 (1) ◽  
pp. 1-19
Author(s):  
My Kieu ◽  
Andrew D. Bagdanov ◽  
Marco Bertini
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Challenging Problem ◽  
Top Down ◽  
Bottom Up ◽  
Security Applications ◽  
Lighting Conditions ◽  
Initial Layers ◽  
Single Modality

Pedestrian detection is a canonical problem for safety and security applications, and it remains a challenging problem due to the highly variable lighting conditions in which pedestrians must be detected. This article investigates several domain adaptation approaches to adapt RGB-trained detectors to the thermal domain. Building on our earlier work on domain adaptation for privacy-preserving pedestrian detection, we conducted an extensive experimental evaluation comparing top-down and bottom-up domain adaptation and also propose two new bottom-up domain adaptation strategies. For top-down domain adaptation, we leverage a detector pre-trained on RGB imagery and efficiently adapt it to perform pedestrian detection in the thermal domain. Our bottom-up domain adaptation approaches include two steps: first, training an adapter segment corresponding to initial layers of the RGB-trained detector adapts to the new input distribution; then, we reconnect the adapter segment to the original RGB-trained detector for final adaptation with a top-down loss. To the best of our knowledge, our bottom-up domain adaptation approaches outperform the best-performing single-modality pedestrian detection results on KAIST and outperform the state of the art on FLIR.

scholarly journals The Influence of Light Exposure in Ambiance during Pregnancy in Maternal and Fetal Outcomes: An Experimental Study

2018 ◽  
Vol 41 (01) ◽  
pp. 024-030
Author(s):  
Vitor Sarri ◽  
Beatriz Ferrari ◽  
Larissa Magalhães ◽  
Paula Rodrigues ◽  
Almir Rezende ◽  
...  
Keyword(s):  
Light Exposure ◽  
Good Practice ◽  
Histological Evaluation ◽  
Reproductive Parameters ◽  
Anthropometric Measures ◽  
Dark Light ◽  
Mean Values ◽  
Lighting Conditions ◽  
Significant Difference ◽  
Environmental Lighting

Objective The aim of this study is to evaluate whether exposure to different environmental lighting conditions affects the reproductive parameters of pregnant mice and the development of their offspring. Methods Fifteen pregnant albino mice were divided into three groups: light/dark, light, and dark. The animals were euthanized on day 18 of pregnancy following the Brazilian Good Practice Guide for Euthanasia of Animals. Maternal and fetal specimens were measured and collected for histological evaluation. Analysis of variance (ANOVA) test was used for comparison of the groups considering p ≤ 0.05 to be statistically significant. Results There was no significant difference in the maternal variables between the three groups. Regarding fetal variables, significant differences were observed in the anthropometric measures between the groups exposed to different environmental lighting conditions, with the highest mean values in the light group. The histological evaluation showed the same structural pattern of the placenta in all groups, which was within the normal range. However, evaluation of the uterus revealed a discrete to moderate number of endometrial glands in the light/dark and light groups, which were poorly developed in most animals. In the fetuses, pulmonary analysis revealed morphological features consistent with the transition from the canalicular to the saccular phase in all groups. Conclusion Exposure to different environmental lighting conditions had no influence on the reproductive parameters of female mice, while the offspring of mothers exposed to light for 24 hours exhibited better morphometric features.

scholarly journals Use of KNN Classifier for Emotion Recognition Based on Distance Measures

2019 ◽  
Vol 9 (1) ◽  
pp. 1294-1298
Keyword(s):  
Gabor Filter ◽  
Texture Features ◽  
Distance Measures ◽  
Still Images ◽  
City Block ◽  
Lighting Conditions ◽  
Knn Classifier ◽  
The Face ◽  
City Block Distance

Human-computer interaction (HCI), in recent times, is gaining a lot of significance. The systems based on HCI have been designed for recognizing different facial expressions. The application areas for face recognition include robotics, safety, and surveillance system. The emotions so captured aid in predicting future actions in addition to providing valuable information. Fear, neutral, sad, surprise, happy are the categories of primary emotions. From the database of still images, certain features can be obtained using Gabor Filter (GF) and Histogram of Oriented Gradient (HOG). These two techniques are being used while extracting features for getting the expressions from the face. This paper focuses on the customized classification of GF and HOG using the KNN classifier.GF provides texture features whereas HOG finds applications for images exhibiting differing lighting conditions. Simplicity and linearity of KNN classifier appeals for its use in the present application. The paper also elaborates various distances used in KNN classifiers like city-block, Euclidean and correlation distance. This paper uses Matlab implementation of GF, HOG and KNN for extracting the required features and classification, respectively. Results exhibit that the accuracy of city- block distance is more .

Performance Analysis of Pose Invariant Face Recognition Approaches in Unconstrained Environments

2015 ◽  
Vol 5 (1) ◽  
pp. 66-81 ◽  
Author(s):  
M. Parisa Beham ◽  
S. M. Mansoor Roomi ◽  
J. Alageshan ◽  
V. Kapileshwaran
Keyword(s):  
Face Recognition ◽  
Nearest Neighbor ◽  
State Of The Art ◽  
Recognition Algorithm ◽  
K Nearest Neighbor ◽  
Research Issues ◽  
Histograms Of Oriented Gradients ◽  
Recognition Phase ◽  
The Face ◽  
Pose Variation

Face recognition and authentication are two significant and dynamic research issues in computer vision applications. There are many factors that should be accounted for face recognition; among them pose variation is a major challenge which severely influence in the performance of face recognition. In order to improve the performance, several research methods have been developed to perform the face recognition process with pose invariant conditions in constrained and unconstrained environments. In this paper, the authors analyzed the performance of a popular texture descriptors viz., Local Binary Pattern, Local Derivative Pattern and Histograms of Oriented Gradients for pose invariant problem. State of the art preprocessing techniques such as Discrete Cosine Transform, Difference of Gaussian, Multi Scale Retinex and Gradient face have also been applied before feature extraction. In the recognition phase K- nearest neighbor classifier is used to accomplish the classification task. To evaluate the efficiency of pose invariant face recognition algorithm three publicly available databases viz. UMIST, ORL and LFW datasets have been used. The above said databases have very wide pose variations and it is proved that the state of the art method is efficient only in constrained situations.

scholarly journals A novel three-in-one silicone model for basic microsurgery training

2020 ◽  
Vol 43 (5) ◽  
pp. 621-626
Author(s):  
Osaid Alser ◽  
Gehad Youssef ◽  
Simon Myers ◽  
Ali M. Ghanem
Keyword(s):  
Animal Models ◽  
Surgical Training ◽  
Face Validity ◽  
Level Of Evidence ◽  
The Face ◽  
Advanced Stages ◽  
Spss Software ◽  
Silicone Model ◽  
Synthetic Models ◽  
Suturing Skills

Abstract Background Microsurgery simulation is an important aspect of surgical training. Animal models have been widely used in simulation training, but they have some limitations including ethical restrictions, cost and availability. This has led to the use of synthetic models that can reduce reliance on animals in line with the 3R (refinement, reduction and replacement) principles. The aim of this paper was to evaluate the face validity of Surgitate™ three-in-one (artery, vein and nerve) silicone model. Methods Fourteen candidates performed one end-to-end anastomosis on artery, vein and nerve. The face validity of the vessel was assessed via questionnaires detailing their previous microsurgical experience and their feedback of using this model using the Likert scale. Data management and analysis were performed using IBM SPSS software (25.0). Results Participants tended to value this model in the earlier stages of microsurgical training particularly in the acquisition of basic microsurgical skills. It could be particularly useful in enhancing suturing skills as a replacement or reduction in the use of chicken models. The model has some drawbacks preluding its utilization into more advanced stages of surgical training. Further studies are needed to validate the model using more objective measures. Conclusion We present a novel synthetic model that can be potentially introduced to early stages of microsurgery training. The model would be ideal to meet the 3R principles of the use of animal models and as an alternative to the commonly used synthetic models. Level of evidence: Not ratable.

scholarly journals Implementing CCTV-Based Attendance Taking Support System Using Deep Face Recognition: A Case Study at FPT Polytechnic College

Symmetry ◽  
2020 ◽  
Vol 12 (2) ◽  
pp. 307 ◽  
Author(s):  
Ngo Tung Son ◽  
Bui Ngoc Anh ◽  
Tran Quy Ban ◽  
Le Phuong Chi ◽  
Bui Dinh Chien ◽  
...  
Keyword(s):  
Face Recognition ◽  
Motion Blur ◽  
Security Monitoring ◽  
Flexible System ◽  
Lighting Conditions ◽  
Satisfactory Performance ◽  
Measurement Results ◽  
The Face ◽  
Recognition Systems

Face recognition (FR) has received considerable attention in the field of security, especially in the use of closed-circuit television (CCTV) cameras in security monitoring. Although significant advances in the field of computer vision are made, advanced face recognition systems provide satisfactory performance only in controlled conditions. They deteriorate significantly in the face of real-world scenarios such as lighting conditions, motion blur, camera resolution, etc. This article shows how we design, implement, and conduct the empirical comparisons of machine learning open libraries in building attendance taking (AT) support systems using indoor security cameras called ATSS. Our trial system was deployed to record the appearances of 120 students in five classes who study on the third floor of FPT Polytechnic College building. Our design allows for flexible system scaling, and it is not only usable for a school but a generic attendance system with CCTV. The measurement results show that the accuracy is suitable for many different environments.

scholarly journals Deep-learning-based macro-pixel synthesis and lossless coding of light field images

2019 ◽  
Vol 8 ◽  
Author(s):  
Ionut Schiopu ◽  
Adrian Munteanu
Keyword(s):  
Light Field ◽  
State Of The Art ◽  
View Synthesis ◽  
Base Layer ◽  
Synthesis Process ◽  
Enhancement Layer ◽  
Novel Approach ◽  
Average Improvement ◽  
Lossless Coding ◽  
One Step

Abstract This paper proposes a novel approach for lossless coding of light field (LF) images based on a macro-pixel (MP) synthesis technique which synthesizes the entire LF image in one step. The reference views used in the synthesis process are selected based on four different view configurations and define the reference LF image. This image is stored as an array of reference MPs which collect one pixel from each reference view, being losslessly encoded as a base layer. A first contribution focuses on a novel network design for view synthesis which synthesizes the entire LF image as an array of synthesized MPs. A second contribution proposes a network model for coding which computes the MP prediction used for lossless encoding of the remaining views as an enhancement layer. Synthesis results show an average distortion of 29.82 dB based on four reference views and up to 36.19 dB based on 25 reference views. Compression results show an average improvement of 29.9% over the traditional lossless image codecs and 9.1% over the state-of-the-art.

Sign in / Sign up

Export Citation Format

Share Document