Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses

An improved spectral reflectance estimation method was developed to transform captured RGB images to spectral reflectance. The novelty of our method is an iteratively reweighted regulated model that combines polynomial expansion signals, which was developed for spectral reflectance estimation, and a cross-polarized imaging system, which is used to eliminate glare and specular highlights. Two RGB images are captured under two illumination conditions. The method was tested using ColorChecker charts. The results demonstrate that the proposed method could make a significant improvement of the accuracy in both spectral and colorimetric: it can achieve 23.8% improved accuracy in mean CIEDE2000 color difference, while it achieves 24.6% improved accuracy in RMS error compared with classic regularized least squares (RLS) method. The proposed method is sufficiently accurate in predicting the spectral properties and their performance within an acceptable range, i.e., typical customer tolerance of less than 3 DE units in the graphic arts industry.

Spectral-Reflectance Estimation under Multiple Light Sources

We describe a comprehensive method for estimating the surface-spectral reflectance from the image data of objects acquired under multiple light sources. This study uses the objects made of an inhomogeneous dielectric material with specular highlights. A spectral camera is used as an imaging system. The overall appearance of objects in a scene results from the chromatic factors such as reflectance and illuminant and the shading terms such as surface geometry and position. We first describe the method of estimating the illuminant spectra of multiple light sources based on detecting highlights appearing on object surfaces. The highlight candidates are detected first, and then some appropriate highlight areas are interactively selected among the candidates. Next, we estimate the spectral reflectance from a wide area selected from an object's surface. The color signals observed from the selected area are described using the estimated illuminant spectra, the surfacespectral reflectance, and the shading terms. This estimation utilizes the fact that the definition domains of reflectance and shading terms are different in each other. We develop an iterative algorithm for estimating the reflectance and the shading terms in two steps repeatedly. Finally, the feasibility of the proposed method is confirmed in an experiment using everyday objects under the illumination environment with multiple light sources.

Impact of View-Dependent Image-Based Effects on Perception of Visual Realism and Presence in Virtual Reality Environments Created Using Multi-Camera Systems

Several recent works have presented image-based methods for creating high-fidelity immersive virtual environments from photographs of real-world scenes. In this paper, we provide a user-centered evaluation of such methods by way of a user study investigating their impact on viewers’ perception of visual realism and sense of presence. In particular, we focus on two specific elements commonly introduced by image-based approaches. First, we investigate the extent to which using dedicated image-based rendering algorithms to render the scene with view-dependent effects (such as specular highlights) causes users to perceive it as being more realistic. Second, we study whether making the scene fade out beyond a fixed volume in 3D space significantly reduces participants’ feeling of being there, examining different sizes for this viewing volume. To provide details on the virtual environment used in the study, we also describe how we recreated a museum gallery for room-scale virtual reality using a custom-built multi-camera rig. The results of our study show that using image-based rendering to render view-dependent effects can effectively enhance the perception of visual realism and elicit a stronger sense of presence, even when it implies constraining the viewing volume to a small range of motion.

Developing a multi-spectral imaging system using a RGB camera under two illuminations

This paper proposes a multi-spectral imaging system, developed using a commercial-grade camera, under two commonly used illumination. Rather than using conventional direct or diffuse light, the novelty of our method is to use a cross-polarized imaging system to eliminate glare and specular highlights. Two RGB images are captured under two different color temperature lighting conditions. An improved reflectance estimation method is developed to transform camera RGB under two illumination to spectral reflectance using a regulated model, combining the polynomial expansion of the camera signals with optimally selected feature. The method was tested using both a semi-gloss ColorChecker SG (140) and matte ColorChecker DC (240) chart. The results indicate that the proposed method significantly outperforms the traditional methods both in terms of spectra and colorimetric accuracy. This new multi-spectral imaging system is sufficiently precise to predict spectra properties and its performance within an acceptable range.

Structurally assisted super black in colourful peacock spiders

Male peacock spiders ( Maratus , Salticidae) compete to attract female mates using elaborate, sexually selected displays. They evolved both brilliant colour and velvety black. Here, we use scanning electron microscopy, hyperspectral imaging and finite-difference time-domain optical modelling to investigate the deep black surfaces of peacock spiders. We found that super black regions reflect less than 0.5% of light (for a 30° collection angle) in Maratus speciosus (0.44%) and Maratus karrie (0.35%) owing to microscale structures. Both species evolved unusually high, tightly packed cuticular bumps (microlens arrays), and M. karrie has an additional dense covering of black brush-like scales atop the cuticle. Our optical models show that the radius and height of spider microlenses achieve a balance between (i) decreased surface reflectance and (ii) enhanced melanin absorption (through multiple scattering, diffraction out of the acceptance cone of female eyes and increased path length of light through absorbing melanin pigments). The birds of paradise (Paradiseidae), ecological analogues of peacock spiders, also evolved super black near bright colour patches. Super black locally eliminates white specular highlights, reference points used to calibrate colour perception, making nearby colours appear brighter, even luminous, to vertebrates. We propose that this pre-existing, qualitative sensory experience—‘sensory bias’—is also found in spiders, leading to the convergent evolution of super black for mating displays in jumping spiders.