Plasmonic color filter array based visible light spectroscopy

Compared with traditional Fabry–Perot optical filters, plasmonic color filters could greatly remedy the complexity and reduce the cost of manufacturing. In this paper we present end-to-end demonstration of visible light spectroscopy based on highly selective plasmonic color filter array based on resonant grating structure. The spectra of 6 assorted samples were measured using an array of 20 narrowband color filters and detected signals were used to reconstruct original spectra by using new unmixing algorithm and by solving least squares problem with smoothing regularization. The original spectra were reconstructed with less than 0.137 root mean squared error. This works shows promise towards fully integrating plasmonic color filter array in imagers used in hyperspectral cameras.

Data-Driven Convolutional Model for Digital Color Image Demosaicing

Modern digital cameras use specific arrangement of Color Filter Array to sample light wavelength corresponding to visible colors. The most common Color Filter Array is the Bayer filter that samples only one color per pixel. To recover the full resolution image, an interpolation algorithm can be used. This process is called demosaicing and it is one of the first processing stages of a digital imaging pipeline. We introduce a novel data-driven model for demosaicing that takes into account the different requirements for reconstruction of the image Luma and Chrominance channels. The final model is a parallel composition of two reconstruction networks with individual architecture and trained with distinct loss functions. In order to solve the overfitting problem, we prepared a dataset that contains groups of patches that share common chromatic and spectral characteristics. We reported the reconstruction error on noise-free images and measured the effect of random noise and quantization noise in the demosaicing reconstruction. To test our model performance, we implemented the network on NVIDIA Jetson Nano, obtaining an end-to-end running time of less than one second for a full frame 12 MPixel image.

Further Improvements of CFA 3.0 by Combining Inpainting and Pansharpening Techniques

Color Filter Array (CFA) has been widely used in digital cameras. There are many variants of CFAs in the literature. Recently, a new CFA known as CFA 3.0 was proposed by us and has been shown to yield reasonable performance as compared to some standard ones. In this paper, we investigate the use of inpainting algorithms to further improve the demosaicing performance of CFA 3.0. Six conventional and deep learning based inpainting algorithms were compared. Extensive experiments demonstrated that one algorithm improved over other approaches.

An Acquisition Method for Visible and Near Infrared Images from Single CMYG Color Filter Array-Based Sensor

Near-infrared (NIR) images are very useful in many image processing applications, including banknote recognition, vein detection, and surveillance, to name a few. To acquire the NIR image together with visible range signals, an imaging device should be able to simultaneously capture NIR and visible range images. An implementation of such a system having separate sensors for NIR and visible light has practical shortcomings due to its size and hardware cost. To overcome this, a single sensor-based acquisition method is investigated in this paper. The proposed imaging system is equipped with a conventional color filter array of cyan, magenta, yellow, and green, and achieves signal separation by applying a proposed separation matrix which is derived by mathematical modeling of the signal acquisition structure. The elements of the separation matrix are calculated through color space conversion and experimental data. Subsequently, an additional denoising process is implemented to enhance the quality of the separated images. Experimental results show that the proposed method successfully separates the acquired mixed image of visible and near-infrared signals into individual red, green, and blue (RGB) and NIR images. The separation performance of the proposed method is compared to that of related work in terms of the average peak-signal-to-noise-ratio (PSNR) and color distance. The proposed method attains average PSNR value of 37.04 and 33.29 dB, respectively for the separated RGB and NIR images, which is respectively 6.72 and 2.55 dB higher than the work used for comparison.

Demosaicing of CFA 3.0 with Applications to Low Lighting Images

Low lighting images usually contain Poisson noise, which is pixel amplitude-dependent. More panchromatic or white pixels in a color filter array (CFA) are believed to help the demosaicing performance in dark environments. In this paper, we first introduce a CFA pattern known as CFA 3.0 that has 75% white pixels, 12.5% green pixels, and 6.25% of red and blue pixels. We then present algorithms to demosaic this CFA, and demonstrate its performance for normal and low lighting images. In addition, a comparative study was performed to evaluate the demosaicing performance of three CFAs, namely the Bayer pattern (CFA 1.0), the Kodak CFA 2.0, and the proposed CFA 3.0. Using a clean Kodak dataset with 12 images, we emulated low lighting conditions by introducing Poisson noise into the clean images. In our experiments, normal and low lighting images were used. For the low lighting conditions, images with signal-to-noise (SNR) of 10 dBs and 20 dBs were studied. We observed that the demosaicing performance in low lighting conditions was improved when there are more white pixels. Moreover, denoising can further enhance the demosaicing performance for all CFAs. The most important finding is that CFA 3.0 performs better than CFA 1.0, but is slightly inferior to CFA 2.0, in low lighting images.

An Improved Demosaicking Algorithm Based On Region-Dividing

The Bayer Color Filter Array (CFA) is commonly used in such industries as digital cameras. However, due to the arrangement of color channels in the Bayer CFA, it becomes a problem to estimate the missed color information in each pixel. The algorithms that deal with this problem are named "demosaicking algorithms". There are many demosaicking algorithms, which show different efficiency and image quality for different images. This paper proposes an algorithm that combines two existing algorithms to reach better image qualities and acceptable computing complexities. The experimental results indicate effectiveness in terms of the balance between complexity and quality.