A practical approach on non-regular sampling and universal demosaicing of raw image sensor data

Non-regular sampling is a well-known method to avoid aliasing in digital images. However, the vast majority of single sensor cameras use regular organized color filter arrays (CFAs), that require an optical-lowpass filter (OLPF) and sophisticated demosaicing algorithms to suppress sampling errors. In this paper a variety of non-regular sampling patterns are evaluated, and a new universal demosaicing algorithm based on the frequency selective reconstruction is presented. By simulating such sensors it is shown that images acquired with non-regular CFAs and no OLPF can lead to a similar image quality compared to their filtered and regular sampled counterparts. The MATLAB source code and results are available at: http://github. com/PhilippBackes/dFSR

Download Full-text

Deep Image Demosaicing for Submicron Image Sensors

Journal of Imaging Science and Technology ◽

10.2352/j.imagingsci.technol.2019.63.6.060410 ◽

2019 ◽

Vol 63 (6) ◽

pp. 60410-1-60410-12

Author(s):

Irina Kim ◽

Seongwook Song ◽

Soonkeun Chang ◽

Sukhwan Lim ◽

Kai Guo

Keyword(s):

Image Quality ◽

Mobile Devices ◽

State Of The Art ◽

Image Sensor ◽

Sensor Technology ◽

Color Filter ◽

Computationally Efficient ◽

Linear Feature ◽

Visual Artifacts ◽

Very High

Abstract Latest trend in image sensor technology allowing submicron pixel size for high-end mobile devices comes at very high image resolutions and with irregularly sampled Quad Bayer color filter array (CFA). Sustaining image quality becomes a challenge for the image signal processor (ISP), namely for demosaicing. Inspired by the success of deep learning approach to standard Bayer demosaicing, we aim to investigate how artifacts-prone Quad Bayer array can benefit from it. We found that deeper networks are capable to improve image quality and reduce artifacts; however, deeper networks can be hardly deployed on mobile devices given very high image resolutions: 24MP, 36MP, 48MP. In this article, we propose an efficient end-to-end solution to bridge this gap—a duplex pyramid network (DPN). Deep hierarchical structure, residual learning, and linear feature map depth growth allow very large receptive field, yielding better details restoration and artifacts reduction, while staying computationally efficient. Experiments show that the proposed network outperforms state of the art for standard and Quad Bayer demosaicing. For the challenging Quad Bayer CFA, the proposed method reduces visual artifacts better than state-of-the-art deep networks including artifacts existing in conventional commercial solutions. While superior in image quality, it is 2‐25 times faster than state-of-the-art deep neural networks and therefore feasible for deployment on mobile devices, paving the way for a new era of on-device deep ISPs.

Download Full-text

Noise Removal in the Developing Process of Digital Negatives

Sensors ◽

10.3390/s20030902 ◽

2020 ◽

Vol 20 (3) ◽

pp. 902

Author(s):

Marek Szczepański ◽

Filip Giemza

Keyword(s):

Noise Reduction ◽

Color Image ◽

Image Acquisition ◽

Noise Removal ◽

Image Sensor ◽

Sensor Data ◽

Color Filter ◽

Local Means ◽

Real Process ◽

Cmos Sensor

Most modern color digital cameras are equipped with a single image sensor with a color filter array (CFA). One of the most important stages of preprocessing is noise reduction. Most research related to this topic ignores the problem associated with the actual color image acquisition process and assumes that we are processing the image in the sRGB space. In the presented paper, the real process of developing raw images obtained from the CFA sensor was analyzed. As part of the work, a diverse database of test images in the form of a digital negative and its reference version was prepared. The main problem posed in the work was the location of the denoising and demosaicing algorithms in the entire raw image processing pipeline. For this purpose, all stages of processing the digital negative are reproduced. The process of noise generation in the image sensors was also simulated, parameterizing it with ISO sensitivity for a specific CMOS sensor. In this work, we tested commonly used algorithms based on the idea of non-local means, such as NLM or BM3D, in combination with various techniques of interpolation of CFA sensor data. Our experiments have shown that the use of noise reduction methods directly on the raw sensor data, improves the final result only in the case of highly disturbed images, which corresponds to the process of image acquisition in difficult lighting conditions.

Download Full-text

ROI Based Post Image Quality Assessment Technique on Multiple Localized Filtering Method On Kinect HD Sensor Mobile Robot

Kinetik Game Technology Information System Computer Network Computing Electronics and Control ◽

10.22219/kinetik.v3i2.598 ◽

2018 ◽

Vol 3 (2) ◽

pp. 145

Author(s):

Kholilatul Wardani ◽

Aditya Kurniawan

Keyword(s):

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Similarity Index ◽

Assessment Model ◽

Sensor Data ◽

Edge Region ◽

Kinect Sensor ◽

Image Region ◽

Filtering Technique

The ROI (Region of Interest) Image Quality Assessment is an image quality assessment model based on the SSI (Structural Similarity Index) index used in the specific image region desired to be assessed. Output assessmen value used by this image assessment model is 1 which means identical and -1 which means not identical. Assessment model of ROI Quality Assessment in this research is used to measure image quality on Kinect sensor capture result used in Mobile HD Robot after applied Multiple Localized Filtering Technique. The filter is applied to each capture sensor depth result on Kinect, with the aim to eliminate structural noise that occurs in the Kinect sensor. Assessment is done by comparing image quality before filter and after filter applied to certain region. The kinect sensor will be conditioned to capture a square black object measuring 10cm x 10cm perpendicular to a homogeneous background (white with RGB code 255,255,255). The results of kinect sensor data will be taken through EWRF 3022 by visual basic 6.0 program periodically 10 times each session with frequency 1 time per minute. The results of this trial show the same similar index (value 1: identical) in the luminance, contrast, and structural section of the edge region or edge region of the specimen. The value indicates that the Multiple Localized Filtering Technique applied to the noise generated by the Kinect sensor, based on the ROI Image Quality Assessment model has no effect on the image quality generated by the sensor.

Download Full-text

Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

Applied System Innovation ◽

10.3390/asi4010003 ◽

2021 ◽

Vol 4 (1) ◽

pp. 3

Author(s):

Parag Narkhede ◽

Rahee Walambe ◽

Shruti Mandaokar ◽

Pulkit Chandel ◽

Ketan Kotecha ◽

...

Keyword(s):

Sensor Fusion ◽

Network Architecture ◽

Cost Effective ◽

Sensor Data ◽

Dense Layer ◽

Gaseous Emissions ◽

Multiple Sensors ◽

Single Output ◽

Novel Approach ◽

Single Sensor

With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissions using the multimodal AI fusion techniques. Most of the gases and their fumes are colorless, odorless, and tasteless, thereby challenging our normal human senses. Sensing based on a single sensor may not be accurate, and sensor fusion is essential for robust and reliable detection in several real-world applications. We manually collected 6400 gas samples (1600 samples per class for four classes) using two specific sensors: the 7-semiconductor gas sensors array, and a thermal camera. The early fusion method of multimodal AI, is applied The network architecture consists of a feature extraction module for individual modality, which is then fused using a merged layer followed by a dense layer, which provides a single output for identifying the gas. We obtained the testing accuracy of 96% (for fused model) as opposed to individual model accuracies of 82% (based on Gas Sensor data using LSTM) and 93% (based on thermal images data using CNN model). Results demonstrate that the fusion of multiple sensors and modalities outperforms the outcome of a single sensor.

Download Full-text

Image Reconstruction on Quincunx Patterned Green Channel

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.717.493 ◽

2013 ◽

Vol 717 ◽

pp. 493-496

Author(s):

Gwang Gil Jeon

Keyword(s):

Image Reconstruction ◽

Least Squares ◽

Least Squares Method ◽

Interpolation Method ◽

Experimental Results ◽

Color Filter ◽

Image Artifacts ◽

Green Channel ◽

Filter Array ◽

Single Sensor

This paper addresses the issue of the quincunx patterned green channel interpolation method that is obtained by single sensor cameras. Our goal is to reconstruct the green channel in Bayer color filter array (CFA) data. We present a new filter-based method for the reduction of image artifacts in green channel. To reconstruct green channel, we trained a filter using least squares method. Experimental results confirm the effectiveness of the proposed method. Compared to other bilinear and bicubic filters, the improvement in quality has been achieved.

Download Full-text

Health Monitoring and Diagnosis of Equipment Based on Multi-sensor Fusion

International Journal of Online Engineering (iJOE) ◽

10.3991/ijoe.v14i04.8315 ◽

2018 ◽

Vol 14 (04) ◽

pp. 4

Author(s):

Xuemei Yao ◽

Shaobo Li ◽

Yong Yao ◽

Xiaoting Xie

Keyword(s):

Evidence Theory ◽

Sensor Data ◽

Uncertain Information ◽

Fusion Algorithm ◽

Monitoring And Diagnosis ◽

Multi Sensor Data Fusion ◽

Basic Probability ◽

Optimal Fusion ◽

Single Sensor ◽

Mechanical Devices

As the information measured by a single sensor cannot reflect the real situation of mechanical devices completely, a multi-sensor data fusion based on evidence theory is introduced. Evidence theory has the advantage of dealing with uncertain information. However, it produces unreasonable conclusions when the evidence conflicts. An improved fusion method is proposed to solve this problem. Basic probability assignment of evidence is corrected according to evidence and sensor weights, and an optimal fusion algorithm is selected by comparing an introduced threshold and a conflict factor. The effectiveness and practicability of the algorithm are tested by simulating the monitoring and diagnosis of rolling bearings. The result shows that the method has better robustness.

Download Full-text