Pre-Flight Calibration of the Mars 2020 Rover Mastcam Zoom (Mastcam-Z) Multispectral, Stereoscopic Imager

The NASA Perseverance rover Mast Camera Zoom (Mastcam-Z) system is a pair of zoomable, focusable, multi-spectral, and color charge-coupled device (CCD) cameras mounted on top of a 1.7 m Remote Sensing Mast, along with associated electronics and two calibration targets. The cameras contain identical optical assemblies that can range in focal length from 26 mm ($25.5^{\circ }\, \times 19.1^{\circ }\ \mathrm{FOV}$ 25.5 ∘ × 19.1 ∘ FOV ) to 110 mm ($6.2^{\circ } \, \times 4.2^{\circ }\ \mathrm{FOV}$ 6.2 ∘ × 4.2 ∘ FOV ) and will acquire data at pixel scales of 148-540 μm at a range of 2 m and 7.4-27 cm at 1 km. The cameras are mounted on the rover’s mast with a stereo baseline of $24.3\pm 0.1$ 24.3 ± 0.1 cm and a toe-in angle of $1.17\pm 0.03^{\circ }$ 1.17 ± 0.03 ∘ (per camera). Each camera uses a Kodak KAI-2020 CCD with $1600\times 1200$ 1600 × 1200 active pixels and an 8 position filter wheel that contains an IR-cutoff filter for color imaging through the detectors’ Bayer-pattern filters, a neutral density (ND) solar filter for imaging the sun, and 6 narrow-band geology filters (16 total filters). An associated Digital Electronics Assembly provides command data interfaces to the rover, 11-to-8 bit companding, and JPEG compression capabilities. Herein, we describe pre-flight calibration of the Mastcam-Z instrument and characterize its radiometric and geometric behavior. Between April 26$^{th}$ t h and May 9$^{th}$ t h , 2019, ∼45,000 images were acquired during stand-alone calibration at Malin Space Science Systems (MSSS) in San Diego, CA. Additional data were acquired during Assembly Test and Launch Operations (ATLO) at the Jet Propulsion Laboratory and Kennedy Space Center. Results of the radiometric calibration validate a 5% absolute radiometric accuracy when using camera state parameters investigated during testing. When observing using camera state parameters not interrogated during calibration (e.g., non-canonical zoom positions), we conservatively estimate the absolute uncertainty to be $<10\%$ < 10 % . Image quality, measured via the amplitude of the Modulation Transfer Function (MTF) at Nyquist sampling (0.35 line pairs per pixel), shows $\mathrm{MTF}_{\mathit{Nyquist}}=0.26-0.50$ MTF Nyquist = 0.26 − 0.50 across all zoom, focus, and filter positions, exceeding the $>0.2$ > 0.2 design requirement. We discuss lessons learned from calibration and suggest tactical strategies that will optimize the quality of science data acquired during operation at Mars. While most results matched expectations, some surprises were discovered, such as a strong wavelength and temperature dependence on the radiometric coefficients and a scene-dependent dynamic component to the zero-exposure bias frames. Calibration results and derived accuracies were validated using a Geoboard target consisting of well-characterized geologic samples.

Comparison of Denoising Algorithms for Demosacing Low Lighting Images Using CFA 2.0

In modern digital cameras, the Bayer color filter array (CFA) has been widely used. It is also widely known as CFA 1.0. However, Bayer pattern is inferior to the red-green-blue-white (RGBW) pattern, which is also known as CFA 2.0, in low lighting conditions in which Poisson noise is present. It is well known that demosaicing algorithms cannot effectively deal with Poisson noise and additional denoising is needed in order to improve the image quality. In this paper, we propose to evaluate various conventional and deep learning based denoising algorithms for CFA 2.0 in low lighting conditions. We will also investigate the impact of the location of denoising, which refers to whether the denoising is done before or after a critical step of demosaicing. Extensive experiments show that some denoising algorithms can indeed improve the image quality in low lighting conditions. We also noticed that the location of denoising plays an important role in the overall demosaicing performance.

Demosaicing of CFA 3.0 with Applications to Low Lighting Images

Low lighting images usually contain Poisson noise, which is pixel amplitude-dependent. More panchromatic or white pixels in a color filter array (CFA) are believed to help the demosaicing performance in dark environments. In this paper, we first introduce a CFA pattern known as CFA 3.0 that has 75% white pixels, 12.5% green pixels, and 6.25% of red and blue pixels. We then present algorithms to demosaic this CFA, and demonstrate its performance for normal and low lighting images. In addition, a comparative study was performed to evaluate the demosaicing performance of three CFAs, namely the Bayer pattern (CFA 1.0), the Kodak CFA 2.0, and the proposed CFA 3.0. Using a clean Kodak dataset with 12 images, we emulated low lighting conditions by introducing Poisson noise into the clean images. In our experiments, normal and low lighting images were used. For the low lighting conditions, images with signal-to-noise (SNR) of 10 dBs and 20 dBs were studied. We observed that the demosaicing performance in low lighting conditions was improved when there are more white pixels. Moreover, denoising can further enhance the demosaicing performance for all CFAs. The most important finding is that CFA 3.0 performs better than CFA 1.0, but is slightly inferior to CFA 2.0, in low lighting images.

Demosaicing of Bayer and CFA 2.0 Patterns for Low Lighting Images

It is commonly believed that having more white pixels in a color filter array (CFA) will help the demosaicing performance for images collected in low lighting conditions. However, to the best of our knowledge, a systematic study to demonstrate the above statement does not exist. We present a comparative study to systematically and thoroughly evaluate the performance of demosaicing for low lighting images using two CFAs: the standard Bayer pattern (aka CFA 1.0) and the Kodak CFA 2.0 (RGBW pattern with 50% white pixels). Using the clean Kodak dataset containing 12 images, we first emulated low lighting images by injecting Poisson noise at two signal-to-noise (SNR) levels: 10 dBs and 20 dBs. We then created CFA 1.0 and CFA 2.0 images for the noisy images. After that, we applied more than 15 conventional and deep learning based demosaicing algorithms to demosaic the CFA patterns. Using both objectives with five performance metrics and subjective visualization, we observe that having more white pixels indeed helps the demosaicing performance in low lighting conditions. This thorough comparative study is our first contribution. With denoising, we observed that the demosaicing performance of both CFAs has been improved by several dBs. This can be considered as our second contribution. Moreover, we noticed that denoising before demosaicing is more effective than denoising after demosaicing. Answering the question of where denoising should be applied is our third contribution. We also noticed that denoising plays a slightly more important role in 10 dBs signal-to-noise ratio (SNR) as compared to 20 dBs SNR. Some discussions on the following phenomena are also included: (1) why CFA 2.0 performed better than CFA 1.0; (2) why denoising was more effective before demosaicing than after demosaicing; and (3) why denoising helped more at low SNRs than at high SNRs.