Subpixel-Scale Topography Retrieval of Mars Using Single-Image DTM Estimation and Super-Resolution Restoration

We propose using coupled deep learning based super-resolution restoration (SRR) and single-image digital terrain model (DTM) estimation (SDE) methods to produce subpixel-scale topography from single-view ESA Trace Gas Orbiter Colour and Stereo Surface Imaging System (CaSSIS) and NASA Mars Reconnaissance Orbiter High Resolution Imaging Science Experiment (HiRISE) images. We present qualitative and quantitative assessments of the resultant 2 m/pixel CaSSIS SRR DTM mosaic over the ESA and Roscosmos Rosalind Franklin ExoMars rover’s (RFEXM22) planned landing site at Oxia Planum. Quantitative evaluation shows SRR improves the effective resolution of the resultant CaSSIS DTM by a factor of 4 or more, while achieving a fairly good height accuracy measured by root mean squared error (1.876 m) and structural similarity (0.607), compared to the ultra-high-resolution HiRISE SRR DTMs at 12.5 cm/pixel. We make available, along with this paper, the resultant CaSSIS SRR image and SRR DTM mosaics, as well as HiRISE full-strip SRR images and SRR DTMs, to support landing site characterisation and future rover engineering for the RFEXM22.

MADNet 2.0: Pixel-Scale Topography Retrieval from Single-View Orbital Imagery of Mars Using Deep Learning

The High-Resolution Imaging Science Experiment (HiRISE) onboard the Mars Reconnaissance Orbiter provides remotely sensed imagery at the highest spatial resolution at 25–50 cm/pixel of the surface of Mars. However, due to the spatial resolution being so high, the total area covered by HiRISE targeted stereo acquisitions is very limited. This results in a lack of the availability of high-resolution digital terrain models (DTMs) which are better than 1 m/pixel. Such high-resolution DTMs have always been considered desirable for the international community of planetary scientists to carry out fine-scale geological analysis of the Martian surface. Recently, new deep learning-based techniques that are able to retrieve DTMs from single optical orbital imagery have been developed and applied to single HiRISE observational data. In this paper, we improve upon a previously developed single-image DTM estimation system called MADNet (1.0). We propose optimisations which we collectively call MADNet 2.0, which is based on a supervised image-to-height estimation network, multi-scale DTM reconstruction, and 3D co-alignment processes. In particular, we employ optimised single-scale inference and multi-scale reconstruction (in MADNet 2.0), instead of multi-scale inference and single-scale reconstruction (in MADNet 1.0), to produce more accurate large-scale topographic retrieval with boosted fine-scale resolution. We demonstrate the improvements of the MADNet 2.0 DTMs produced using HiRISE images, in comparison to the MADNet 1.0 DTMs and the published Planetary Data System (PDS) DTMs over the ExoMars Rosalind Franklin rover’s landing site at Oxia Planum. Qualitative and quantitative assessments suggest the proposed MADNet 2.0 system is capable of producing pixel-scale DTM retrieval at the same spatial resolution (25 cm/pixel) of the input HiRISE images.

Evaluating Stereo Digital Terrain Model Quality at Mars Rover Landing Sites with HRSC, CTX, and HiRISE Images

We have used high-resolution digital terrain models (DTMs) of two rover landing sites based on mosaicked images from the High-Resolution Imaging Science Experiment (HiRISE) camera as a reference to evaluate DTMs based on High-Resolution Stereo Camera (HRSC) and Context Camera (CTX) images. The Next-Generation Automatic Terrain Extraction (NGATE) matcher in the SOCET SET and GXP® commercial photogrammetric systems produces DTMs with good (small) horizontal resolution but large vertical error. Somewhat surprisingly, results for NGATE are terrain dependent, with poorer resolution and smaller errors on smoother surfaces. Multiple approaches to smoothing the NGATE DTMs give similar tradeoffs between resolution and error; a 5 × 5 lowpass filter is near optimal in terms of both combined resolution-error performance and local slope estimation. Smoothing with an area-based matcher, the standard processing for U.S. Geological Survey planetary DTMs, yields similar errors to the 5 × 5 filter at slightly worse resolution. DTMs from the HRSC team processing pipeline fall within this same trade space but are less sensitive to terrain roughness. DTMs produced with the Ames Stereo Pipeline also fall in this space at resolutions intermediate between NGATE and the team pipeline. Considered individually, resolution and error each varied by approximately a factor of 2. Matching errors were 0.2–0.5 pixels but most results fell in the 0.2–0.3 pixel range that has been stated as a rule of thumb in multiple prior studies. Horizontal resolutions of 10–20 image pixels were found, consistently greater than the 3–5 pixel spacing generally used for stereo DTM production. Resolution and precision were inversely correlated; their product varied by ≤20% (4–5 pixels squared). Refinement of the stereo DTM by photoclinometry can yield quantitative improvement in resolution (more than a factor of 2), provided that albedo variations over distances smaller than the stereo DTM resolution are not too severe. We offer specific guidance for both producers and users of planetary stereo DTMs, based on our results.

Large Area High-Resolution 3D Mapping of Oxia Planum: The Landing Site for the ExoMars Rosalind Franklin Rover

We demonstrate an end-to-end application of the in-house deep learning-based surface modelling system, called MADNet, to produce three large area 3D mapping products from single images taken from the ESA Mars Express’s High Resolution Stereo Camera (HRSC), the NASA Mars Reconnaissance Orbiter’s Context Camera (CTX), and the High Resolution Imaging Science Experiment (HiRISE) imaging data over the ExoMars 2022 Rosalind Franklin rover’s landing site at Oxia Planum on Mars. MADNet takes a single orbital optical image as input, provides pixelwise height predictions, and uses a separate coarse Digital Terrain Model (DTM) as reference, to produce a DTM product from the given input image. Initially, we demonstrate the resultant 25 m/pixel HRSC DTM mosaic covering an area of 197 km × 182 km, providing fine-scale details to the 50 m/pixel HRSC MC-11 level-5 DTM mosaic. Secondly, we demonstrate the resultant 12 m/pixel CTX MADNet DTM mosaic covering a 114 km × 117 km area, showing much more detail in comparison to photogrammetric DTMs produced using the open source in-house developed CASP-GO system. Finally, we demonstrate the resultant 50 cm/pixel HiRISE MADNet DTM mosaic, produced for the first time, covering a 74.3 km × 86.3 km area of the 3-sigma landing ellipse and partially the ExoMars team’s geological characterisation area. The resultant MADNet HiRISE DTM mosaic shows fine-scale details superior to existing Planetary Data System (PDS) HiRISE DTMs and covers a larger area that is considered difficult for existing photogrammetry and photoclinometry pipelines to achieve, especially given the current limitations of stereo HiRISE coverage. All of the resultant DTM mosaics are co-aligned with each other, and ultimately with the Mars Global Surveyor’s Mars Orbiter Laser Altimeter (MOLA) DTM, providing high spatial and vertical congruence. In this paper, technical details are presented, issues that arose are discussed, along with a visual evaluation and quantitative assessments of the resultant DTM mosaic products.

Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets

The lack of adequate stereo coverage and where available, lengthy processing time, various artefacts, and unsatisfactory quality and complexity of automating the selection of the best set of processing parameters, have long been big barriers for large-area planetary 3D mapping. In this paper, we propose a deep learning-based solution, called MADNet (Multi-scale generative Adversarial u-net with Dense convolutional and up-projection blocks), that avoids or resolves all of the above issues. We demonstrate the wide applicability of this technique with the ExoMars Trace Gas Orbiter Colour and Stereo Surface Imaging System (CaSSIS) 4.6 m/pixel images on Mars. Only a single input image and a coarse global 3D reference are required, without knowing any camera models or imaging parameters, to produce high-quality and high-resolution full-strip Digital Terrain Models (DTMs) in a few seconds. In this paper, we discuss technical details of the MADNet system and provide detailed comparisons and assessments of the results. The resultant MADNet 8 m/pixel CaSSIS DTMs are qualitatively very similar to the 1 m/pixel HiRISE DTMs. The resultant MADNet CaSSIS DTMs display excellent agreement with nested Mars Reconnaissance Orbiter Context Camera (CTX), Mars Express’s High-Resolution Stereo Camera (HRSC), and Mars Orbiter Laser Altimeter (MOLA) DTMs at large-scale, and meanwhile, show fairly good correlation with the High-Resolution Imaging Science Experiment (HiRISE) DTMs for fine-scale details. In addition, we show how MADNet outperforms traditional photogrammetric methods, both on speed and quality, for other datasets like HRSC, CTX, and HiRISE, without any parameter tuning or re-training of the model. We demonstrate the results for Oxia Planum (the landing site of the European Space Agency’s Rosalind Franklin ExoMars rover 2023) and a couple of sites of high scientific interest.

What is the “opposite” of “blue”? The language of colour wheels

© Society for Imaging Science and Technology 2019 A color wheel is a tool for ordering and understanding hue. Different color wheels differ in the spacing of the colors around the wheel. The opponent-color theory, Munsell's color system, the standard printer's primaries, the artist's primaries, and Newton's rainbow all present different variations of the color wheel. I show that some of this variation is owing to imprecise use of language, based on Berlin and Kay's theory of basic color names. I also show that the artist's color wheel is an outlier that does not match well to the technical color wheels because its principal colors are so strongly connected to the basic color names.

What is the “opposite” of “blue”? The language of colour wheels

© Society for Imaging Science and Technology 2019 A color wheel is a tool for ordering and understanding hue. Different color wheels differ in the spacing of the colors around the wheel. The opponent-color theory, Munsell's color system, the standard printer's primaries, the artist's primaries, and Newton's rainbow all present different variations of the color wheel. I show that some of this variation is owing to imprecise use of language, based on Berlin and Kay's theory of basic color names. I also show that the artist's color wheel is an outlier that does not match well to the technical color wheels because its principal colors are so strongly connected to the basic color names.

FURTHER ADVENTURES IN MARS DTM QUALITY: SMOOTHING ERRORS, SHARPENING DETAILS

We have used high-precision, high-resolution digital terrain models (DTMs) of the NASA Mars Science Laboratory and Mars 2020 rover landing sites based on mosaicked images from the Mars Reconnaissance Orbiter High Resolution Imaging Science Experiment (MRO HiRISE) camera as a reference data set to evaluate DTMs based on Mars Express High Resolution Stereo Camera (MEX HRSC) images. The Next Generation Automatic Terrain Extraction (NGATE) matcher in the SOCET SET/GXP® commercial photogram- metric system produces DTMs with relatively good (small) horizontal resolution but high error, and results are terrain dependent, with poorer resolution and smaller errors on smoother surfaces. Multiple approaches to smoothing the NGATE DTMs give very similar tradeoffs between resolution and error. Smoothing the NGATE DTMs with a 5x5 lowpass filter is near optimal in terms of both combined resolution-error performance and local slope estimation, but smoothing with a single pass of an area-based matcher, which has been the standard approach for generating planetary DTMs at the U.S. Geological Survey to date results in similar errors and only slightly worse resolution. DTMs from the HRSC team processing pipeline fall within this same trade space but are less sensitive to terrain roughness. DTMs produced with the Ames Stereo Pipeline also fall in this space at resolutions intermediate between NGATE and the team pipeline. Although DTM resolution and error each vary by a factor of 2, the product of resolution and error is much more consistent, varying by ≤20% across multiple image sets and matching algorithms. Refinement of the stereo DTM by photoclinometry can yield significant quantitative improvement in resolution and some improvement in error (improving their product by as much as a factor of 2), provided that albedo variations over distances smaller than the stereo DTM resolution are not too severe.

DETECTING THE SOURCES OF ICE BLOCK FALLS AT THE MARTIAN NORTH POLAR SCARPS BY ANALYSIS OF MULTI-TEMPORAL HIRISE IMAGERY

We have developed a method for automatically detecting the sources of ice block falls at the Martian north polar scarps. Multitemporal red-filter High Resolution Imaging Science Experiment (HiRISE) images were processed by using the open source NASA Ames Stereo Pipeline in combination with the USGS Integrated Software for Imagers and Spectrometers to produce 0.25 m resolution images as well as a 1 m resolution DTM. The multi-temporal HiRISE images were firstly ortho-rectified by the DTM, and then co-registered by using the Enhanced Correlation Coefficient Maximization (ECC) algorithm. We applied the change detection method on the well-aligned sub-meter scale HiRISE images, which were taken in Mars Year 29 and Mars Year 30, to investigate mass wasting at the scarp area centred at 85.0°N, 151.5°E. The idea of the change detection method is to identify changing shadow patterns based on the grayscale difference between the images. The final results show that erosion events occurred at the full length of this study’s scarp within one Mars Year. However, only the upper and lower part of the scarp show erosion activity, whereas the intermediate parts seem inactive, and this correlates with the slope.