scholarly journals Mars3DNet: CNN-Based High-Resolution 3D Reconstruction of the Martian Surface from Single Images

2021 ◽  
Vol 13 (5) ◽  
pp. 839
Author(s):  
Zeyu Chen ◽  
Bo Wu ◽  
Wai Chung Liu
Keyword(s):  
High Resolution ◽  
3D Reconstruction ◽  
Real Data ◽  
Experimental Results ◽  
Training Data ◽  
Stereo Images ◽  
Martian Surface ◽  
Laser Altimetry ◽  
3D Surface ◽  
Imaging Science

Three-dimensional (3D) surface models, e.g., digital elevation models (DEMs), are important for planetary exploration missions and scientific research. Current DEMs of the Martian surface are mainly generated by laser altimetry or photogrammetry, which have respective limitations. Laser altimetry cannot produce high-resolution DEMs; photogrammetry requires stereo images, but high-resolution stereo images of Mars are rare. An alternative is the convolutional neural network (CNN) technique, which implicitly learns features by assigning corresponding inputs and outputs. In recent years, CNNs have exhibited promising performance in the 3D reconstruction of close-range scenes. In this paper, we present a CNN-based algorithm that is capable of generating DEMs from single images; the DEMs have the same resolutions as the input images. An existing low-resolution DEM is used to provide global information. Synthetic and real data, including context camera (CTX) images and DEMs from stereo High-Resolution Imaging Science Experiment (HiRISE) images, are used as training data. The performance of the proposed method is evaluated using single CTX images of representative landforms on Mars, and the generated DEMs are compared with those obtained from stereo HiRISE images. The experimental results show promising performance of the proposed method. The topographic details are well reconstructed, and the geometric accuracies achieve root-mean-square error (RMSE) values ranging from 2.1 m to 12.2 m (approximately 0.5 to 2 pixels in the image space). The experimental results show that the proposed CNN-based method has great potential for 3D surface reconstruction in planetary applications.

scholarly journals DEEP LEARNING FOR 3D RECONSTRUCTION OF THE MARTIAN SURFACE USING MONOCULAR IMAGES: A FIRST GLANCE

2020 ◽  
Vol XLIII-B3-2020 ◽  
pp. 1111-1116
Author(s):  
Z. Chen ◽  
B. Wu ◽  
W. C. Liu
Keyword(s):  
Deep Learning ◽  
3D Reconstruction ◽  
Experimental Results ◽  
Training Data ◽  
Network System ◽  
Stereo Images ◽  
Martian Surface ◽  
Preliminary Results ◽  
Low Level ◽  
Traditional Technology

Abstract. The paper presents our efforts on CNN-based 3D reconstruction of the Martian surface using monocular images. The Viking colorized global mosaic and Mar Express HRSC blended DEM are used as training data. An encoder-decoder network system is employed in the framework. The encoder section extracts features from the images, which includes convolution layers and reduction layers. The decoder section consists of deconvolution layers and is to integrate features and convert the images to desired DEMs. In addition, skip connection between encoder and decoder section is applied, which offers more low-level features for the decoder section to improve its performance. Monocular Context Camera (CTX) images are used to test and verify the performance of the proposed CNN-based approach. Experimental results show promising performances of the proposed approach. Features in images are well utilized, and topographical details in images are successfully recovered in the DEMs. In most cases, the geometric accuracies of the generated DEMs are comparable to those generated by the traditional technology of photogrammetry using stereo images. The preliminary results show that the proposed CNN-based approach has great potential for 3D reconstruction of the Martian surface.

scholarly journals MADNet 2.0: Pixel-Scale Topography Retrieval from Single-View Orbital Imagery of Mars Using Deep Learning

2021 ◽  
Vol 13 (21) ◽  
pp. 4220
Author(s):  
Yu Tao ◽  
Jan-Peter Muller ◽  
Siting Xiong ◽  
Susan J. Conway
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Spatial Resolution ◽  
Large Scale ◽  
Fine Scale ◽  
Martian Surface ◽  
Multi Scale ◽  
Single Scale ◽  
Imaging Science ◽  
Pixel Scale

The High-Resolution Imaging Science Experiment (HiRISE) onboard the Mars Reconnaissance Orbiter provides remotely sensed imagery at the highest spatial resolution at 25–50 cm/pixel of the surface of Mars. However, due to the spatial resolution being so high, the total area covered by HiRISE targeted stereo acquisitions is very limited. This results in a lack of the availability of high-resolution digital terrain models (DTMs) which are better than 1 m/pixel. Such high-resolution DTMs have always been considered desirable for the international community of planetary scientists to carry out fine-scale geological analysis of the Martian surface. Recently, new deep learning-based techniques that are able to retrieve DTMs from single optical orbital imagery have been developed and applied to single HiRISE observational data. In this paper, we improve upon a previously developed single-image DTM estimation system called MADNet (1.0). We propose optimisations which we collectively call MADNet 2.0, which is based on a supervised image-to-height estimation network, multi-scale DTM reconstruction, and 3D co-alignment processes. In particular, we employ optimised single-scale inference and multi-scale reconstruction (in MADNet 2.0), instead of multi-scale inference and single-scale reconstruction (in MADNet 1.0), to produce more accurate large-scale topographic retrieval with boosted fine-scale resolution. We demonstrate the improvements of the MADNet 2.0 DTMs produced using HiRISE images, in comparison to the MADNet 1.0 DTMs and the published Planetary Data System (PDS) DTMs over the ExoMars Rosalind Franklin rover’s landing site at Oxia Planum. Qualitative and quantitative assessments suggest the proposed MADNet 2.0 system is capable of producing pixel-scale DTM retrieval at the same spatial resolution (25 cm/pixel) of the input HiRISE images.

scholarly journals From 2D Images to 3D Tangible Models: Autostereoscopic and Haptic Visualization of Martian Rocks in Virtual Environments

2007 ◽  
Vol 16 (1) ◽  
pp. 1-15 ◽  
Author(s):  
Cagatay Basdogan
Keyword(s):  
3D Reconstruction ◽  
3D Models ◽  
Rock Surface ◽  
Surface Model ◽  
Stereo Images ◽  
Martian Surface ◽  
Reconstruction Process ◽  
Low Transmission ◽  
2D Images ◽  
Multimodal Visualization

A planetary rover acquires a large collection of images while exploring its surrounding environment. For example, 2D stereo images of the Martian surface captured by the lander and the Sojourner rover during the Mars Pathfinder mission in 1997 were transmitted to Earth for scientific analysis and navigation planning. Due to the limited memory and computational power of the Sojourner rover, most of the images were captured by the lander and then transmitted to Earth directly for processing. If these images were merged together at the rover site to reconstruct a 3D representation of the rover's environment using its on-board resources, more information could potentially be transmitted to Earth in a compact manner. However, construction of a 3D model from multiple views is a highly challenging task to accomplish even for the new generation rovers (Spirit and Opportunity) running on the Mars surface at the time this article was written. Moreover, low transmission rates and communication intervals between Earth and Mars make the transmission of any data more difficult. We propose a robust and computationally efficient method for progressive transmission of multi-resolution 3D models of Martian rocks and soil reconstructed from a series of stereo images. For visualization of these models on Earth, we have developed a new multimodal visualization setup that integrates vision and touch. Our scheme for 3D reconstruction of Martian rocks from 2D images for visualization on Earth involves four main steps: a) acquisition of scans: depth maps are generated from stereo images, b) integration of scans: the scans are correctly positioned and oriented with respect to each other and fused to construct a 3D volumetric representation of the rocks using an octree, c) transmission: the volumetric data is encoded and progressively transmitted to Earth, d) visualization: a surface model is reconstructed from the transmitted data on Earth and displayed to a user through a new autostereoscopic visualization table and a haptic device for providing touch feedback. To test the practical utility of our approach, we first captured a sequence of stereo images of a rock surface from various viewpoints in JPL MarsYard using a mobile cart and then performed a series of 3D reconstruction experiments. In this paper, we discuss the steps of our reconstruction process, our multimodal visualization system, and the tradeoffs that have to be made to transmit multiresolution 3D models to Earth in an efficient manner under the constraints of limited computational resources, low transmission rate, and communication interval between Earth and Mars.

scholarly journals IDENTIFYING SURFACE CHANGES ON HRSC IMAGES OF THE MARS SOUTH POLAR RESIDUAL CAP (SPRC)

2016 ◽  
Vol XLI-B4 ◽  
pp. 463-469 ◽  
Author(s):  
Alfiah Rizky Diana Putri ◽  
Panagiotis Sidiropoulos ◽  
Jan-Peter Muller
Keyword(s):  
High Resolution ◽  
Seasonal Changes ◽  
Polar Region ◽  
European Space Agency ◽  
Polar Regions ◽  
Martian Surface ◽  
Mars Express ◽  
Imaging Science ◽  
South Polar ◽  
Surface Changes

The surface of Mars has been an object of interest for planetary research since the launch of Mariner 4 in 1964. Since then different cameras such as the Viking Visual Imaging Subsystem (VIS), Mars Global Surveyor (MGS) Mars Orbiter Camera (MOC), and Mars Reconnaissance Orbiter (MRO) Context Camera (CTX) and High Resolution Imaging Science Experiment (HiRISE) have been imaging its surface at ever higher resolution. The High Resolution Stereo Camera (HRSC) on board of the European Space Agency (ESA) Mars Express, has been imaging the Martian surface, since 25th December 2003 until the present-day. HRSC has covered 100&thinsp;% of the surface of Mars, about 70&thinsp;% of the surface with panchromatic images at 10-20&thinsp;m/pixel, and about 98&thinsp;% at better than 100&thinsp;m/pixel (Neukum et. al., 2004), including the polar regions of Mars. The Mars polar regions have been studied intensively recently by analysing images taken by the Mars Express and MRO missions (Plaut et al., 2007). <br><br> The South Polar Residual Cap (SPRC) does not change very much in volume overall but there are numerous examples of dynamic phenomena associated with seasonal changes in the atmosphere. In particular, we can examine the time variation of layers of solid carbon dioxide and water ice with dust deposition (Bibring, 2004), spider-like channels (Piqueux et al., 2003) and so-called Swiss Cheese Terrain (Titus et al., 2004). Because of seasonal changes each Martian year, due to the sublimation and deposition of water and CO<sub>2</sub> ice on the Martian south polar region, clearly identifiable surface changes occur in otherwise permanently icy region. In this research, good quality HRSC images of the Mars South Polar region are processed based on previous identification as the optimal coverage of clear surfaces (Campbell et al., 2015). HRSC images of the Martian South Pole are categorized in terms of quality, time, and location to find overlapping areas, processed into high quality Digital Terrain Models (DTMs) and Orthorectified Images (ORIs) and projected into polar stereographic projection using DLR (Deutsches Zentrum für Luft- und Raumfahrt; German Aerospace Center)’s VICAR and GIS software with modifications developed by Kim & Muller (2009). Surface changes are identified in the Mars SPRC region and analysed based on their appearance in the HRSC images.

scholarly journals Pheno4D: A spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0256340
Author(s):  
David Schunck ◽  
Federico Magistri ◽  
Radu Alexandru Rosu ◽  
André Cornelißen ◽  
Nived Chebrolu ◽  
...  
Keyword(s):  
High Resolution ◽  
3D Reconstruction ◽  
Point Clouds ◽  
Training Data ◽  
Plant Phenotyping ◽  
Growth Stages ◽  
Plant Analysis ◽  
Rigid Registration ◽  
3D Point Clouds ◽  
Spatio Temporal

Understanding the growth and development of individual plants is of central importance in modern agriculture, crop breeding, and crop science. To this end, using 3D data for plant analysis has gained attention over the last years. High-resolution point clouds offer the potential to derive a variety of plant traits, such as plant height, biomass, as well as the number and size of relevant plant organs. Periodically scanning the plants even allows for performing spatio-temporal growth analysis. However, highly accurate 3D point clouds from plants recorded at different growth stages are rare, and acquiring this kind of data is costly. Besides, advanced plant analysis methods from machine learning require annotated training data and thus generate intense manual labor before being able to perform an analysis. To address these issues, we present with this dataset paper a multi-temporal dataset featuring high-resolution registered point clouds of maize and tomato plants, which we manually labeled for computer vision tasks, such as for instance segmentation and 3D reconstruction, providing approximately 260 million labeled 3D points. To highlight the usability of the data and to provide baselines for other researchers, we show a variety of applications ranging from point cloud segmentation to non-rigid registration and surface reconstruction. We believe that our dataset will help to develop new algorithms to advance the research for plant phenotyping, 3D reconstruction, non-rigid registration, and deep learning on raw point clouds. The dataset is freely accessible at https://www.ipb.uni-bonn.de/data/pheno4d/.

scholarly journals IDENTIFYING SURFACE CHANGES ON HRSC IMAGES OF THE MARS SOUTH POLAR RESIDUAL CAP (SPRC)

2016 ◽  
Vol XLI-B4 ◽  
pp. 463-469
Author(s):  
Alfiah Rizky Diana Putri ◽  
Panagiotis Sidiropoulos ◽  
Jan-Peter Muller
Keyword(s):  
High Resolution ◽  
European Space Agency ◽  
Martian Surface ◽  
Mars Global Surveyor ◽  
Space Agency ◽  
Imaging Science ◽  
Resolution Imaging ◽  
Visual Imaging ◽  
Better Than ◽  
Surface Changes

The surface of Mars has been an object of interest for planetary research since the launch of Mariner 4 in 1964. Since then different cameras such as the Viking Visual Imaging Subsystem (VIS), Mars Global Surveyor (MGS) Mars Orbiter Camera (MOC), and Mars Reconnaissance Orbiter (MRO) Context Camera (CTX) and High Resolution Imaging Science Experiment (HiRISE) have been imaging its surface at ever higher resolution. The High Resolution Stereo Camera (HRSC) on board of the European Space Agency (ESA) Mars Express, has been imaging the Martian surface, since 25th December 2003 until the present-day. HRSC has covered 100&thinsp;% of the surface of Mars, about 70&thinsp;% of the surface with panchromatic images at 10-20&thinsp;m/pixel, and about 98&thinsp;% at better than 100&thinsp;

scholarly journals Assessing Riyadh’s Urban Change Utilizing High-Resolution Imagery

Land ◽  
2019 ◽  
Vol 8 (12) ◽  
pp. 193
Author(s):  
Ali Alghamdi ◽  
Anthony R. Cummings
Keyword(s):  
Maximum Likelihood ◽  
High Resolution ◽  
Accuracy Assessment ◽  
Middle Eastern ◽  
Object Oriented ◽  
Training Data ◽  
High Resolution Imagery ◽  
Significant Research ◽  
Desert Areas ◽  
Spot 5

The implications of change on local processes have attracted significant research interest in recent times. In urban settings, green spaces and forests have attracted much attention. Here, we present an assessment of change within the predominantly desert Middle Eastern city of Riyadh, an understudied setting. We utilized high-resolution SPOT 5 data and two classification techniques—maximum likelihood classification and object-oriented classification—to study the changes in Riyadh between 2004 and 2014. Imagery classification was completed with training data obtained from the SPOT 5 dataset, and an accuracy assessment was completed through a combination of field surveys and an application developed in ESRI Survey 123 tool. The Survey 123 tool allowed residents of Riyadh to present their views on land cover for the 2004 and 2014 imagery. Our analysis showed that soil or ‘desert’ areas were converted to roads and buildings to accommodate for Riyadh’s rapidly growing population. The object-oriented classifier provided higher overall accuracy than the maximum likelihood classifier (74.71% and 73.79% vs. 92.36% and 90.77% for 2004 and 2014). Our work provides insights into the changes within a desert environment and establishes a foundation for understanding change in this understudied setting.

scholarly journals High‐resolution MRI of the human palatine tonsil and its schematic anatomic 3D reconstruction

2021 ◽  
Author(s):  
Karl‐Heinz Herrmann ◽  
Franziska Hoffmann ◽  
Günther Ernst ◽  
David Pertzborn ◽  
Daniela Pelzel ◽  
...  
Keyword(s):  
High Resolution ◽  
3D Reconstruction ◽  
Palatine Tonsil ◽  
High Resolution Mri

scholarly journals Data-Driven Intelligent 3D Surface Measurement in Smart Manufacturing: Review and Outlook

Machines ◽  
2021 ◽  
Vol 9 (1) ◽  
pp. 13
Author(s):  
Yuhang Yang ◽  
Zhiqiao Dong ◽  
Yuquan Meng ◽  
Chenhui Shao
Keyword(s):  
High Resolution ◽  
High Performance ◽  
Sampling Design ◽  
Three Dimensional ◽  
Measurement Data ◽  
Data Driven ◽  
Surface Measurement ◽  
Smart Manufacturing ◽  
Future Research ◽  
3D Surface

High-fidelity characterization and effective monitoring of spatial and spatiotemporal processes are crucial for high-performance quality control of many manufacturing processes and systems in the era of smart manufacturing. Although the recent development in measurement technologies has made it possible to acquire high-resolution three-dimensional (3D) surface measurement data, it is generally expensive and time-consuming to use such technologies in real-world production settings. Data-driven approaches that stem from statistics and machine learning can potentially enable intelligent, cost-effective surface measurement and thus allow manufacturers to use high-resolution surface data for better decision-making without introducing substantial production cost induced by data acquisition. Among these methods, spatial and spatiotemporal interpolation techniques can draw inferences about unmeasured locations on a surface using the measurement of other locations, thus decreasing the measurement cost and time. However, interpolation methods are very sensitive to the availability of measurement data, and their performances largely depend on the measurement scheme or the sampling design, i.e., how to allocate measurement efforts. As such, sampling design is considered to be another important field that enables intelligent surface measurement. This paper reviews and summarizes the state-of-the-art research in interpolation and sampling design for surface measurement in varied manufacturing applications. Research gaps and future research directions are also identified and can serve as a fundamental guideline to industrial practitioners and researchers for future studies in these areas.

Sign in / Sign up

Export Citation Format

Share Document