Configurable 3D Scene Synthesis and 2D Image Rendering with Per-pixel Ground Truth Using Stochastic Grammars

Abstract. Stereo dense matching is a fundamental task for 3D scene reconstruction. Recently, deep learning based methods have proven effective on some benchmark datasets, for example Middlebury and KITTI stereo. However, it is not easy to find a training dataset for aerial photogrammetry. Generating ground truth data for real scenes is a challenging task. In the photogrammetry community, many evaluation methods use digital surface models (DSM) to generate the ground truth disparity for the stereo pairs, but in this case interpolation may bring errors in the estimated disparity. In this paper, we publish a stereo dense matching dataset based on ISPRS Vaihingen dataset, and use it to evaluate some traditional and deep learning based methods. The evaluation shows that learning-based methods outperform traditional methods significantly when the fine tuning is done on a similar landscape. The benchmark also investigates the impact of the base to height ratio on the performance of the evaluated methods. The dataset can be found in https://github.com/whuwuteng/benchmark_ISPRS2021.

Download Full-text

SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self

10.22541/au.163335790.03073492/v1 ◽

2021 ◽

Author(s):

Guangming Wang ◽

Chaokang Jiang ◽

Zehang Shen ◽

Yanzi Miao ◽

Hesheng Wang

Keyword(s):

Point Cloud ◽

Ground Truth ◽

Autonomous Driving ◽

Generative Adversarial Networks ◽

Real Point ◽

Current Frame ◽

3D Motion ◽

3D Space ◽

Scene Flow ◽

3D Scene

3D scene flow presents the 3D motion of each point in the 3D space, which forms the fundamental 3D motion perception for autonomous driving and server robots. Although the RGBD camera or LiDAR capture discrete 3D points in space, the objects and motions usually are continuous in the macro world. That is, the objects keep themselves consistent as they flow from the current frame to the next frame. Based on this insight, the Generative Adversarial Networks (GAN) is utilized to self-learn 3D scene flow with no need for ground truth. The fake point cloud of the second frame is synthesized from the predicted scene flow and the point cloud of the first frame. The adversarial training of the generator and discriminator is realized through synthesizing indistinguishable fake point cloud and discriminating the real point cloud and the synthesized fake point cloud. The experiments on KITTI scene flow dataset show that our method realizes promising results without ground truth. Just like a human observing a real-world scene, the proposed approach is capable of determining the consistency of the scene at different moments in spite of the exact flow value of each point is unknown in advance. Corresponding author(s) Email: [email protected]

Download Full-text

CLASSIFICATION OF MLS POINT CLOUDS IN URBAN SCENES USING DETRENDED GEOMETRIC FEATURES FROM SUPERVOXEL-BASED LOCAL CONTEXTS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-2-271-2018 ◽

2018 ◽

Vol IV-2 ◽

pp. 271-278 ◽

Cited By ~ 6

Author(s):

Z. Sun ◽

Y. Xu ◽

L. Hoegner ◽

U. Stilla

Keyword(s):

Feature Extraction ◽

Ground Truth ◽

Point Clouds ◽

Local Context ◽

Geometric Feature ◽

Geometric Features ◽

Robust Solution ◽

Urban Scenes ◽

Feature Extraction Method ◽

3D Scene

In this work, we propose a classification method designed for the labeling of MLS point clouds, with detrended geometric features extracted from the points of the supervoxel-based local context. To achieve the analysis of complex 3D urban scenes, acquired points of the scene should be tagged with individual labels of different classes. Thus, assigning a unique label to the points of an object that belong to the same category plays an essential role in the entire 3D scene analysis workflow. Although plenty of studies in this field have been reported, this work is still a challenging task. Specifically, in this work: 1) A novel geometric feature extraction method, detrending the redundant and in-salient information in the local context, is proposed, which is proved to be effective for extracting local geometric features from the 3D scene. 2) Instead of using individual point as basic element, the supervoxel-based local context is designed to encapsulate geometric characteristics of points, providing a flexible and robust solution for feature extraction. 3) Experiments using complex urban scene with manually labeled ground truth are conducted, and the performance of proposed method with respect to different methods is analyzed. With the testing dataset, we have obtained a result of 0.92 for overall accuracy for assigning eight semantic classes.

Download Full-text

A SYNTHETIC 3D SCENE FOR THE VALIDATION OF PHOTOGRAMMETRIC ALGORITHMS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w13-1221-2019 ◽

2019 ◽

Vol XLII-2/W13 ◽

pp. 1221-1228

Author(s):

D. Frommholz

Keyword(s):

Optical Sensors ◽

Measurement Errors ◽

Ground Truth ◽

Point Clouds ◽

Visual Assessment ◽

Double Precision ◽

Data Set ◽

Global Matching ◽

3D Scene ◽

Programming Knowledge

<p><strong>Abstract.</strong> This paper describes the construction and composition of a synthetic test world for the validation of photogrammetric algorithms. Since its 3D objects are entirely generated by software, the geometric accuracy of the scene does not suffer from measurement errors which existing real-world ground truth is inherently afflicted with. The resulting data set covers an area of 13188 by 6144 length units and exposes positional residuals as small as the machine epsilon of the double-precision floating point numbers used exclusively for the coordinates. It is colored with high-resolution textures to accommodate the simulation of virtual flight campaigns with large optical sensors and laser scanners in both aerial and close-range scenarios. To specifically support the derivation of image samples and point clouds, the synthetic scene gets stored in the human-readable Alias/Wavefront OBJ and POV-Ray data formats. While conventional rasterization remains possible, using the open-source ray tracer as a render tool facilitates the creation of ideal pinhole bitmaps, consistent digital surface models (DSMs), true ortho-mosaics (TOMs) and orientation metadata without programming knowledge. To demonstrate the application of the constructed 3D scene, example validation recipes are discussed in detail for a state-of-the-art implementation of semi-global matching and a perspective-correct multi-source texture mapper. For the latter, beyond the visual assessment, a statistical evaluation of the achieved texture quality is given.</p>

Download Full-text

A Comparison of Simple Structure Rotation Criteria in Temporal Exploratory Factor Analysis for Event-Related Potential Data

Methodology ◽

10.1027/1614-2241/a000175 ◽

2019 ◽

Vol 15 (Supplement 1) ◽

pp. 43-60 ◽

Cited By ~ 3

Author(s):

Florian Scharf ◽

Steffen Nestler

Keyword(s):

Factor Analysis ◽

Exploratory Factor Analysis ◽

Simple Structure ◽

Ground Truth ◽

Event Related Potential ◽

Suitable Alternative ◽

Factor Rotation ◽

Rotation Methods ◽

Simple Structure Rotation ◽

Temporal Overlap

Abstract. It is challenging to apply exploratory factor analysis (EFA) to event-related potential (ERP) data because such data are characterized by substantial temporal overlap (i.e., large cross-loadings) between the factors, and, because researchers are typically interested in the results of subsequent analyses (e.g., experimental condition effects on the level of the factor scores). In this context, relatively small deviations in the estimated factor solution from the unknown ground truth may result in substantially biased estimates of condition effects (rotation bias). Thus, in order to apply EFA to ERP data researchers need rotation methods that are able to both recover perfect simple structure where it exists and to tolerate substantial cross-loadings between the factors where appropriate. We had two aims in the present paper. First, to extend previous research, we wanted to better understand the behavior of the rotation bias for typical ERP data. To this end, we compared the performance of a variety of factor rotation methods under conditions of varying amounts of temporal overlap between the factors. Second, we wanted to investigate whether the recently proposed component loss rotation is better able to decrease the bias than traditional simple structure rotation. The results showed that no single rotation method was generally superior across all conditions. Component loss rotation showed the best all-round performance across the investigated conditions. We conclude that Component loss rotation is a suitable alternative to simple structure rotation. We discuss this result in the light of recently proposed sparse factor analysis approaches.

Download Full-text

Brain FDG PET for the Etiological Diagnosis of Clinically Uncertain Cognitive Impairment During Delirium in Remission

Journal of Alzheimer s Disease ◽

10.3233/jad-200530 ◽

2020 ◽

Vol 77 (4) ◽

pp. 1609-1622

Author(s):

Franziska Mathies ◽

Catharina Lange ◽

Anja Mäurer ◽

Ivayla Apostolova ◽

Susanne Klutmann ◽

...

Keyword(s):

Cognitive Impairment ◽

Fdg Pet ◽

False Negative ◽

Ground Truth ◽

Etiological Diagnosis ◽

Geriatric Unit ◽

Positron Emission ◽

The Brain ◽

Hospitalized Geriatric Patients

Background: Positron emission tomography (PET) of the brain with 2-[F-18]-fluoro-2-deoxy-D-glucose (FDG) is widely used for the etiological diagnosis of clinically uncertain cognitive impairment (CUCI). Acute full-blown delirium can cause reversible alterations of FDG uptake that mimic neurodegenerative disease. Objective: This study tested whether delirium in remission affects the performance of FDG PET for differentiation between neurodegenerative and non-neurodegenerative etiology of CUCI. Methods: The study included 88 patients (82.0±5.7 y) with newly detected CUCI during hospitalization in a geriatric unit. Twenty-seven (31%) of the patients were diagnosed with delirium during their current hospital stay, which, however, at time of enrollment was in remission so that delirium was not considered the primary cause of the CUCI. Cases were categorized as neurodegenerative or non-neurodegenerative etiology based on visual inspection of FDG PET. The diagnosis at clinical follow-up after ≥12 months served as ground truth to evaluate the diagnostic performance of FDG PET. Results: FDG PET was categorized as neurodegenerative in 51 (58%) of the patients. Follow-up after 16±3 months was obtained in 68 (77%) of the patients. The clinical follow-up diagnosis confirmed the FDG PET-based categorization in 60 patients (88%, 4 false negative and 4 false positive cases with respect to detection of neurodegeneration). The fraction of correct PET-based categorization did not differ between patients with delirium in remission and patients without delirium (86% versus 89%, p = 0.666). Conclusion: Brain FDG PET is useful for the etiological diagnosis of CUCI in hospitalized geriatric patients, as well as in patients with delirium in remission.

Download Full-text

Construction of Subjective Vehicle Detection Evaluation Model Considering Shift from Ground Truth Position

IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences ◽

10.1587/transfun.e102.a.1246 ◽

2019 ◽

Vol E102.A (9) ◽

pp. 1246-1249 ◽

Cited By ~ 1

Author(s):

Naho ITO ◽

Most Shelina AKTAR ◽

Yuukou HORITA

Keyword(s):

Evaluation Model ◽

Vehicle Detection ◽

Ground Truth ◽

Detection Evaluation

Download Full-text

Improved 3D Scene Modeling for Image Registration in Change Detection

Electronic Imaging ◽

10.2352/issn.2470-1173.2019.7.iriacv-465 ◽

2019 ◽

Vol 2019 (7) ◽

pp. 465-1-465-7

Author(s):

Sjors van Riel ◽

Dennis van de Wouw ◽

Peter de With

Keyword(s):

Image Registration ◽

Change Detection ◽

Scene Modeling ◽

3D Scene

Download Full-text

A Flying Gray Ball Multi-illuminant Image Dataset for Color Research

Journal of Imaging Science and Technology ◽

10.2352/j.imagingsci.technol.2020.64.5.050411 ◽

2020 ◽

Vol 64 (5) ◽

pp. 50411-1-50411-8

Author(s):

Hoda Aghaei ◽

Brian Funt

Keyword(s):

Machine Vision ◽

Spectral Reflectance ◽

Color Constancy ◽

Ground Truth ◽

Practical Approach ◽

Post Processing ◽

Illumination Estimation ◽

Processing Step ◽

Image Dataset

Abstract For research in the field of illumination estimation and color constancy, there is a need for ground-truth measurement of the illumination color at many locations within multi-illuminant scenes. A practical approach to obtaining such ground-truth illumination data is presented here. The proposed method involves using a drone to carry a gray ball of known percent surface spectral reflectance throughout a scene while photographing it frequently during the flight using a calibrated camera. The captured images are then post-processed. In the post-processing step, machine vision techniques are used to detect the gray ball within each frame. The camera RGB of light reflected from the gray ball provides a measure of the illumination color at that location. In total, the dataset contains 30 scenes with 100 illumination measurements on average per scene. The dataset is available for download free of charge.

Download Full-text