Influence of Pitch Angle Errors in 3D Scene Reconstruction Based on U-V Disparity: A Sensitivity Study

U-V disparity is a technique that is commonly used to detect obstacles in 3D scenes, modeling them as a set of vertical planes. In this paper, the authors describe the general lines of a method based on this technique for fully reconstructing 3D scenes, and conduct an analytical study of its performance and sensitivity to errors in the pitch angle of the stereoscopic vision system. The equations of the planes calculated for a given error in this angle yield the deviation with respect to the ideal planes (with a zero error in the angle) for a large test set consisting of planes with different orientations, which is represented graphically to analyze the method’s qualitative and quantitative performance. The relationship between the deviation of the planes and the error in the pitch angle is observed to be linear. Two major conclusions are drawn from this study: first, that the deviation between the calculated and ideal planes is always less than or equal to the error considered in the pitch angle; and second, that even though in some cases the deviation of the plane is zero or very small, the probability that a plane of the scene deviates from the ideal by the greatest amount possible, which matches the error in the pitch angle, is very high.

INVESTIGATION OF POINTNET FOR SEMANTIC SEGMENTATION OF LARGE-SCALE OUTDOOR POINT CLOUDS

Semantic segmentation of point clouds is indispensable for 3D scene understanding. Point clouds have credibility for capturing geometry of objects including shape, size, and orientation. Deep learning (DL) has been recognized as the most successful approach for image semantic segmentation. Applied to point clouds, performance of the many DL algorithms degrades, because point clouds are often sparse and have irregular data format. As a result, point clouds are regularly first transformed into voxel grids or image collections. PointNet was the first promising algorithm that feeds point clouds directly into the DL architecture. Although PointNet achieved remarkable performance on indoor point clouds, its performance has not been extensively studied in large-scale outdoor point clouds. So far, we know, no study on large-scale aerial point clouds investigates the sensitivity of the hyper-parameters used in the PointNet. This paper evaluates PointNet’s performance for semantic segmentation through three large-scale Airborne Laser Scanning (ALS) point clouds of urban environments. Reported results show that PointNet has potential in large-scale outdoor scene semantic segmentation. A remarkable limitation of PointNet is that it does not consider local structure induced by the metric space made by its local neighbors. Experiments exhibit PointNet is expressively sensitive to the hyper-parameters like batch-size, block partition and the number of points in a block. For an ALS dataset, we get significant difference between overall accuracies of 67.5% and 72.8%, for the block sizes of 5m × 5m and 10m × 10m, respectively. Results also discover that the performance of PointNet depends on the selection of input vectors.

Cluttering Reduction for Interactive Navigation and Visualization of Historical Images

Iconographic representations, such as historical photos of geographic spaces, are precious cultural heritage resources capable of describing a particular geographical area’s evolution over time. These photographic collections may vary in size, between hundreds and thousands of items. With the advent of the digital era, many of these documents have been digitized, spatialized, and are available online. Browsing through these digital image collections represents new challenges. This paper examines the topic of historical image exploration in a virtual environment enabling the co-visualization of historical photos into a contemporary 3D scene. We address the topic of user interaction considering the potential volume of the input data. Our methodology is based on design guidelines that rely on visual perception techniques to ease visual complexity and improve saliency on specific cues. The designs are additionally implemented following an image-based rendering approach and evaluated in a group of users. Overall, these propositions may be a notable addition to creating innovative ways to visualize and discover historical images in a virtual geographic environment.

A Layered Approach for Quality Assessment of DIBR-Synthesized Images

Multiview video plus depth (MVD) is a popular video format that supports three-dimensional television (3DTV) and free viewpoint television (FTV). 3DTV and FTV provide depth sensation to the viewer by presenting two views of the same scene but with slightly different angles. In MVD, few views are captured, and each view has the color image and the corresponding depth map which is used in depth image-based rendering (DIBR) to generate views at novel viewpoints. The DIBR can introduce various artifacts in the synthesized view resulting in poor quality. Therefore, evaluating the quality of the synthesized image is crucial to provide an appreciable quality of experience (QoE) to the viewer. In a 3D scene, objects are at a different distance from the camera, characterized by their depth. In this paper, we investigate the effect that objects at a different distance make on the overall QoE. In particular, we find that the quality of the closer objects contributes more to the overall quality as compared to the background objects. Based on this phenomenon, we propose a 3D quality assessment metric to evaluate the quality of the synthesized images. The proposed metric using the depth of the scene divides the image into different layers where each layer represents the objects at a different distance from the camera. The quality of each layer is individually computed, and their scores are pooled together to obtain a single quality score that represents the quality of the synthesized image. The performance of the proposed metric is evaluated on two benchmark DIBR image databases. The results show that the proposed metric is highly accurate and performs better than most existing 2D and 3D quality assessment algorithms.