3D Reconstruction through Fusion of Cross-View Images

3D recovery from multi-stereo and stereo images, as an important application of the image-based perspective geometry, serves many applications in computer vision, remote sensing, and Geomatics. In this chapter, the authors utilize the imaging geometry and present approaches that perform 3D reconstruction from cross-view images that are drastically different in their viewpoints. We introduce our project work that takes ground-view images and satellite images for full 3D recovery, which includes necessary methods in satellite and ground-based point cloud generation from images, 3D data co-registration, fusion, and mesh generation. We demonstrate our proposed framework on a dataset consisting of twelve satellite images and 150 k video frames acquired through a vehicle-mounted Go-pro camera and demonstrate the reconstruction results. We have also compared our results with results generated from an intuitive processing pipeline that involves typical geo-registration and meshing methods.

Keep It Real: a Window to Real Reality in Virtual Reality

This paper proposed a new interaction paradigm in the virtual reality (VR) environments, which consists of a virtual mirror or window projected onto a virtual surface, representing the correct perspective geometry of a mirror or window reflecting the real world. This technique can be applied to various videos, live streaming apps, augmented and virtual reality settings to provide an interactive and immersive user experience. To support such a perspective-accurate representation, we implemented computer vision algorithms for feature detection and correspondence matching. To constrain the solutions, we incorporated an automatically tuning scaling factor upon the homography transform matrix such that each image frame follows a smooth transition with the user in sight. The system is a real-time rendering framework where users can engage their real-life presence with the virtual space.

Anamorphic Atmospheres

The principles of linear perspective geometry were applied to both the representation and the form of the Renaissance city to reflect the collective proprietorial ambitions of church and state. Anamorphosis was developed by intellectual dissidents as a drawing mechanism and as a counter to the previous representational constraints imposed by linear perspective. The contemporary city image relies upon on an array of pixels mediated by technology to foster existing relationships between power and place. The paper argues that digital technologies initiate anamorphic viewing conditions that correspond to previous attempts to destabilise the covert ambitions of linear perspective. By presenting digital anamorphic representations of contemporary urban space, it shows how the temporal nature of the image and the pixel-based geometry of the digital array not only contest the promotional city view but multiply the opportunity to understand previously unexplored qualitative, atmospheric properties of urban space.

Geometry-Constrained Car Recognition Using a 3D Perspective Network

We present a novel learning framework for vehicle recognition from a single RGB image. Unlike existing methods which only use attention mechanisms to locate 2D discriminative information, our work learns a novel 3D perspective feature representation of a vehicle, which is then fused with 2D appearance feature to predict the category. The framework is composed of a global network (GN), a 3D perspective network (3DPN), and a fusion network. The GN is used to locate the region of interest (RoI) and generate the 2D global feature. With the assistance of the RoI, the 3DPN estimates the 3D bounding box under the guidance of the proposed vanishing point loss, which provides a perspective geometry constraint. Then the proposed 3D representation is generated by eliminating the viewpoint variance of the 3D bounding box using perspective transformation. Finally, the 3D and 2D feature are fused to predict the category of the vehicle. We present qualitative and quantitative results on the vehicle classification and verification tasks in the BoxCars dataset. The results demonstrate that, by learning such a concise 3D representation, we can achieve superior performance to methods that only use 2D information while retain 3D meaningful information without the challenge of requiring a 3D CAD model.

Automatic Registration of Footsteps in Contact Regions for Reactive Agility Training in Sports

In collective sports, reactive agility training methodologies allow to evaluate and improve the player performance, being able to consider a mixture of technical, tactical, physical, and psychological abilities, similarly to real game-play situations. In this article, we present a new methodology for reactive agility training (neural training), the technological setup for the methodology, and a new footstep tracking algorithm, as the key element for automating the speed data gathering process, necessary for obtaining the relevant variables of the neural training approach. This new methodology is oriented to accurately measure two of the most relevant variables for reactive agility training: total response time (sprint time) and response correctness, related to a stimuli sequence presented to a player. The stimuli were designed to properly represent realistic competitive conditions for player training, contextualized to soccer. In order to automate the gathering process, a new computer vision based automatic footstep detection algorithm has been integrated to the system. The algorithm combines Kalman Filters, segmentation techniques, and perspective geometry, for obtaining highly precise detections of the moment a relevant footstep occurs in real-time, reaching a precision higher than 97%. Plus, the algorithm does not require any special marker, invasive sensor, or clothing constraint on the player.

Single-Camera Trilateration

This paper presents a single-camera trilateration scheme which estimates the instantaneous 3D pose of a regular forward-looking camera from a single image of landmarks at known positions. Derived on the basis of the classical pinhole camera model and principles of perspective geometry, the proposed algorithm estimates the camera position and orientation successively. It provides a convenient self-localization tool for mobile robots and vehicles equipped with onboard cameras. Performance analysis has been conducted through extensive simulations with representative examples, which provides an insight into how the input errors and the geometric arrangement of the camera and landmarks affect the performance of the proposed algorithm. The effectiveness of the proposed algorithm has been further verified through an experiment.

Relations between Arithmetic and Geometry in Piero della Francesca’s Libellus de quinque corporibus regularibus (c.1412 - 1492)

This work aim to analyse relations between Arithmetic and Geometry indicated by Piero della Francesca (c.1412 - 1492) in his treatise Libellus de quinque corporibus regularibus. Piero della Francesca was a painter and scholar of perspective, geometry and arithmetic, in his time. He carried out investigations on pictorial, geometric and architectural issues. Of the treatises he wrote, only three are preserved, on perspective (De Prospectiva Pingendi), Geometry (Libellus de Quinque Corporibus Regullaribus) and Arithmetic (Trattado d’Abaco). The central document selected for this research was the manuscript Libellus de quinque corporibus regularibus, deposited in the Vatican Apostolic Library in digitized copy. Throughout this work we have tried to understand the Libellus de quinque corporibus regularibus as a result of the work of an artist and scholar who proposed ways of relating Arithmetic and Geometry using both his practical knowledge and the works of authors of antiquity.

USE OF VERTICAL AERIAL IMAGES FOR SEMI-OBLIQUE MAPPING

The paper proposes a methodology for the use of the oblique sections of images from large-format photogrammetric cameras, by exploiting the effect of the central perspective geometry in the lateral parts of the nadir images (“semi-oblique” images). The point of origin of the investigation was the execution of a photogrammetric flight over Norcia (Italy), which was seriously damaged after the earthquake of 30/10/2016. Contrary to the original plan of oblique acquisitions, the flight was executed on 15/11/2017 using an UltraCam Eagle camera with focal length 80 mm, and combining two flight plans, rotated by 90º (“crisscross” flight). The images (GSD 5 cm) were used to extract a 2.5D DSM cloud, sampled to a XY-grid size of 2 GSD, a 3D point clouds with a mean spatial resolution of 1 GSD and a 3D mesh model at a resolution of 10 cm of the historic centre of Norcia for a quantitative assessment of the damages. From the acquired nadir images the “semi-oblique” images (forward, backward, left and right views) could be extracted and processed in a modified version of GEOBLY software for measurements and restitution purposes. The potential of such semi-oblique image acquisitions from nadir-view cameras is hereafter shown and commented.

The Vault of Perception: Are Straight Lines Seen as Curved?

There is a widespread belief in art praxis that linear perspective is only a geometric approximation to the ‘true’ properties of perspective as experienced in the perception of the world, which are thought to involve some form of curvilinear perspective. The origins of that belief are examined from Roman times to the present, with a focus on the generation of perspective curvature by the active viewer as a means of elucidating the underlying perceptual principles involved. It is concluded that the only valid form of perspective for the flat canvas is linear perspective, and that it is valid only for a viewing location at the geometric center of projection for which the picture was constructed. Viewing from any other location (particularly in the case of wide-field images viewed from greater than the required distance) generates perceived distortions that have often been misinterpreted to imply that linear perspective geometry is inadequate and that some form of curvilinear perspective would be more representative. However, as long as it is viewed with one eye from the center of projection, the perceptual experience of accurate linear perspective is of a full, explorable 3D space, in contrast to any other form of perspective convention.