scholarly journals A Deep Learning Architecture For 3D Mapping Urban Landscapes

2021 ◽  
Vol 11 (23) ◽  
pp. 11551
Author(s):  
Armando Levid Rodríguez-Santiago ◽  
José Aníbal Arias-Aguilar ◽  
Hiroshi Takemura ◽  
Alberto Elías Petrilli-Barceló
Keyword(s):  
Deep Learning ◽  
Network Architecture ◽  
Point Cloud ◽  
Three Dimensional ◽  
Aerial Image ◽  
Urban Landscapes ◽  
Feature Maps ◽  
Generative Adversarial Network ◽  
Outdoor Environments ◽  
2D Images

In this paper, an approach through a Deep Learning architecture for the three-dimensional reconstruction of outdoor environments in challenging terrain conditions is presented. The architecture proposed is configured as an Autoencoder. However, instead of the typical convolutional layers, some differences are proposed. The Encoder stage is set as a residual net with four residual blocks, which have been provided with the necessary knowledge to extract the feature maps from aerial images of outdoor environments. On the other hand, the Decoder stage is set as a Generative Adversarial Network (GAN) and called a GAN-Decoder. The proposed network architecture uses a sequence of the 2D aerial image as input. The Encoder stage works for the extraction of the vector of features that describe the input image, while the GAN-Decoder generates a point cloud based on the information obtained in the previous stage. By supplying a sequence of frames that a percentage of overlap between them, it is possible to determine the spatial location of each generated point. The experiments show that with this proposal it is possible to perform a 3D representation of an area flown over by a drone using the point cloud generated with a deep architecture that has a sequence of aerial 2D images as input. In comparison with other works, our proposed system is capable of performing three-dimensional reconstructions in challenging urban landscapes. Compared with the results obtained using commercial software, our proposal was able to generate reconstructions in less processing time, with less overlapping percentage between 2D images and is invariant to the type of flight path.

scholarly journals Deep-learning Reconstruction of Three-dimensional Galaxy Distributions with Intensity Mapping Observations

2021 ◽  
Vol 923 (1) ◽  
pp. L7
Author(s):  
Kana Moriwaki ◽  
Naoki Yoshida
Keyword(s):  
Deep Learning ◽  
Emission Line ◽  
Network Architecture ◽  
Large Scale ◽  
Three Dimensional ◽  
Line Emission ◽  
Correlation Coefficients ◽  
Emission Lines ◽  
Generative Adversarial Network ◽  
Intensity Mapping

Abstract Line-intensity mapping is emerging as a novel method that can measure the collective intensity fluctuations of atomic/molecular line emission from distant galaxies. Several observational programs with various wavelengths are ongoing and planned, but there remains a critical problem of line confusion; emission lines originating from galaxies at different redshifts are confused at the same observed wavelength. We devise a generative adversarial network that extracts designated emission-line signals from noisy three-dimensional data. Our novel network architecture allows two input data, in which the same underlying large-scale structure is traced by two emission lines of H α and [Oiii], so that the network learns the relative contributions at each wavelength and is trained to decompose the respective signals. After being trained with a large number of realistic mock catalogs, the network is able to reconstruct the three-dimensional distribution of emission-line galaxies at z = 1.3−2.4. Bright galaxies are identified with a precision of 84%, and the cross correlation coefficients between the true and reconstructed intensity maps are as high as 0.8. Our deep-learning method can be readily applied to data from planned spaceborne and ground-based experiments.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

Planar Segment Based Three-dimensional Point Cloud Registration in Outdoor Environments

2013 ◽  
Vol 30 (4) ◽  
pp. 552-582 ◽  
Author(s):  
Junhao Xiao ◽  
Benjamin Adler ◽  
Jianwei Zhang ◽  
Houxiang Zhang
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Point Cloud Registration ◽  
Outdoor Environments

scholarly journals PCCT: A POINT CLOUD CLASSIFICATION TOOL TO CREATE 3D TRAINING DATA TO ADJUST AND DEVELOP 3D CONVNET

2019 ◽  
Vol XLII-2/W16 ◽  
pp. 35-40
Author(s):  
E. Barnefske ◽  
H. Sternberg
Keyword(s):  
Network Architecture ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Classification Performance ◽  
Training Data ◽  
Process Step ◽  
Camera Systems ◽  
Classification Tool ◽  
Point Cloud Classification

<p><strong>Abstract.</strong> Point clouds give a very detailed and sometimes very accurate representation of the geometry of captured objects. In surveying, point clouds captured with laser scanners or camera systems are an intermediate result that must be processed further. Often the point cloud has to be divided into regions of similar types (object classes) for the next process steps. These classifications are very time-consuming and cost-intensive compared to acquisition. In order to automate this process step, conventional neural networks (ConvNet), which take over the classification task, are investigated in detail. In addition to the network architecture, the classification performance of a ConvNet depends on the training data with which the task is learned. This paper presents and evaluates the point clould classification tool (PCCT) developed at HCU Hamburg. With the PCCT, large point cloud collections can be semi-automatically classified. Furthermore, the influence of erroneous points in three-dimensional point clouds is investigated. The network architecture PointNet is used for this investigation.</p>

scholarly journals AN EFFICIENT DEEP LEARNING APPROACH FOR GROUND POINT FILTERING IN AERIAL LASER SCANNING POINT CLOUDS

2021 ◽  
Vol XLIII-B1-2021 ◽  
pp. 31-38
Author(s):  
A. Nurunnabi ◽  
F. N. Teferle ◽  
J. Li ◽  
R. C. Lindenbergh ◽  
A. Hunegnaw
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Laser Scanning ◽  
Three Dimensional ◽  
Ground Surface ◽  
Point Clouds ◽  
Urban Environments ◽  
Training Data ◽  
Ground Point ◽  
End To End

Abstract. Ground surface extraction is one of the classic tasks in airborne laser scanning (ALS) point cloud processing that is used for three-dimensional (3D) city modelling, infrastructure health monitoring, and disaster management. Many methods have been developed over the last three decades. Recently, Deep Learning (DL) has become the most dominant technique for 3D point cloud classification. DL methods used for classification can be categorized into end-to-end and non end-to-end approaches. One of the main challenges of using supervised DL approaches is getting a sufficient amount of training data. The main advantage of using a supervised non end-to-end approach is that it requires less training data. This paper introduces a novel local feature-based non end-to-end DL algorithm that generates a binary classifier for ground point filtering. It studies feature relevance, and investigates three models that are different combinations of features. This method is free from the limitations of point clouds’ irregular data structure and varying data density, which is the biggest challenge for using the elegant convolutional neural network. The new algorithm does not require transforming data into regular 3D voxel grids or any rasterization. The performance of the new method has been demonstrated through two ALS datasets covering urban environments. The method successfully labels ground and non-ground points in the presence of steep slopes and height discontinuity in the terrain. Experiments in this paper show that the algorithm achieves around 97% in both F1-score and model accuracy for ground point labelling.

scholarly journals Change Detection in Aerial Images Using Three-Dimensional Feature Maps

2020 ◽  
Vol 12 (9) ◽  
pp. 1404
Author(s):  
Saleh Javadi ◽  
Mattias Dahl ◽  
Mats I. Pettersson
Keyword(s):  
Change Detection ◽  
Sampling Rate ◽  
Three Dimensional ◽  
Image Resolution ◽  
Aerial Images ◽  
Aerial Image ◽  
Detection Methods ◽  
Surveillance Systems ◽  
Feature Maps ◽  
Aerial Imaging

Interest in aerial image analysis has increased owing to recent developments in and availability of aerial imaging technologies, like unmanned aerial vehicles (UAVs), as well as a growing need for autonomous surveillance systems. Variant illumination, intensity noise, and different viewpoints are among the main challenges to overcome in order to determine changes in aerial images. In this paper, we present a robust method for change detection in aerial images. To accomplish this, the method extracts three-dimensional (3D) features for segmentation of objects above a defined reference surface at each instant. The acquired 3D feature maps, with two measurements, are then used to determine changes in a scene over time. In addition, the important parameters that affect measurement, such as the camera’s sampling rate, image resolution, the height of the drone, and the pixel’s height information, are investigated through a mathematical model. To exhibit its applicability, the proposed method has been evaluated on aerial images of various real-world locations and the results are promising. The performance indicates the robustness of the method in addressing the problems of conventional change detection methods, such as intensity differences and shadows.

scholarly journals Connected-UNets: a deep learning architecture for breast mass segmentation

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Asma Baccouche ◽  
Begonya Garcia-Zapirain ◽  
Cristian Castillo Olea ◽  
Adel S. Elmaghraby
Keyword(s):  
Deep Learning ◽  
Network Architecture ◽  
Breast Imaging ◽  
Rapid Development ◽  
Synthetic Data ◽  
Screening Mammography ◽  
Medical Image Segmentation ◽  
Breast Mass ◽  
Generative Adversarial Network ◽  
Mass Segmentation

AbstractBreast cancer analysis implies that radiologists inspect mammograms to detect suspicious breast lesions and identify mass tumors. Artificial intelligence techniques offer automatic systems for breast mass segmentation to assist radiologists in their diagnosis. With the rapid development of deep learning and its application to medical imaging challenges, UNet and its variations is one of the state-of-the-art models for medical image segmentation that showed promising performance on mammography. In this paper, we propose an architecture, called Connected-UNets, which connects two UNets using additional modified skip connections. We integrate Atrous Spatial Pyramid Pooling (ASPP) in the two standard UNets to emphasize the contextual information within the encoder–decoder network architecture. We also apply the proposed architecture on the Attention UNet (AUNet) and the Residual UNet (ResUNet). We evaluated the proposed architectures on two publically available datasets, the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and additionally on a private dataset. Experiments were also conducted using additional synthetic data using the cycle-consistent Generative Adversarial Network (CycleGAN) model between two unpaired datasets to augment and enhance the images. Qualitative and quantitative results show that the proposed architecture can achieve better automatic mass segmentation with a high Dice score of 89.52%, 95.28%, and 95.88% and Intersection over Union (IoU) score of 80.02%, 91.03%, and 92.27%, respectively, on CBIS-DDSM, INbreast, and the private dataset.

scholarly journals Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring

2020 ◽  
Author(s):  
Malika Nisal Ratnayake ◽  
Adrian G Dyer ◽  
Alan Dorin
Keyword(s):  
Deep Learning ◽  
Natural Habitat ◽  
Hybrid Approach ◽  
Three Dimensional ◽  
Non Invasive ◽  
Detection And Tracking ◽  
Hybrid Detection ◽  
Outdoor Environments ◽  
Background Detail ◽  
Invasive Techniques

AbstractMonitoring animals in their natural habitat is essential for advancement of animal behavioural studies, especially in pollination studies. Non-invasive techniques are preferred for these purposes as they reduce opportunities for research apparatus to interfere with behaviour. One potentially valuable approach is image-based tracking. However, the complexity of tracking unmarked wild animals using video is challenging in uncontrolled outdoor environments. Out-of-the-box algorithms currently present several problems in this context that can compromise accuracy, especially in cases of occlusion in a 3D environment. To address the issue, we present a novel hybrid detection and tracking algorithm to monitor unmarked insects outdoors. Our software can detect an insect, identify when a tracked insect becomes occluded from view and when it re-emerges, determine when an insect exits the camera field of view, and our software assembles a series of insect locations into a coherent trajectory. The insect detecting component of the software uses background subtraction and deep learning-based detection together to accurately and efficiently locate the insect among a cluster of wildflowers.We applied our method to track honeybees foraging outdoors using a new dataset that includes complex background detail, wind-blown foliage, and insects moving into and out of occlusion beneath leaves and among three-dimensional plant structures. We evaluated our software against human observations and previous techniques. It tracked honeybees at a rate of 86.6% on our dataset, 43% higher than the computationally more expensive, standalone deep learning model YOLOv2. We illustrate the value of our approach to quantify fine-scale foraging of honeybees. The ability to track unmarked insect pollinators in this way will help researchers better understand pollination ecology. The increased efficiency of our hybrid approach paves the way for the application of deep learning-based techniques to animal tracking in real-time using low-powered devices suitable for continuous monitoring.

scholarly journals Three-dimensional Mapping with Augmented Navigation Cost through Deep Learning

2020 ◽  
Author(s):  
Felipe Oliveira ◽  
Mario F. Campos ◽  
Douglas Macharet
Keyword(s):  
Deep Learning ◽  
Cost Estimation ◽  
Three Dimensional ◽  
Three Dimensional Mapping ◽  
Localization And Mapping ◽  
Lidar Measurements ◽  
Outdoor Environments ◽  
Terrain Features ◽  
Dimensional Mapping ◽  
Terrain Characteristics

This work addresses the problem of mapping terrain features based on inertial and LiDAR measurements to estimate the navigation cost for an autonomous ground robot. Unlike most indoor applications, where surfaces are usually human-made, flat, and structured, external environments may be unpredictable regarding the types and conditions of the travel surfaces, such as traction characteristics and inclination. Attaining full autonomy in outdoor environments requires a mobile ground robot to perform the fundamental localization and mapping tasks in unfamiliar environments, but with the added challenge of unknown terrain conditions. A fuller representation of the environment is undamental to increase confidence and to reduce navigation costs. To this end, we propose a methodology composed of five main steps: i) speed-invariant inertial transformation; ii) roughness level classification; iii) navigation cost estimation; iv) sensor fusion through Deep Learning; and v) estimation of navigation costs for untraveled regions. To validate the methodology, we carried out experiments using ground robots in different outdoor environments with different terrain characteristics. Results show that the terrain maps thus obtained are a faithful representation of outdoor environments allowing for accurate and reliable path planning.

Sign in / Sign up

Export Citation Format

Share Document