SEMANTIC SEGMENTATION OF TERRESTRIAL LIDAR DATA USING CO-REGISTERED RGB DATA

Abstract. This paper proposes a semantic segmentation pipeline for terrestrial laser scanning data. We achieve this by combining co-registered RGB and 3D point cloud information. Semantic segmentation is performed by applying a pre-trained off-the-shelf 2D convolutional neural network over a set of projected images extracted from a panoramic photograph. This allows the network to exploit the visual image features that are learnt in a state-of-the-art segmentation models trained on very large datasets. The study focuses on the adoption of the spherical information from the laser capture and assessing the results using image classification metrics. The obtained results demonstrate that the approach is a promising alternative for asset identification in laser scanning data. We demonstrate comparable performance with spherical machine learning frameworks, however, avoid both the labelling and training efforts required with such approaches.

Download Full-text

Dense cellular segmentation for EM using 2D–3D neural network ensembles

Scientific Reports ◽

10.1038/s41598-021-81590-0 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Matthew D. Guay ◽

Zeyad A. S. Emam ◽

Adam B. Anderson ◽

Maria A. Aronova ◽

Irina D. Pokrovskaya ◽

...

Keyword(s):

Blood Platelets ◽

Semantic Segmentation ◽

3D Models ◽

Image Features ◽

Cell Models ◽

Whole Cells ◽

Large Numbers ◽

The Creation ◽

Segmentation Task ◽

Neural Network Ensembles

AbstractBiologists who use electron microscopy (EM) images to build nanoscale 3D models of whole cells and their organelles have historically been limited to small numbers of cells and cellular features due to constraints in imaging and analysis. This has been a major factor limiting insight into the complex variability of cellular environments. Modern EM can produce gigavoxel image volumes containing large numbers of cells, but accurate manual segmentation of image features is slow and limits the creation of cell models. Segmentation algorithms based on convolutional neural networks can process large volumes quickly, but achieving EM task accuracy goals often challenges current techniques. Here, we define dense cellular segmentation as a multiclass semantic segmentation task for modeling cells and large numbers of their organelles, and give an example in human blood platelets. We present an algorithm using novel hybrid 2D–3D segmentation networks to produce dense cellular segmentations with accuracy levels that outperform baseline methods and approach those of human annotators. To our knowledge, this work represents the first published approach to automating the creation of cell models with this level of structural detail.

Download Full-text

Remote Sensing for Land Administration

Remote Sensing ◽

10.3390/rs12152497 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2497

Author(s):

Rohan Bennett ◽

Peter van Oosterom ◽

Christiaan Lemmen ◽

Mila Koeva

Keyword(s):

Remote Sensing ◽

Land Tenure ◽

Laser Scanning ◽

Poverty Reduction ◽

Spatial Information ◽

Cost Effective ◽

Terrestrial Lidar ◽

Land Administration ◽

Tools And Techniques ◽

Vertical Development

Land administration constitutes the socio-technical systems that govern land tenure, use, value and development within a jurisdiction. The land parcel is the fundamental unit of analysis. Each parcel has identifiable boundaries, associated rights, and linked parties. Spatial information is fundamental. It represents the boundaries between land parcels and is embedded in cadastral sketches, plans, maps and databases. The boundaries are expressed in these records using mathematical or graphical descriptions. They are also expressed physically with monuments or natural features. Ideally, the recorded and physical expressions should align, however, in practice, this may not occur. This means some boundaries may be physically invisible, lacking accurate documentation, or potentially both. Emerging remote sensing tools and techniques offers great potential. Historically, the measurements used to produce recorded boundary representations were generated from ground-based surveying techniques. The approach was, and remains, entirely appropriate in many circumstances, although it can be timely, costly, and may only capture very limited contextual boundary information. Meanwhile, advances in remote sensing and photogrammetry offer improved measurement speeds, reduced costs, higher image resolutions, and enhanced sampling granularity. Applications of unmanned aerial vehicles (UAV), laser scanning, both airborne and terrestrial (LiDAR), radar interferometry, machine learning, and artificial intelligence techniques, all provide examples. Coupled with emergent societal challenges relating to poverty reduction, rapid urbanisation, vertical development, and complex infrastructure management, the contemporary motivation to use these new techniques is high. Fundamentally, they enable more rapid, cost-effective, and tailored approaches to 2D and 3D land data creation, analysis, and maintenance. This Special Issue hosts papers focusing on this intersection of emergent remote sensing tools and techniques, applied to domain of land administration.

Download Full-text

EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-1-347-2018 ◽

2018 ◽

Vol XLII-1 ◽

pp. 347-354 ◽

Cited By ~ 5

Author(s):

F. Politz ◽

M. Sester

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Semantic Segmentation ◽

Point Clouds ◽

Good Alternative ◽

Aerial Images ◽

Learning Approaches ◽

Advantages And Disadvantages ◽

Sensing Applications ◽

High Level

Abstract. Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96% in an ALS and 83% in a DIM test set.

Download Full-text

Semantic segmentation of pole like road furniture in mobile laser scanning data

10.3990/1.9789036548106 ◽

2019 ◽

Author(s):

F. Li

Keyword(s):

Laser Scanning ◽

Semantic Segmentation

Download Full-text

Micro-Topography Mapping through Terrestrial LiDAR in Densely Vegetated Coastal Environments

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10100665 ◽

2021 ◽

Vol 10 (10) ◽

pp. 665

Author(s):

Xukai Zhang ◽

Xuelian Meng ◽

Chunyan Li ◽

Nan Shang ◽

Jiaze Wang ◽

...

Keyword(s):

Correction Factor ◽

Laser Scanning ◽

Accuracy Assessment ◽

Correction Method ◽

Terrain Correction ◽

Surface Model ◽

Coastal Environments ◽

Terrestrial Lidar ◽

Dense Vegetation ◽

Micro Topography

Terrestrial Light Detection And Ranging (LiDAR), also referred to as terrestrial laser scanning (TLS), has gained increasing popularity in terms of providing highly detailed micro-topography with millimetric measurement precision and accuracy. However, accurately depicting terrain under dense vegetation remains a challenge due to the blocking of signal and the lack of nearby ground. Without dependence on historical data, this research proposes a novel and rapid solution to map densely vegetated coastal environments by integrating terrestrial LiDAR with GPS surveys. To verify and improve the application of terrestrial LiDAR in coastal dense-vegetation areas, we set up eleven scans of terrestrial LiDAR in October 2015 along a sand berm with vegetation planted in Plaquemines Parish of Louisiana. At the same time, 2634 GPS points were collected for the accuracy assessment of terrain mapping and terrain correction. Object-oriented classification was applied to classify the whole berm into tall vegetation, low vegetation and bare ground, with an overall accuracy of 92.7% and a kappa value of 0.89. Based on the classification results, terrain correction was conducted for the tall-vegetation and low-vegetation areas, respectively. An adaptive correction factor was applied to the tall-vegetation area, and the 95th percentile error was calculated as the correction factor from the surface model instead of the terrain model for the low-vegetation area. The terrain correction method successfully reduced the mean error from 0.407 m to −0.068 m (RMSE errors from 0.425 m to 0.146 m) in low vegetation and from 0.993 m to −0.098 m (RMSE from 1.070 m to 0.144 m) in tall vegetation.

Download Full-text

Latent 3D Volume for Joint Depth Estimation and Semantic Segmentation from a Single Image

Sensors ◽

10.3390/s20205765 ◽

2020 ◽

Vol 20 (20) ◽

pp. 5765 ◽

Cited By ~ 1

Author(s):

Seiya Ito ◽

Naoshi Kaneko ◽

Kazuhiko Sumi

Keyword(s):

3D Structure ◽

Three Dimensional ◽

Semantic Segmentation ◽

Depth Estimation ◽

Image Features ◽

Feature Representation ◽

Single Image ◽

Feature Vectors ◽

3D Space ◽

3D Volume

This paper proposes a novel 3D representation, namely, a latent 3D volume, for joint depth estimation and semantic segmentation. Most previous studies encoded an input scene (typically given as a 2D image) into a set of feature vectors arranged over a 2D plane. However, considering the real world is three-dimensional, this 2D arrangement reduces one dimension and may limit the capacity of feature representation. In contrast, we examine the idea of arranging the feature vectors in 3D space rather than in a 2D plane. We refer to this 3D volumetric arrangement as a latent 3D volume. We will show that the latent 3D volume is beneficial to the tasks of depth estimation and semantic segmentation because these tasks require an understanding of the 3D structure of the scene. Our network first constructs an initial 3D volume using image features and then generates latent 3D volume by passing the initial 3D volume through several 3D convolutional layers. We apply depth regression and semantic segmentation by projecting the latent 3D volume onto a 2D plane. The evaluation results show that our method outperforms previous approaches on the NYU Depth v2 dataset.

Download Full-text

Virtual Disassembling of Historical Edifices: Experiments and Assessments of an Automatic Approach for Classifying Multi-Scalar Point Clouds into Architectural Elements

Sensors ◽

10.3390/s20082161 ◽

2020 ◽

Vol 20 (8) ◽

pp. 2161 ◽

Cited By ~ 4

Author(s):

Arnadi Murtiyoso ◽

Pierre Grussenmeyer

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Semantic Annotation ◽

Semantic Segmentation ◽

Point Clouds ◽

Training Data ◽

Algorithmic Approach ◽

Architectural Elements ◽

Semantic Labeling ◽

Geometric Point

3D heritage documentation has seen a surge in the past decade due to developments in reality-based 3D recording techniques. Several methods such as photogrammetry and laser scanning are becoming ubiquitous amongst architects, archaeologists, surveyors, and conservators. The main result of these methods is a 3D representation of the object in the form of point clouds. However, a solely geometric point cloud is often insufficient for further analysis, monitoring, and model predicting of the heritage object. The semantic annotation of point clouds remains an interesting research topic since traditionally it requires manual labeling and therefore a lot of time and resources. This paper proposes an automated pipeline to segment and classify multi-scalar point clouds in the case of heritage object. This is done in order to perform multi-level segmentation from the scale of a historical neighborhood up until that of architectural elements, specifically pillars and beams. The proposed workflow involves an algorithmic approach in the form of a toolbox which includes various functions covering the semantic segmentation of large point clouds into smaller, more manageable and semantically labeled clusters. The first part of the workflow will explain the segmentation and semantic labeling of heritage complexes into individual buildings, while a second part will discuss the use of the same toolbox to segment the resulting buildings further into architectural elements. The toolbox was tested on several historical buildings and showed promising results. The ultimate intention of the project is to help the manual point cloud labeling, especially when confronted with the large training data requirements of machine learning-based algorithms.

Download Full-text

Toward a Novel Laser-Based Approach for Estimating Snow Interception

Remote Sensing ◽

10.3390/rs12071146 ◽

2020 ◽

Vol 12 (7) ◽

pp. 1146 ◽

Cited By ~ 1

Author(s):

Micah Russell ◽

Jan U. H. Eitel ◽

Andrew J. Maguire ◽

Timothy E. Link

Keyword(s):

Laser Scanning ◽

Three Dimensional ◽

Point Clouds ◽

Primary Objective ◽

Snow Accumulation ◽

Future Research ◽

Snow Density ◽

Canopy Interception ◽

Terrestrial Lidar ◽

Large Trees

Forests reduce snow accumulation on the ground through canopy interception and subsequent evaporative losses. To understand snow interception and associated hydrological processes, studies have typically relied on resource-intensive point scale measurements derived from weighed trees or indirect measurements that compared snow accumulation between forested sites and nearby clearings. Weighed trees are limited to small or medium-sized trees, and indirect comparisons can be confounded by wind redistribution of snow, branch unloading, and clearing size. A potential alternative method could use terrestrial lidar (light detection and ranging) because three-dimensional lidar point clouds can be generated for any size tree and can be utilized to calculate volume of the intercepted snow. The primary objective of this study was to provide a feasibility assessment for estimating snow interception volume with terrestrial laser scanning (TLS), providing information on challenges and opportunities for future research. During the winters of 2017 and 2018, intercepted snow masses were continuously measured for two model trees suspended from load-cells. Simultaneously, autonomous terrestrial lidar scanning (ATLS) was used to develop volumetric estimates of intercepted snow. Multiplying ATLS volume estimates by snow density estimates (derived from empirical models based on air temperature) enabled the comparison of predicted vs. measured snow mass. Results indicate agreement between predicted and measured values (R2 ≥ 0.69, RMSE ≥ 0.91 kg, slope ≥ 0.97, intercept ≥ −1.39) when multiplying TLS snow interception volume with a constant snow density estimate. These results suggest that TLS might be a viable alternative to traditional approaches for mapping snow interception, potentially useful for estimating snow loads on large trees, collecting data in difficult to access terrain, and calibrating snow interception models to new forest types around the globe.

Download Full-text

Calpurnia aurea (Aiton) Benth Extracts Reduce Quorum Sensing Controlled Virulence Factors in Pseudomonas aeruginosa

Molecules ◽

10.3390/molecules25102283 ◽

2020 ◽

Vol 25 (10) ◽

pp. 2283

Author(s):

Sekelwa Cosa ◽

Jostina R. Rakoma ◽

Abdullahi A. Yusuf ◽

Thilivhali E. Tshikalange

Keyword(s):

Pseudomonas Aeruginosa ◽

Molecular Docking ◽

Quorum Sensing ◽

Virulence Factors ◽

Laser Scanning ◽

Laser Scanning Microscopy ◽

Confocal Laser ◽

Promising Alternative ◽

The Individual

Pseudomonas aeruginosa is the causative agent of several life-threatening human infections. Like many other pathogens, P. aeruginosa exhibits quorum sensing (QS) controlled virulence factors such as biofilm during disease progression, complicating treatment with conventional antibiotics. Thus, impeding the pathogen’s QS circuit appears as a promising alternative strategy to overcome pseudomonas infections. In the present study, Calpurnia aurea were evaluated for their antibacterial (minimum inhibitory concentrations (MIC)), anti-quorum sensing/antivirulence (AQS), and antibiofilm potential against P. aeruginosa. AQS and antivirulence (biofilm formation, swimming, and swarming motility) activities of plant extracts were evaluated against Chromobacterium violaceum and P. aeruginosa, respectively. The in vitro AQS potential of the individual compounds were validated using in silico molecular docking. Acetone and ethanolic extracts of C. aurea showed MIC at 1.56 mg/mL. The quantitative violacein inhibition (AQS) assay showed ethyl acetate extracts as the most potent at a concentration of 1 mg/mL. GCMS analysis of C. aurea revealed 17 compounds; four (pentadecanol, dimethyl terephthalate, terephthalic acid, and methyl mannose) showed potential AQS through molecular docking against the CviR protein of C. violaceum. Biofilm of P. aeruginosa was significantly inhibited by ≥60% using 1-mg/mL extract of C. aurea. Confocal laser scanning microscopy correlated the findings of crystal violet assay with the extracts significantly altering the swimming motility. C. aurea extracts reduced the virulence of pseudomonas, albeit in a strain- and extract-specific manner, showing their suitability for the identification of lead compounds with QS inhibitory potential for the control of P. aeruginosa infections.

Download Full-text