scholarly journals MAGO APPROACH FOR SEMANTIC SEGMENTATION: THE CASE STUDY OF UAVID BENCHMARK DATASET

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 353-359
Author(s):  
S. Gagliolo ◽  
D. Sguerso
Keyword(s):  
Point Cloud ◽  
Moving Objects ◽  
Adaptive Mesh ◽  
Semantic Segmentation ◽  
Segmentation Strategy ◽  
Harris Corner ◽  
Aerial Vehicle ◽  
Urban Scene ◽  
The Moment ◽  
Oblique Images

Abstract. The present work is focused on a semantic segmentation strategy implemented in the workflow of the tool MAGO (standing for “Adaptive Mesh for Orthophoto Generation”), considering the contribution of the 3D geometry and the colour information, both deriving from the point cloud of the scene. Moreover, the 2D source imagery, previously used to obtain the photogrammetric point cloud, is employed even to enhance the procedure with the recognition of moving objects, comparing the evolution of epochs.The analysed context is an urban scene, deriving from the UAVid dataset proposed for the ISPRS benchmark. In particular, the so-called “seq18”, a set of high-resolution oblique images taken by UAV (Unmanned Aerial Vehicle), has been used to test the semantic segmentation. The workflow includes the production of two Digital Surface Models (DSMs), containing the geometric and radiometric information, respectively, and their processing by means of the Harris corner detector, allowing the understanding of the image variability. Then, starting from the source geometry and colour information and combining them with their variability mapping, a preliminary classification is performed. Further criteria allow the segmentation of the humans and cars present in the scene. In particular, static objects are identified according to the content of the neighbour pixels in a certain kernel, while the evolution in time of moving elements is recognized by means of the comparison of the projected images belonging to the different epochs. The presented preliminary achievements show some criticalities that require further attention and improvement. In particular, the strategy could be enriched getting more information from the source 2D images, which at the moment are directly used only for the comparison of consecutive epochs.

scholarly journals FUSION OF FEATURE BASED AND DEEP LEARNING METHODS FOR CLASSIFICATION OF MMS POINT CLOUDS

2019 ◽  
Vol XLII-2/W16 ◽  
pp. 235-242 ◽  
Author(s):  
D. Tosic ◽  
S. Tuttas ◽  
L. Hoegner ◽  
U. Stilla
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Local Geometry ◽  
Learning Approach ◽  
Semantic Classification ◽  
Feature Based ◽  
Urban Scene

<p><strong>Abstract.</strong> This work proposes an approach for semantic classification of an outdoor-scene point cloud acquired with a high precision Mobile Mapping System (MMS), with major goal to contribute to the automatic creation of High Definition (HD) Maps. The automatic point labeling is achieved by utilizing the combination of a feature-based approach for semantic classification of point clouds and a deep learning approach for semantic segmentation of images. Both, point cloud data, as well as the data from a multi-camera system are used for gaining spatial information in an urban scene. Two types of classification applied for this task are: 1) Feature-based approach, in which the point cloud is organized into a supervoxel structure for capturing geometric characteristics of points. Several geometric features are then extracted for appropriate representation of the local geometry, followed by removing the effect of local tendency for each supervoxel to enhance the distinction between similar structures. And lastly, the Random Forests (RF) algorithm is applied in the classification phase, for assigning labels to supervoxels and therefore to points within them. 2) The deep learning approach is employed for semantic segmentation of MMS images of the same scene. To achieve this, an implementation of Pyramid Scene Parsing Network is used. Resulting segmented images with each pixel containing a class label are then projected onto the point cloud, enabling label assignment for each point. At the end, experiment results are presented from a complex urban scene and the performance of this method is evaluated on a manually labeled dataset, for the deep learning and feature-based classification individually, as well as for the result of the labels fusion. The achieved overall accuracy with fusioned output is 0.87 on the final test set, which significantly outperforms the results of individual methods on the same point cloud. The labeled data is published on the TUM-PF Semantic-Labeling-Benchmark.</p>

scholarly journals Slice-Based Instance and Semantic Segmentation for Low-Channel Roadside LiDAR Data

2020 ◽  
Vol 12 (22) ◽  
pp. 3830
Author(s):  
Hui Liu ◽  
Ciyun Lin ◽  
Dayong Wu ◽  
Bowen Gong
Keyword(s):  
Point Cloud ◽  
Moving Objects ◽  
Regional Growth ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Lidar Data ◽  
Seed Point ◽  
Growth Method ◽  
The Impact ◽  
Instance Segmentation

More and more scholars are committed to light detection and ranging (LiDAR) as a roadside sensor to obtain traffic flow data. Filtering and clustering are common methods to extract pedestrians and vehicles from point clouds. This kind of method ignores the impact of environmental information on traffic. The segmentation process is a crucial part of detailed scene understanding, which could be especially helpful for locating, recognizing, and classifying objects in certain scenarios. However, there are few studies on the segmentation of low-channel (16 channels in this paper) roadside 3D LiDAR. This paper presents a novel segmentation (slice-based) method for point clouds of roadside LiDAR. The proposed method can be divided into two parts: the instance segmentation part and semantic segmentation part. The part of the instance segmentation of point cloud is based on the regional growth method, and we proposed a seed point generation method for low-channel LiDAR data. Furthermore, we optimized the instance segmentation effect under occlusion. The part of semantic segmentation of a point cloud is realized by classifying and labeling the objects obtained by instance segmentation. For labeling static objects, we represented and classified a certain object through the related features derived from its slices. For labeling moving objects, we proposed a recurrent neural network (RNN)-based model, of which the accuracy could be up to 98.7%. The result implies that the slice-based method can obtain a good segmentation effect and the slice has good potential for point cloud segmentation.

scholarly journals Symmetry Analysis of Oriental Polygonal Pagodas Using 3D Point Clouds for Cultural Heritage

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1228
Author(s):  
Ting On Chan ◽  
Linyuan Xia ◽  
Yimin Chen ◽  
Wei Lang ◽  
Tingting Chen ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Unmanned Aerial Vehicle ◽  
Point Cloud ◽  
Geometric Model ◽  
Digital Camera ◽  
Point Clouds ◽  
Symmetry Analysis ◽  
3D Point Clouds ◽  
Transmission Towers ◽  
Aerial Vehicle

Ancient pagodas are usually parts of hot tourist spots in many oriental countries due to their unique historical backgrounds. They are usually polygonal structures comprised by multiple floors, which are separated by eaves. In this paper, we propose a new method to investigate both the rotational and reflectional symmetry of such polygonal pagodas through developing novel geometric models to fit to the 3D point clouds obtained from photogrammetric reconstruction. The geometric model consists of multiple polygonal pyramid/prism models but has a common central axis. The method was verified by four datasets collected by an unmanned aerial vehicle (UAV) and a hand-held digital camera. The results indicate that the models fit accurately to the pagodas’ point clouds. The symmetry was realized by rotating and reflecting the pagodas’ point clouds after a complete leveling of the point cloud was achieved using the estimated central axes. The results show that there are RMSEs of 5.04 cm and 5.20 cm deviated from the perfect (theoretical) rotational and reflectional symmetries, respectively. This concludes that the examined pagodas are highly symmetric, both rotationally and reflectionally. The concept presented in the paper not only work for polygonal pagodas, but it can also be readily transformed and implemented for other applications for other pagoda-like objects such as transmission towers.

scholarly journals Identifying the Branch of Kiwifruit Based on Unmanned Aerial Vehicle (UAV) Images Using Deep Learning Method

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4442
Author(s):  
Zijie Niu ◽  
Juntao Deng ◽  
Xu Zhang ◽  
Jun Zhang ◽  
Shijia Pan ◽  
...  
Keyword(s):  
Deep Learning ◽  
Unmanned Aerial Vehicle ◽  
Semantic Segmentation ◽  
Dynamic Monitoring ◽  
Support Vector ◽  
Distribution Maps ◽  
Time Operation ◽  
Aerial Vehicle ◽  
Uav Images ◽  
Segmentation Image

It is important to obtain accurate information about kiwifruit vines to monitoring their physiological states and undertake precise orchard operations. However, because vines are small and cling to trellises, and have branches laying on the ground, numerous challenges exist in the acquisition of accurate data for kiwifruit vines. In this paper, a kiwifruit canopy distribution prediction model is proposed on the basis of low-altitude unmanned aerial vehicle (UAV) images and deep learning techniques. First, the location of the kiwifruit plants and vine distribution are extracted from high-precision images collected by UAV. The canopy gradient distribution maps with different noise reduction and distribution effects are generated by modifying the threshold and sampling size using the resampling normalization method. The results showed that the accuracies of the vine segmentation using PSPnet, support vector machine, and random forest classification were 71.2%, 85.8%, and 75.26%, respectively. However, the segmentation image obtained using depth semantic segmentation had a higher signal-to-noise ratio and was closer to the real situation. The average intersection over union of the deep semantic segmentation was more than or equal to 80% in distribution maps, whereas, in traditional machine learning, the average intersection was between 20% and 60%. This indicates the proposed model can quickly extract the vine distribution and plant position, and is thus able to perform dynamic monitoring of orchards to provide real-time operation guidance.

scholarly journals PCT: Point cloud transformer

2021 ◽  
Vol 7 (2) ◽  
pp. 187-199
Author(s):  
Meng-Hao Guo ◽  
Jun-Xiong Cai ◽  
Zheng-Ning Liu ◽  
Tai-Jiang Mu ◽  
Ralph R. Martin ◽  
...  
Keyword(s):  
Language Processing ◽  
Point Cloud ◽  
Nearest Neighbor ◽  
Semantic Segmentation ◽  
Nearest Neighbor Search ◽  
Local Context ◽  
Irregular Domain ◽  
Cloud Processing ◽  
Neighbor Search ◽  
Farthest Point

AbstractThe irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer (PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation, semantic segmentation, and normal estimation tasks.

scholarly journals Comparing High Accuracy t-LiDAR and UAV-SfM Derived Point Clouds for Geomorphological Change Detection

2021 ◽  
Vol 10 (6) ◽  
pp. 367
Author(s):  
Simoni Alexiou ◽  
Georgios Deligiannakis ◽  
Aggelos Pallikarakis ◽  
Ioannis Papanikolaou ◽  
Emmanouil Psomiadis ◽  
...  
Keyword(s):  
Soil Erosion ◽  
Change Detection ◽  
Point Cloud ◽  
Laser Scanning ◽  
Point Clouds ◽  
Mean Value ◽  
Detection Methods ◽  
Mountainous Catchments ◽  
Aerial Vehicle ◽  
Soil Erosion Assessment

Analysis of two small semi-mountainous catchments in central Evia island, Greece, highlights the advantages of Unmanned Aerial Vehicle (UAV) and Terrestrial Laser Scanning (TLS) based change detection methods. We use point clouds derived by both methods in two sites (S1 & S2), to analyse the effects of a recent wildfire on soil erosion. Results indicate that topsoil’s movements in the order of a few centimetres, occurring within a few months, can be estimated. Erosion at S2 is precisely delineated by both methods, yielding a mean value of 1.5 cm within four months. At S1, UAV-derived point clouds’ comparison quantifies annual soil erosion more accurately, showing a maximum annual erosion rate of 48 cm. UAV-derived point clouds appear to be more accurate for channel erosion display and measurement, while the slope wash is more precisely estimated using TLS. Analysis of Point Cloud time series is a reliable and fast process for soil erosion assessment, especially in rapidly changing environments with difficult access for direct measurement methods. This study will contribute to proper georesource management by defining the best-suited methodology for soil erosion assessment after a wildfire in Mediterranean environments.

scholarly journals Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images

2021 ◽  
Vol 13 (16) ◽  
pp. 3065
Author(s):  
Libo Wang ◽  
Rui Li ◽  
Dongzhi Wang ◽  
Chenxi Duan ◽  
Teng Wang ◽  
...  
Keyword(s):  
Large Scale ◽  
Texture Features ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Research Field ◽  
Learning Approaches ◽  
Fine Grained ◽  
Urban Scene ◽  
Fine Resolution ◽  
With Memory

Semantic segmentation from very fine resolution (VFR) urban scene images plays a significant role in several application scenarios including autonomous driving, land cover classification, urban planning, etc. However, the tremendous details contained in the VFR image, especially the considerable variations in scale and appearance of objects, severely limit the potential of the existing deep learning approaches. Addressing such issues represents a promising research field in the remote sensing community, which paves the way for scene-level landscape pattern analysis and decision making. In this paper, we propose a Bilateral Awareness Network which contains a dependency path and a texture path to fully capture the long-range relationships and fine-grained details in VFR images. Specifically, the dependency path is conducted based on the ResT, a novel Transformer backbone with memory-efficient multi-head self-attention, while the texture path is built on the stacked convolution operation. In addition, using the linear attention mechanism, a feature aggregation module is designed to effectively fuse the dependency features and texture features. Extensive experiments conducted on the three large-scale urban scene image segmentation datasets, i.e., ISPRS Vaihingen dataset, ISPRS Potsdam dataset, and UAVid dataset, demonstrate the effectiveness of our BANet. Specifically, a 64.6% mIoU is achieved on the UAVid dataset.

scholarly journals An Inverse Node Graph-Based Method for the Urban Scene Segmentation of 3D Point Clouds

2021 ◽  
Vol 13 (15) ◽  
pp. 3021
Author(s):  
Bufan Zhao ◽  
Xianghong Hua ◽  
Kegen Yu ◽  
Xiaoxing He ◽  
Weixing Xue ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Point Clouds ◽  
Intelligent Vehicles ◽  
Critical Data ◽  
Multi Scale ◽  
3D Point Clouds ◽  
Cluster Optimization ◽  
Urban Scene ◽  
Processing Steps

Urban object segmentation and classification tasks are critical data processing steps in scene understanding, intelligent vehicles and 3D high-precision maps. Semantic segmentation of 3D point clouds is the foundational step in object recognition. To identify the intersecting objects and improve the accuracy of classification, this paper proposes a segment-based classification method for 3D point clouds. This method firstly divides points into multi-scale supervoxels and groups them by proposed inverse node graph (IN-Graph) construction, which does not need to define prior information about the node, it divides supervoxels by judging the connection state of edges between them. This method reaches minimum global energy by graph cutting, obtains the structural segments as completely as possible, and retains boundaries at the same time. Then, the random forest classifier is utilized for supervised classification. To deal with the mislabeling of scattered fragments, higher-order CRF with small-label cluster optimization is proposed to refine the classification results. Experiments were carried out on mobile laser scan (MLS) point dataset and terrestrial laser scan (TLS) points dataset, and the results show that overall accuracies of 97.57% and 96.39% were obtained in the two datasets. The boundaries of objects were retained well, and the method achieved a good result in the classification of cars and motorcycles. More experimental analyses have verified the advantages of the proposed method and proved the practicability and versatility of the method.

scholarly journals Emergency Landing Spot Detection Algorithm for Unmanned Aerial Vehicles

2021 ◽  
Vol 13 (10) ◽  
pp. 1930
Author(s):  
Gabriel Loureiro ◽  
André Dias ◽  
Alfredo Martins ◽  
José Almeida
Keyword(s):  
Point Cloud ◽  
Principal Component ◽  
Detection Algorithm ◽  
Cloud Data ◽  
Safety Issues ◽  
Spot Detection ◽  
Cloud Clusters ◽  
Aerial Vehicle ◽  
Incoming Light ◽  
Landing Spot

The use and research of Unmanned Aerial Vehicle (UAV) have been increasing over the years due to the applicability in several operations such as search and rescue, delivery, surveillance, and others. Considering the increased presence of these vehicles in the airspace, it becomes necessary to reflect on the safety issues or failures that the UAVs may have and the appropriate action. Moreover, in many missions, the vehicle will not return to its original location. If it fails to arrive at the landing spot, it needs to have the onboard capability to estimate the best area to safely land. This paper addresses the scenario of detecting a safe landing spot during operation. The algorithm classifies the incoming Light Detection and Ranging (LiDAR) data and store the location of suitable areas. The developed method analyses geometric features on point cloud data and detects potential right spots. The algorithm uses the Principal Component Analysis (PCA) to find planes in point cloud clusters. The areas that have a slope less than a threshold are considered potential landing spots. These spots are evaluated regarding ground and vehicle conditions such as the distance to the UAV, the presence of obstacles, the area’s roughness, and the spot’s slope. Finally, the output of the algorithm is the optimum spot to land and can vary during operation. The proposed approach evaluates the algorithm in simulated scenarios and an experimental dataset presenting suitability to be applied in real-time operations.

Sign in / Sign up

Export Citation Format

Share Document