Point Cloud Semantic Segmentation with Cross-Correction Features

Real time large scale point cloud segmentation is an important but challenging task for practical application like autonomous driving. Existing real time methods have achieved acceptance performance by aggregating local information. However, most of them only exploit local spatial information or local semantic information dependently, few considering the complementarity of both. In this paper, we propose a model named Spatial-Semantic Incorporation Network (SSI-Net) for real time large scale point cloud segmentation. A Spatial-Semantic Cross-correction (SSC) module is introduced in SSI-Net as a basic unit. High quality contextual features can be learned through SSC by correct and update semantic features using spatial cues, and vice verse. Adopting the plug-and-play SSC module, we design SSI-Net as an encoder-decoder architecture. To ensure efficiency, it also adopts a random sample based hierarchical network structure. Extensive experiments on several prevalent datasets demonstrate that our method can achieve state-of-the-art performance.

Panicle-3D: Efficient Phenotyping Tool for Precise Semantic Segmentation of Rice Panicle Point Cloud

The automated measurement of crop phenotypic parameters is of great significance to the quantitative study of crop growth. The segmentation and classification of crop point cloud help to realize the automation of crop phenotypic parameter measurement. At present, crop spike-shaped point cloud segmentation has problems such as fewer samples, uneven distribution of point clouds, occlusion of stem and spike, disorderly arrangement of point clouds, and lack of targeted network models. The traditional clustering method can realize the segmentation of the plant organ point cloud with relatively independent spatial location, but the accuracy is not acceptable. This paper first builds a desktop-level point cloud scanning apparatus based on a structured-light projection module to facilitate the point cloud acquisition process. Then, the rice ear point cloud was collected, and the rice ear point cloud data set was made. In addition, data argumentation is used to improve sample utilization efficiency and training accuracy. Finally, a 3D point cloud convolutional neural network model called Panicle-3D was designed to achieve better segmentation accuracy. Specifically, the design of Panicle-3D is aimed at the multiscale characteristics of plant organs, combined with the structure of PointConv and long and short jumps, which accelerates the convergence speed of the network and reduces the loss of features in the process of point cloud downsampling. After comparison experiments, the segmentation accuracy of Panicle-3D reaches 93.4%, which is higher than PointNet. Panicle-3D is suitable for other similar crop point cloud segmentation tasks.

Neighborhood co-occurrence modeling in 3D point cloud segmentation

A significant performance boost has been achieved in point cloud semantic segmentation by utilization of the encoder-decoder architecture and novel convolution operations for point clouds. However, co-occurrence relationships within a local region which can directly influence segmentation results are usually ignored by current works. In this paper, we propose a neighborhood co-occurrence matrix (NCM) to model local co-occurrence relationships in a point cloud. We generate target NCM and prediction NCM from semantic labels and a prediction map respectively. Then, Kullback-Leibler (KL) divergence is used to maximize the similarity between the target and prediction NCMs to learn the co-occurrence relationship. Moreover, for large scenes where the NCMs for a sampled point cloud and the whole scene differ greatly, we introduce a reverse form of KL divergence which can better handle the difference to supervise the prediction NCMs. We integrate our method into an existing backbone and conduct comprehensive experiments on three datasets: Semantic3D for outdoor space segmentation, and S3DIS and ScanNet v2 for indoor scene segmentation. Results indicate that our method can significantly improve upon the backbone and outperform many leading competitors.

3D Point Cloud of Single Tree Branches and Leaves Semantic Segmentation Based on Modified PointNet Network

Semantic segmentation of single tree 3D point cloud is one of the key technologies in building tree model. It plays an important role in tree skeleton extraction, tree pruning, tree model reconstruction and other fields. Because the area of a single leaf is smaller than that of the whole tree, the segmentation of branches and leaves is a challenging problem. In view of the above problems, this paper first migrates PointNet to the tree branch and leaf point cloud segmentation, and proposes an automatic segmentation method based on improved PointNet. According to the difference of normal direction between leaves and branches, the point cloud information of three dimensions coordinates, color and normal vector is input into the point feature space. In data processing, increase the number of each block data, so that the network can better learn features. MLP is added to the original PointNet network to improve the ability of extracting and learning local features. In addition, in the process of feature extraction, jump connection is added to realize feature reuse and make full use of different levels of features. The original 1×1 filter of PointNet is replaced by 3×1 filter to improve the segmentation accuracy of tree point cloud. The focus loss function focal loss is introduced into the field of 3D point cloud to reduce the impact of the imbalance of point cloud samples on the results. The results show that the improved method improves the accuracy of tree branch point cloud segmentation compared with the original PointNet for branch and leaf segmentation. The segmentation accuracy of structural elements of branches and leaves is more than 88%, and MIoU is 48%.