scholarly journals Multi-View Fusion-Based 3D Object Detection for Robot Indoor Scene Perception

Sensors ◽  
2019 ◽  
Vol 19 (19) ◽  
pp. 4092 ◽  
Author(s):  
Li Wang ◽  
Ruifeng Li ◽  
Jingwen Sun ◽  
Xingxing Liu ◽  
Lijun Zhao ◽  
...  
Keyword(s):  
Object Detection ◽  
Scene Perception ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Service Robot ◽  
Multiple Views ◽  
Object Point ◽  
3D Object ◽  
Bounding Box ◽  
3D Object Detection

To autonomously move and operate objects in cluttered indoor environments, a service robot requires the ability of 3D scene perception. Though 3D object detection can provide an object-level environmental description to fill this gap, a robot always encounters incomplete object observation, recurring detections of the same object, error in detection, or intersection between objects when conducting detection continuously in a cluttered room. To solve these problems, we propose a two-stage 3D object detection algorithm which is to fuse multiple views of 3D object point clouds in the first stage and to eliminate unreasonable and intersection detections in the second stage. For each view, the robot performs a 2D object semantic segmentation and obtains 3D object point clouds. Then, an unsupervised segmentation method called Locally Convex Connected Patches (LCCP) is utilized to segment the object accurately from the background. Subsequently, the Manhattan Frame estimation is implemented to calculate the main orientation of the object and subsequently, the 3D object bounding box can be obtained. To deal with the detected objects in multiple views, we construct an object database and propose an object fusion criterion to maintain it automatically. Thus, the same object observed in multi-view is fused together and a more accurate bounding box can be calculated. Finally, we propose an object filtering approach based on prior knowledge to remove incorrect and intersecting objects in the object dataset. Experiments are carried out on both SceneNN dataset and a real indoor environment to verify the stability and accuracy of 3D semantic segmentation and bounding box detection of the object with multi-view fusion.

scholarly journals KDA3D: Key-Point Densification and Multi-Attention Guidance for 3D Object Detection

2020 ◽  
Vol 12 (11) ◽  
pp. 1895 ◽  
Author(s):  
Jiarong Wang ◽  
Ming Zhu ◽  
Bo Wang ◽  
Deyao Sun ◽  
Hua Wei ◽  
...  
Keyword(s):  
Object Detection ◽  
Network Architecture ◽  
Feature Learning ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Networks ◽  
3D Object ◽  
Rgb Images ◽  
Bounding Boxes ◽  
3D Object Detection

In this paper, we propose a novel 3D object detector KDA3D, which achieves high-precision and robust classification, segmentation, and localization with the help of key-point densification and multi-attention guidance. The proposed end-to-end neural network architecture takes LIDAR point clouds as the main inputs that can be optionally complemented by RGB images. It consists of three parts: part-1 segments 3D foreground points and generates reliable proposals; part-2 (optional) enhances point cloud density and reconstructs the more compact full-point feature map; part-3 refines 3D bounding boxes and adds semantic segmentation as extra supervision. Our designed lightweight point-wise and channel-wise attention modules can adaptively strengthen the “skeleton” and “distinctiveness” point-features to help feature learning networks capture more representative or finer patterns. The proposed key-point densification component can generate pseudo-point clouds containing target information from monocular images through the distance preference strategy and K-means clustering so as to balance the density distribution and enrich sparse features. Extensive experiments on the KITTI and nuScenes 3D object detection benchmarks show that our KDA3D produces state-of-the-art results while running in near real-time with a low memory footprint.

scholarly journals Strong-Weak Feature Alignment for 3D Object Detection

Electronics ◽  
2021 ◽  
Vol 10 (10) ◽  
pp. 1205
Author(s):  
Zhiyu Wang ◽  
Li Wang ◽  
Bin Dai
Keyword(s):  
Object Detection ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Feature Representation ◽  
Alignment Algorithm ◽  
3D Object ◽  
3D Point Clouds ◽  
Object Feature ◽  
3D Object Detection ◽  
Feature Alignment

Object detection in 3D point clouds is still a challenging task in autonomous driving. Due to the inherent occlusion and density changes of the point cloud, the data distribution of the same object will change dramatically. Especially, the incomplete data with sparsity or occlusion can not represent the complete characteristics of the object. In this paper, we proposed a novel strong–weak feature alignment algorithm between complete and incomplete objects for 3D object detection, which explores the correlations within the data. It is an end-to-end adaptive network that does not require additional data and can be easily applied to other object detection networks. Through a complete object feature extractor, we achieve a robust feature representation of the object. It serves as a guarding feature to help the incomplete object feature generator to generate effective features. The strong–weak feature alignment algorithm reduces the gap between different states of the same object and enhances the ability to represent the incomplete object. The proposed adaptation framework is validated on the KITTI object benchmark and gets about 6% improvement in detection average precision on 3D moderate difficulty compared to the basic model. The results show that our adaptation method improves the detection performance of incomplete 3D objects.

Real-Time 3D object detection using improved convolutional neural network based on image-driven point cloud

2021 ◽  
Vol 14 ◽  
Author(s):  
Zhiyong Gao ◽  
Jianhong Xiang
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Real Time ◽  
Point Cloud ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Object ◽  
3D Object Detection ◽  
Instance Segmentation

Background: While detecting the object directly from the 3D point cloud, the natural 3D patterns and invariance of 3D data are often obscure. Objective: In this work, we aimed at studying the 3D object detection from discrete, disordered and sparse 3D point clouds. Methods: The CNN is composed of the frustum sequence module, 3D instance segmentation module S-NET, 3D point cloud transformation module T-NET, and 3D boundary box estimation module E-NET. The search space of the object is determined by the frustum sequence module. The instance segmentation of the point cloud is performed by the 3D instance segmentation module. The 3D coordinates of the object are confirmed by the transformation module and the 3D bounding box estimation module. Results: Evaluated on KITTI benchmark dataset, our method outperforms the state of the art by remarkable margins while having real-time capability. Conclusion: We achieve real-time 3D object detection by proposing an improved convolutional neural network (CNN) based on image-driven point clouds.

Adversarial Training on Point Clouds for Sim-to-Real 3D Object Detection

2021 ◽  
pp. 1-1
Author(s):  
Robert Debortoli ◽  
Fuxin Li ◽  
Ashish Kapoor ◽  
Geoffrey Hollinger
Keyword(s):  
Object Detection ◽  
Point Clouds ◽  
3D Object ◽  
Adversarial Training ◽  
3D Object Detection

Generating Adversarial Point Clouds on Multi-modal Fusion Based 3D Object Detection Model

2021 ◽  
pp. 187-203
Author(s):  
Huiying Wang ◽  
Huixin Shen ◽  
Boyang Zhang ◽  
Yu Wen ◽  
Dan Meng
Keyword(s):  
Object Detection ◽  
Point Clouds ◽  
3D Object ◽  
Detection Model ◽  
3D Object Detection

scholarly journals 3D Object Detection Using Scale Invariant and Feature Reweighting Networks

2019 ◽  
Vol 33 ◽  
pp. 9267-9274 ◽  
Author(s):  
Xin Zhao ◽  
Zhe Liu ◽  
Ruolan Hu ◽  
Kaiqi Huang
Keyword(s):  
Object Detection ◽  
Network Architecture ◽  
Point Clouds ◽  
Scale Invariant ◽  
3D Object ◽  
Outdoor Scenes ◽  
Indoor Scenes ◽  
Bounding Boxes ◽  
The One ◽  
3D Object Detection

3D object detection plays an important role in a large number of real-world applications. It requires us to estimate the localizations and the orientations of 3D objects in real scenes. In this paper, we present a new network architecture which focuses on utilizing the front view images and frustum point clouds to generate 3D detection results. On the one hand, a PointSIFT module is utilized to improve the performance of 3D segmentation. It can capture the information from different orientations in space and the robustness to different scale shapes. On the other hand, our network obtains the useful features and suppresses the features with less information by a SENet module. This module reweights channel features and estimates the 3D bounding boxes more effectively. Our method is evaluated on both KITTI dataset for outdoor scenes and SUN-RGBD dataset for indoor scenes. The experimental results illustrate that our method achieves better performance than the state-of-the-art methods especially when point clouds are highly sparse.

scholarly journals An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds

2020 ◽  
pp. 266-282
Author(s):  
Rui Huang ◽  
Wanyue Zhang ◽  
Abhijit Kundu ◽  
Caroline Pantofaru ◽  
David A. Ross ◽  
...  
Keyword(s):  
Object Detection ◽  
Point Clouds ◽  
3D Object ◽  
3D Object Detection

scholarly journals 3D object detection combining semantic and geometric features from point clouds

Cobot ◽  
2022 ◽  
Vol 1 ◽  
pp. 2
Author(s):  
Hao Peng ◽  
Guofeng Tong ◽  
Zheng Li ◽  
Yaqi Wang ◽  
Yuyuan Shao
Keyword(s):  
Object Detection ◽  
Feature Learning ◽  
Point Clouds ◽  
Difficulty Level ◽  
Geometric Feature ◽  
Geometric Features ◽  
3D Object ◽  
Point Module ◽  
Institute Of Technology ◽  
3D Object Detection

Background: 3D object detection based on point clouds in road scenes has attracted much attention recently. The voxel-based methods voxelize the scene to regular grids, which can be processed with the advanced feature learning frameworks based on convolutional layers for semantic feature learning. The point-based methods can extract the geometric feature of the point due to the coordinate reservations. The combination of the two is effective for 3D object detection. However, the current methods use a voxel-based detection head with anchors for classification and localization. Although the preset anchors cover the entire scene, it is not suitable for detection tasks with larger scenes and multiple categories of objects, due to the limitation of the voxel size. Additionally, the misalignment between the predicted confidence and proposals in the Regions of the Interest (ROI) selection bring obstacles to 3D object detection. Methods: We investigate the combination of voxel-based methods and point-based methods for 3D object detection. Additionally, a voxel-to-point module that captures semantic and geometric features is proposed in the paper. The voxel-to-point module is conducive to the detection of small-size objects and avoids the presets of anchors in the inference stage. Moreover, a confidence adjustment module with the center-boundary-aware confidence attention is proposed to solve the misalignment between the predicted confidence and proposals in the regions of the interest selection. Results: The proposed method has achieved state-of-the-art results for 3D object detection in the  Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) object detection dataset. Actually, as of September 19, 2021, our method ranked 1st in the 3D and Bird Eyes View (BEV) detection of cyclists tagged with difficulty level ‘easy’, and ranked 2nd in the 3D detection of cyclists tagged with ‘moderate’. Conclusions: We propose an end-to-end two-stage 3D object detector with voxel-to-point module and confidence adjustment module.

scholarly journals Optimization of the PointPillars network for 3D object detection in point clouds

2020 ◽  
Author(s):  
Joanna Stanisz ◽  
Konrad Lis ◽  
Tomasz Kryjak ◽  
Marek Gorgon
Keyword(s):  
Object Detection ◽  
Point Cloud ◽  
Main Part ◽  
Point Clouds ◽  
Lidar Data ◽  
Detection Accuracy ◽  
3D Object ◽  
Fold Reduction ◽  
Low Energy Consumption ◽  
3D Object Detection

In this paper we present our research on the optimisation of a deep neural network for 3D object detection in a point cloud. Techniques like quantisation and pruning available in the Brevitas and PyTorch tools were used. We performed the experiments for the PointPillars network, which offers a reasonable compromise between detection accuracy and calculation complexity. The aim of this work was to propose a variant of the network which we will ultimately implement in an FPGA device. This will allow for real-time LiDAR data processing with low energy consumption. The obtained results indicate that even a significant quantisation from 32-bit floating point to 2-bit integer in the main part of the algorithm, results in 5%-9% decrease of the detection accuracy, while allowing for almost a 16-fold reduction in size of the model.

Sign in / Sign up

Export Citation Format

Share Document