Monocular 3D Object Detection Based on Uncertainty Prediction of Keypoints

Three-dimensional (3D) object detection is an important task in the field of machine vision, in which the detection of 3D objects using monocular vision is even more challenging. We observe that most of the existing monocular methods focus on the design of the feature extraction framework or embedded geometric constraints, but ignore the possible errors in the intermediate process of the detection pipeline. These errors may be further amplified in the subsequent processes. After exploring the existing detection framework of keypoints, we find that the accuracy of keypoints prediction will seriously affect the solution of 3D object position. Therefore, we propose a novel keypoints uncertainty prediction network (KUP-Net) for monocular 3D object detection. In this work, we design an uncertainty prediction module to characterize the uncertainty that exists in keypoint prediction. Then, the uncertainty is used for joint optimization with object position. In addition, we adopt position-encoding to assist the uncertainty prediction, and use a timing coefficient to optimize the learning process. The experiments on our detector are conducted on the KITTI benchmark. For the two levels of easy and moderate, we achieve accuracy of 17.26 and 11.78 in AP3D, and achieve accuracy of 23.59 and 16.63 in APBEV, which are higher than the latest method KM3D.

Download Full-text

3D-GIoU: 3D Generalized Intersection over Union for Object Detection in Point Cloud

Sensors ◽

10.3390/s19194093 ◽

2019 ◽

Vol 19 (19) ◽

pp. 4093 ◽

Cited By ~ 7

Author(s):

Jun Xu ◽

Yanxin Ma ◽

Songhua He ◽

Jiahua Zhu

Keyword(s):

Object Detection ◽

Point Cloud ◽

Pedestrian Detection ◽

Three Dimensional ◽

Average Precision ◽

3D Object ◽

Automatic Driving ◽

3D Computer Vision ◽

High Level ◽

3D Object Detection

Three-dimensional (3D) object detection is an important research in 3D computer vision with significant applications in many fields, such as automatic driving, robotics, and human–computer interaction. However, the low precision is an urgent problem in the field of 3D object detection. To solve it, we present a framework for 3D object detection in point cloud. To be specific, a designed Backbone Network is used to make fusion of low-level features and high-level features, which makes full use of various information advantages. Moreover, the two-dimensional (2D) Generalized Intersection over Union is extended to 3D use as part of the loss function in our framework. Empirical experiments of Car, Cyclist, and Pedestrian detection have been conducted respectively on the KITTI benchmark. Experimental results with average precision (AP) have shown the effectiveness of the proposed network.

Download Full-text

Shift R-CNN: Deep Monocular 3D Object Detection With Closed-Form Geometric Constraints

2019 IEEE International Conference on Image Processing (ICIP) ◽

10.1109/icip.2019.8803397 ◽

2019 ◽

Cited By ~ 12

Author(s):

Andretti Naiden ◽

Vlad Paunescu ◽

Gyeongmo Kim ◽

ByeongMoon Jeon ◽

Marius Leordeanu

Keyword(s):

Object Detection ◽

Closed Form ◽

Geometric Constraints ◽

3D Object ◽

3D Object Detection

Download Full-text

3D Object Detection Using Frustums and Attention Modules for Images and Point Clouds

Signals ◽

10.3390/signals2010009 ◽

2021 ◽

Vol 2 (1) ◽

pp. 98-107

Author(s):

Yiran Li ◽

Han Xie ◽

Hyunchul Shin

Keyword(s):

Object Detection ◽

Three Dimensional ◽

Point Clouds ◽

Practical Method ◽

Autonomous Driving ◽

Feature Maps ◽

3D Object ◽

On The Road ◽

3D Object Detection ◽

Hard Cases

Three-dimensional (3D) object detection is essential in autonomous driving. Three-dimensional (3D) Lidar sensor can capture three-dimensional objects, such as vehicles, cycles, pedestrians, and other objects on the road. Although Lidar can generate point clouds in 3D space, it still lacks the fine resolution of 2D information. Therefore, Lidar and camera fusion has gradually become a practical method for 3D object detection. Previous strategies focused on the extraction of voxel points and the fusion of feature maps. However, the biggest challenge is in extracting enough edge information to detect small objects. To solve this problem, we found that attention modules are beneficial in detecting small objects. In this work, we developed Frustum ConvNet and attention modules for the fusion of images from a camera and point clouds from a Lidar. Multilayer Perceptron (MLP) and tanh activation functions were used in the attention modules. Furthermore, the attention modules were designed on PointNet to perform multilayer edge detection for 3D object detection. Compared with a previous well-known method, Frustum ConvNet, our method achieved competitive results, with an improvement of 0.27%, 0.43%, and 0.36% in Average Precision (AP) for 3D object detection in easy, moderate, and hard cases, respectively, and an improvement of 0.21%, 0.27%, and 0.01% in AP for Bird’s Eye View (BEV) object detection in easy, moderate, and hard cases, respectively, on the KITTI detection benchmarks. Our method also obtained the best results in four cases in AP on the indoor SUN-RGBD dataset for 3D object detection.

Download Full-text

Objects Detection Using Sensors Data Fusion in Autonomous Driving Scenarios

Electronics ◽

10.3390/electronics10232903 ◽

2021 ◽

Vol 10 (23) ◽

pp. 2903

Author(s):

Razvan Bocu ◽

Dorin Bocu ◽

Maksim Iavich

Keyword(s):

Object Detection ◽

Detection System ◽

Autonomous Driving ◽

3D Object ◽

3D Objects ◽

Autonomous Cars ◽

Bounding Boxes ◽

Objects Detection ◽

3D Object Detection ◽

Fine Tune

The relatively complex task of detecting 3D objects is essential in the realm of autonomous driving. The related algorithmic processes generally produce an output that consists of a series of 3D bounding boxes that are placed around specific objects of interest. The related scientific literature usually suggests that the data that are generated by different sensors or data acquisition devices are combined in order to work around inherent limitations that are determined by the consideration of singular devices. Nevertheless, there are practical issues that cannot be addressed reliably and efficiently through this strategy, such as the limited field-of-view, and the low-point density of acquired data. This paper reports a contribution that analyzes the possibility of efficiently and effectively using 3D object detection in a cooperative fashion. The evaluation of the described approach is performed through the consideration of driving data that is collected through a partnership with several car manufacturers. Considering their real-world relevance, two driving contexts are analyzed: a roundabout, and a T-junction. The evaluation shows that cooperative perception is able to isolate more than 90% of the 3D entities, as compared to approximately 25% in the case when singular sensing devices are used. The experimental setup that generated the data that this paper describes, and the related 3D object detection system, are currently actively used by the respective car manufacturers’ research groups in order to fine tune and improve their autonomous cars’ driving modules.

Download Full-text

Learning Deformable Network for 3D Object Detection on Point Clouds

Mobile Information Systems ◽

10.1155/2021/3163470 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Wanyi Zhang ◽

Xiuhua Fu ◽

Wei Li

Keyword(s):

Object Detection ◽

Point Cloud ◽

Three Dimensional ◽

Ground Truth ◽

Point Clouds ◽

Detection Accuracy ◽

3D Object ◽

Cloud Data ◽

Dimensional Object ◽

3D Object Detection

3D object detection based on point cloud data in the unmanned driving scene has always been a research hotspot in unmanned driving sensing technology. With the development and maturity of deep neural networks technology, the method of using neural network to detect three-dimensional object target begins to show great advantages. The experimental results show that the mismatch between anchor and training samples would affect the detection accuracy, but it has not been well solved. The contributions of this paper are as follows. For the first time, deformable convolution is introduced into the point cloud object detection network, which enhances the adaptability of the network to vehicles with different directions and shapes. Secondly, a new generation method of anchor in RPN is proposed, which can effectively prevent the mismatching between the anchor and ground truth and remove the angle classification loss in the loss function. Compared with the state-of-the-art method, the AP and AOS of the detection results are improved.

Download Full-text

GAC3D: improving monocular 3D object detection with ground-guide model and adaptive convolution

PeerJ Computer Science ◽

10.7717/peerj-cs.686 ◽

2021 ◽

Vol 7 ◽

pp. e686

Author(s):

Minh-Quan Viet Bui ◽

Duc Tuan Ngo ◽

Hoang-Anh Pham ◽

Duc Dung Nguyen

Keyword(s):

Object Detection ◽

Ground Plane ◽

Depth Map ◽

Autonomous Driving ◽

Geometric Constraints ◽

Training Data ◽

Depth Information ◽

3D Object ◽

Ill Posed ◽

3D Object Detection

Monocular 3D object detection has recently become prevalent in autonomous driving and navigation applications due to its cost-efficiency and easy-to-embed to existent vehicles. The most challenging task in monocular vision is to estimate a reliable object’s location cause of the lack of depth information in RGB images. Many methods tackle this ill-posed problem by directly regressing the object’s depth or take the depth map as a supplement input to enhance the model’s results. However, the performance relies heavily on the estimated depth map quality, which is bias to the training data. In this work, we propose depth-adaptive convolution to replace the traditional 2D convolution to deal with the divergent context of the image’s features. This lead to significant improvement in both training convergence and testing accuracy. Second, we propose a ground plane model that utilizes geometric constraints in the pose estimation process. With the new method, named GAC3D, we achieve better detection results. We demonstrate our approach on the KITTI 3D Object Detection benchmark, which outperforms existing monocular methods.

Download Full-text