2D-to-3D Projection for Monocular and Multi-View 3D Multi-class Object Detection in Indoor Scenes

In this paper, we propose a novel method of joint 3D object detection and room layout estimation. The proposed method surpasses all existing methods of 3D object detection from monocular images on the indoor SUN RGB-D dataset. Moreover, the proposed method shows competitive results on the ScanNet dataset in multi-view mode. Both these datasets are collected in various residential, administrative, educational and industrial spaces, and altogether they cover almost all possible use cases. Moreover, we are the first to formulate and solve a problem of multi-class 3D object detection from multi-view inputs in indoor scenes. The proposed method can be integrated into the controlling systems of mobile robots. The results of this study can be used to address a navigation task, as well as path planning, capturing and manipulating scene objects, and semantic scene mapping.

Download Full-text

A 2D-3D Object Detection System for Updating Building Information Models with Mobile Robots

2019 IEEE Winter Conference on Applications of Computer Vision (WACV) ◽

10.1109/wacv.2019.00149 ◽

2019 ◽

Cited By ~ 2

Author(s):

Max Ferguson ◽

Kincho Law

Keyword(s):

Object Detection ◽

Mobile Robots ◽

Detection System ◽

3D Object ◽

Information Models ◽

Building Information ◽

Building Information Models ◽

3D Object Detection

Download Full-text

3D Object Detection Using Scale Invariant and Feature Reweighting Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33019267 ◽

2019 ◽

Vol 33 ◽

pp. 9267-9274 ◽

Cited By ~ 6

Author(s):

Xin Zhao ◽

Zhe Liu ◽

Ruolan Hu ◽

Kaiqi Huang

Keyword(s):

Object Detection ◽

Network Architecture ◽

Point Clouds ◽

Scale Invariant ◽

3D Object ◽

Outdoor Scenes ◽

Indoor Scenes ◽

Bounding Boxes ◽

The One ◽

3D Object Detection

3D object detection plays an important role in a large number of real-world applications. It requires us to estimate the localizations and the orientations of 3D objects in real scenes. In this paper, we present a new network architecture which focuses on utilizing the front view images and frustum point clouds to generate 3D detection results. On the one hand, a PointSIFT module is utilized to improve the performance of 3D segmentation. It can capture the information from different orientations in space and the robustness to different scale shapes. On the other hand, our network obtains the useful features and suppresses the features with less information by a SENet module. This module reweights channel features and estimates the 3D bounding boxes more effectively. Our method is evaluated on both KITTI dataset for outdoor scenes and SUN-RGBD dataset for indoor scenes. The experimental results illustrate that our method achieves better performance than the state-of-the-art methods especially when point clouds are highly sparse.

Download Full-text

Task-Aware Monocular Depth Estimation for 3D Object Detection

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6908 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12257-12264 ◽

Cited By ~ 1

Author(s):

Xinlong Wang ◽

Wei Yin ◽

Tao Kong ◽

Yuning Jiang ◽

Lei Li ◽

...

Keyword(s):

Object Detection ◽

State Of The Art ◽

Depth Estimation ◽

3D Perception ◽

Research Attention ◽

3D Object ◽

Depth Prediction ◽

Monocular Depth ◽

Almost All ◽

3D Object Detection

Monocular depth estimation enables 3D perception from a single 2D image, thus attracting much research attention for years. Almost all methods treat foreground and background regions (“things and stuff”) in an image equally. However, not all pixels are equal. Depth of foreground objects plays a crucial role in 3D object recognition and localization. To date how to boost the depth prediction accuracy of foreground objects is rarely discussed. In this paper, we first analyze the data distributions and interaction of foreground and background, then propose the foreground-background separated monocular depth estimation (ForeSeE) method, to estimate the foreground and background depth using separate optimization objectives and decoders. Our method significantly improves the depth estimation performance on foreground objects. Applying ForeSeE to 3D object detection, we achieve 7.5 AP gains and set new state-of-the-art results among other monocular methods. Code will be available at: https://github.com/WXinlong/ForeSeE.

Download Full-text

Visual-Inertial-Semantic Scene Representation for 3D Object Detection

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ◽

10.1109/cvpr.2017.380 ◽

2017 ◽

Cited By ~ 10

Author(s):

Jingming Dong ◽

Xiaohan Fei ◽

Stefano Soatto

Keyword(s):

Object Detection ◽

3D Object ◽

Scene Representation ◽

Semantic Scene ◽

3D Object Detection

Download Full-text

Efficient 3D Object Detection of Indoor Scenes Based on RGB-D Video Stream

Journal of Computer-Aided Design & Computer Graphics ◽

10.3724/sp.j.1089.2021.18630 ◽

2021 ◽

Vol 33 (7) ◽

pp. 1015-1025

Author(s):

Yongwei Miao ◽

Jiahui Chen ◽

Xinjie Zhang ◽

Wenjuan Ma ◽

Shusen Sun

Keyword(s):

Object Detection ◽

Video Stream ◽

3D Object ◽

Indoor Scenes ◽

3D Object Detection

Download Full-text

2D-to-3D Projection for Monocular and Multi-View 3D Object Detection in Outdoor Scenes

PROGRAMMNAYA INGENERIA ◽

10.17587/prin.12.373-384 ◽

2021 ◽

Vol 12 (7) ◽

pp. 373-384

Author(s):

D. D. Rukhovich ◽

Keyword(s):

Object Detection ◽

Detection Methods ◽

Navigation Systems ◽

Detection Problem ◽

3D Object ◽

Car Detection ◽

Extensive Evaluation ◽

Outdoor Scenes ◽

Semantic Scene ◽

3D Object Detection

In this article, we introduce the task of multi-view RGB-based 3D object detection as an end-to-end optimization problem. In a multi-view formulation of the 3D object detection problem, several images of a static scene are used to detect objects in the scene. To address the 3D object detection problem in a multi-view formulation, we propose a novel 3D object detection method named ImVoxelNet. ImVoxelNet is based on a fully convolutional neural network. Unlike existing 3D object detection methods, ImVoxelNet works directly with 3D representations and does not mediate 3D object detection through 2D object detection. The proposed method accepts multi-view inputs. The number of monocular images in each multi-view input can vary during training and inference; actually, this number might be unique for each multi-view input. Moreover, we propose to treat a single RGB image as a special case of a multi-view input. Accordingly, the proposed method can also accept monocular inputs with no modifications. Through extensive evaluation, we demonstrate that the proposed method successfully handles a variety of outdoor scenes. Specifically, it achieves state-of-the-art results in car detection on KITTI (monocular) and nuScenes (multi-view) benchmarks among all methods that accept RGB images. The proposed method operates in real-time, which makes it possible to integrate it into the navigation systems of autonomous devices. The results of this study can be used to address tasks of navigation, path planning, and semantic scene mapping.

Download Full-text