scholarly journals Convolutional neural network-based coarse initial position estimation of a monocular camera in large-scale 3D light detection and ranging maps

2019 ◽  
Vol 16 (6) ◽  
pp. 172988141989351
Author(s):  
Manhui Sun ◽  
Shaowu Yang ◽  
Hengzhu Liu
Keyword(s):  
Neural Network ◽  
Mobile Robot ◽  
Convolutional Neural Network ◽  
Initial Position ◽  
Position Estimation ◽  
Light Detection And Ranging ◽  
Depth Image ◽  
Depth Information ◽  
Light Detection ◽  
Monocular Camera

Initial position estimation in global maps, which is a prerequisite for accurate localization, plays a critical role in mobile robot navigation tasks. Global positioning system signals often become unreliable in disaster sites or indoor areas, which require other localization methods to help the robot in searching and rescuing. Many visual-based approaches focus on estimating a robot’s position within prior maps acquired with cameras. In contrast to conventional methods that need a coarse estimation of initial position to precisely localize a camera in a given map, we propose a novel approach that estimates the initial position of a monocular camera within a given 3D light detection and ranging map using a convolutional neural network with no retraining is required. It enables a mobile robot to estimate a coarse position of itself in 3D maps with only a monocular camera. The key idea of our work is to use depth information as intermediate data to retrieve a camera image in immense point clouds. We employ an unsupervised learning framework to predict the depth from a single image. Then we use a pretrained convolutional neural network model to generate depth image descriptors to construct representations of the places. We retrieve the position by computing similarity scores between the current depth image and the depth images projected from the 3D maps. Experiments on the publicly available KITTI data sets have demonstrated the efficiency and feasibility of the presented algorithm.

scholarly journals Four-Dimension Deep Learning Method for Flower Quality Grading with Depth Information

Electronics ◽  
2021 ◽  
Vol 10 (19) ◽  
pp. 2353
Author(s):  
Xinyan Sun ◽  
Zhenye Li ◽  
Tingting Zhu ◽  
Chao Ni
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Classification Accuracy ◽  
Network Models ◽  
Depth Image ◽  
Depth Information ◽  
Neural Network Models ◽  
Flower Bud ◽  
Flower Quality

Grading the quality of fresh cut flowers is an important practice in the flower industry. Based on the flower maturing status, a classification method based on deep learning and depth information was proposed for the grading of flower quality. Firstly, the RGB image and the depth image of a flower bud were collected and transformed into fused RGBD information. Then, the RGBD information of a flower was set as inputs of a convolutional neural network to determine the flower bud maturing status. Four convolutional neural network models (VGG16, ResNet18, MobileNetV2, and InceptionV3) were adjusted for a four-dimensional (4D) RGBD input to classify flowers, and their classification performances were compared with and without depth information. The experimental results show that the classification accuracy was improved with depth information, and the improved InceptionV3 network with RGBD achieved the highest classification accuracy (up to 98%), which means that the depth information can effectively reflect the characteristics of the flower bud and is helpful for the classification of the maturing status. These results have a certain significance for the intelligent classification and sorting of fresh flowers.

scholarly journals Geometric property-based convolutional neural network for indoor object detection

2021 ◽  
Vol 18 (1) ◽  
pp. 172988142199332
Author(s):  
Xintao Ding ◽  
Boquan Li ◽  
Jinbao Wang
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Geometric Property ◽  
Ground Truth ◽  
Geometric Constraints ◽  
Depth Information ◽  
Training Set ◽  
Object Knowledge ◽  
The Mean

Indoor object detection is a very demanding and important task for robot applications. Object knowledge, such as two-dimensional (2D) shape and depth information, may be helpful for detection. In this article, we focus on region-based convolutional neural network (CNN) detector and propose a geometric property-based Faster R-CNN method (GP-Faster) for indoor object detection. GP-Faster incorporates geometric property in Faster R-CNN to improve the detection performance. In detail, we first use mesh grids that are the intersections of direct and inverse proportion functions to generate appropriate anchors for indoor objects. After the anchors are regressed to the regions of interest produced by a region proposal network (RPN-RoIs), we then use 2D geometric constraints to refine the RPN-RoIs, in which the 2D constraint of every classification is a convex hull region enclosing the width and height coordinates of the ground-truth boxes on the training set. Comparison experiments are implemented on two indoor datasets SUN2012 and NYUv2. Since the depth information is available in NYUv2, we involve depth constraints in GP-Faster and propose 3D geometric property-based Faster R-CNN (DGP-Faster) on NYUv2. The experimental results show that both GP-Faster and DGP-Faster increase the performance of the mean average precision.

Domain Randomization for Detection and Position Estimation of Multiples of a Single Object With Applications to Localizing Bolts on Structures

2019 ◽  
Author(s):  
Ezra Ameperosa ◽  
Pranav A. Bhounsule
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Position Estimation ◽  
Learning Approaches ◽  
Single Object ◽  
Automated Method ◽  
The Neural Network ◽  
Periodic Maintenance ◽  
Camera Position ◽  
Periodic Replacement

Abstract Periodic replacement of fasteners such as bolts are an integral part of many structures (e.g., airplanes, cars, ships) and require periodic maintenance that may involve either their tightening or replacement. Current manual practices are time consuming and costly especially due to the large number of bolts. Thus, an automated method that is able to visually detect and localize bolt positions would be highly beneficial. In this paper, we demonstrate the use of deep neural network using domain randomization for detecting and localizing multiple bolts on a workpiece. In contrast to previous deep learning approaches that require training on real images, the use of domain randomization allows for all training to be done in simulation. The key idea here is to create a wide variety of computer generated synthetic images by varying the texture, color, camera position and orientation, distractor objects, and noise, and train the neural network on these images such that the neural network is robust to scene variability and hence provides accurate results when deployed on real images. Using domain randomization, we train two neural networks, a faster regional convolutional neural network for detecting the bolt and predicting a bounding box, and a regression convolutional neural network for estimating the x- and y-position of the bolt relative to the coordinates fixed to the workpiece. Our results indicate that in the best case we are able to detect bolts with 85% accuracy and are able to predict the position of 75% of bolts within 1.27 cm. The novelty of this work is in the use of domain randomization to detect and localize: (1) multiples of a single object, and (2) small sized objects (0.6 cm × 2.5 cm).

3D Convolutional Neural Network for Falling Detection using Only Depth Information

2020 ◽  
Author(s):  
Sara Luengo Sánchez ◽  
Sergio de López Diz ◽  
David Fuentes-Jiménez ◽  
Cristina Losada-Gutiérrez ◽  
Marta Marrón-Romera ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Depth Information
Sign in / Sign up

Export Citation Format

Share Document