Road segmentation with image-LiDAR data fusion in deep neural network

2019 ◽  
Vol 79 (47-48) ◽  
pp. 35503-35518 ◽  
Author(s):  
Huafeng Liu ◽  
Yazhou Yao ◽  
Zeren Sun ◽  
Xiangrui Li ◽  
Ke Jia ◽  
...  
Keyword(s):  
Neural Network ◽  
Data Fusion ◽  
Deep Neural Network ◽  
Lidar Data ◽  
Road Segmentation

scholarly journals CLASSIFICATION OF TREE SPECIES AND STANDING DEAD TREES BY FUSING UAV-BASED LIDAR DATA AND MULTISPECTRAL IMAGERY IN THE 3D DEEP NEURAL NETWORK POINTNET++

2020 ◽  
Vol V-2-2020 ◽  
pp. 203-210
Author(s):  
S. Briechle ◽  
P. Krzystek ◽  
G. Vosselman
Keyword(s):  
Neural Network ◽  
Tree Species ◽  
Deep Neural Network ◽  
Lidar Data ◽  
Individual Tree ◽  
Dead Trees ◽  
Standing Dead ◽  
Standing Dead Trees ◽  
Multiple Tree

Abstract. Knowledge of tree species mapping and of dead wood in particular is fundamental to managing our forests. Although individual tree-based approaches using lidar can successfully distinguish between deciduous and coniferous trees, the classification of multiple tree species is still limited in accuracy. Moreover, the combined mapping of standing dead trees after pest infestation is becoming increasingly important. New deep learning methods outperform baseline machine learning approaches and promise a significant accuracy gain for tree mapping. In this study, we performed a classification of multiple tree species (pine, birch, alder) and standing dead trees with crowns using the 3D deep neural network (DNN) PointNet++ along with UAV-based lidar data and multispectral (MS) imagery. Aside from 3D geometry, we also integrated laser echo pulse width values and MS features into the classification process. In a preprocessing step, we generated the 3D segments of single trees using a 3D detection method. Our approach achieved an overall accuracy (OA) of 90.2% and was clearly superior to a baseline method using a random forest classifier and handcrafted features (OA = 85.3%). All in all, we demonstrate that the performance of the 3D DNN is highly promising for the classification of multiple tree species and standing dead trees in practice.

scholarly journals Deep Dual-Modal Traffic Objects Instance Segmentation Method Using Camera and LIDAR Data for Autonomous Driving

2020 ◽  
Vol 12 (20) ◽  
pp. 3274
Author(s):  
Keke Geng ◽  
Ge Dong ◽  
Guodong Yin ◽  
Jingyu Hu
Keyword(s):  
Neural Network ◽  
Data Fusion ◽  
Target Detection ◽  
Environmental Conditions ◽  
Detection Performance ◽  
Lidar Data ◽  
Complex Environments ◽  
Middle Stage ◽  
Weight Assignment ◽  
Instance Segmentation

Recent advancements in environmental perception for autonomous vehicles have been driven by deep learning-based approaches. However, effective traffic target detection in complex environments remains a challenging task. This paper presents a novel dual-modal instance segmentation deep neural network (DM-ISDNN) by merging camera and LIDAR data, which can be used to deal with the problem of target detection in complex environments efficiently based on multi-sensor data fusion. Due to the sparseness of the LIDAR point cloud data, we propose a weight assignment function that assigns different weight coefficients to different feature pyramid convolutional layers for the LIDAR sub-network. We compare and analyze the adaptations of early-, middle-, and late-stage fusion architectures in depth. By comprehensively considering the detection accuracy and detection speed, the middle-stage fusion architecture with a weight assignment mechanism, with the best performance, is selected. This work has great significance for exploring the best feature fusion scheme of a multi-modal neural network. In addition, we apply a mask distribution function to improve the quality of the predicted mask. A dual-modal traffic object instance segmentation dataset is established using a 7481 camera and LIDAR data pairs from the KITTI dataset, with 79,118 manually annotated instance masks. To the best of our knowledge, there is no existing instance annotation for the KITTI dataset with such quality and volume. A novel dual-modal dataset, composed of 14,652 camera and LIDAR data pairs, is collected using our own developed autonomous vehicle under different environmental conditions in real driving scenarios, for which a total of 62,579 instance masks are obtained using semi-automatic annotation method. This dataset can be used to validate the detection performance under complex environmental conditions of instance segmentation networks. Experimental results on the dual-modal KITTI Benchmark demonstrate that DM-ISDNN using middle-stage data fusion and the weight assignment mechanism has better detection performance than single- and dual-modal networks with other data fusion strategies, which validates the robustness and effectiveness of the proposed method. Meanwhile, compared to the state-of-the-art instance segmentation networks, our method shows much better detection performance, in terms of AP and F1 score, on the dual-modal dataset collected under complex environmental conditions, which further validates the superiority of our method.

Person Property Estimation Based on 2D LiDAR Data Using Deep Neural Network

2021 ◽  
pp. 763-773
Author(s):  
Mahmudul Hasan ◽  
Riku Goto ◽  
Junichi Hanawa ◽  
Hisato Fukuda ◽  
Yoshinori Kuno ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Lidar Data ◽  
Property Estimation

scholarly journals Mapping Forest Vertical Structure in Sogwang-ri Forest from Full-Waveform Lidar Point Clouds Using Deep Neural Network

2021 ◽  
Vol 13 (18) ◽  
pp. 3736
Author(s):  
Sung-Hwan Park ◽  
Hyung-Sup Jung ◽  
Sunmin Lee ◽  
Eun-Sook Kim
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Vertical Structure ◽  
Point Clouds ◽  
Canopy Height ◽  
Accurate Information ◽  
Lidar Data ◽  
Full Waveform ◽  
Density Maps ◽  
Point Density

The role of forests is increasing because of rapid land use changes worldwide that have implications on ecosystems and the carbon cycle. Therefore, it is necessary to obtain accurate information about forests and build forest inventories. However, it is difficult to assess the internal structure of the forest through 2D remote sensing techniques and fieldwork. In this aspect, we proposed a method for estimating the vertical structure of forests based on full-waveform light detection and ranging (FW LiDAR) data in this study. Voxel-based tree point density maps were generated by estimating the number of canopy height points in each voxel grid from the raster digital terrain model (DTM) and canopy height points after pre-processing the LiDAR point clouds. We applied an unsupervised classification algorithm to the voxel-based tree point density maps and identified seven classes by profile pattern analysis for the forest vertical types. The classification accuracy was found to be 72.73% from the validation from 11 field investigation sites, which was additionally confirmed through comparative analysis with aerial images. Based on this pre-classification reference map, which is assumed to be ground truths, the deep neural network (DNN) model was finally applied to perform the final classification. As a result of accuracy assessment, it showed accuracy of 92.72% with a good performance. These results demonstrate the potential of vertical structure estimation for extensive forests using FW LiDAR data and that the distinction between one-storied and two-storied forests can be clearly represented. This technique is expected to contribute to efficient and effective management of forests based on accurate information derived from the proposed method.

Data Fusion Based Hybrid Deep Neural Network Method for Solar PV Power Forecasting

2019 ◽  
Author(s):  
Dan A. Rosa De Jesus ◽  
Paras Mandal ◽  
Miguel Velez-Reyes ◽  
Shantanu Chakraborty ◽  
Tomonobu Senjyu
Keyword(s):  
Neural Network ◽  
Data Fusion ◽  
Deep Neural Network ◽  
Solar Pv ◽  
Neural Network Method ◽  
Network Method ◽  
Power Forecasting

scholarly journals Multi-Fidelity Aerodynamic Data Fusion with a Deep Neural Network Modeling Method

Entropy ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. 1022
Author(s):  
Lei He ◽  
Weiqi Qian ◽  
Tun Zhao ◽  
Qing Wang
Keyword(s):  
Neural Network ◽  
Data Fusion ◽  
Deep Neural Network ◽  
Linear Part ◽  
Neural Network Modeling ◽  
Test Case ◽  
High Fidelity ◽  
Fusion Model ◽  
Nonlinear Part ◽  
True Value

To generate more high-quality aerodynamic data using the information provided by different fidelity data, where low-fidelity aerodynamic data provides the trend information and high-fidelity aerodynamic data provides value information, we applied a deep neural network (DNN) algorithm to fuse the information of multi-fidelity aerodynamic data. We discuss the relationships between the low-fidelity and high-fidelity data, and then we describe the proposed architecture for an aerodynamic data fusion model. The architecture consists of three fully-connected neural networks that are employed to approximate low-fidelity data, and the linear part and nonlinear part of correlation for the low- and high-fidelity data, respectively. To test the proposed multi-fidelity aerodynamic data fusion method, we calculated Euler and Navier–Stokes simulations for a typical airfoil at various Mach numbers and angles of attack to obtain the aerodynamic coefficients as low- and high-fidelity data. A fusion model of the longitudinal coefficients of lift CL and drag CD was constructed with the proposed method. For comparisons, variable complexity modeling and cokriging models were also built. The accuracy spread between the predicted value and true value was discussed for both the training and test data of the three different methods. We calculated the root mean square error and average relative deviation to demonstrate the performance of the three different methods. The fusion result of the proposed method was satisfactory on the test case, and showed a better performance compared with the other two traditional methods presented. The results provide evidence that the method proposed in this paper can be useful in dealing with the multi-fidelity aerodynamic data fusion problem.

Sign in / Sign up

Export Citation Format

Share Document