scholarly journals Deep Dual-Modal Traffic Objects Instance Segmentation Method Using Camera and LIDAR Data for Autonomous Driving

2020 ◽  
Vol 12 (20) ◽  
pp. 3274
Author(s):  
Keke Geng ◽  
Ge Dong ◽  
Guodong Yin ◽  
Jingyu Hu
Keyword(s):  
Neural Network ◽  
Data Fusion ◽  
Target Detection ◽  
Environmental Conditions ◽  
Detection Performance ◽  
Lidar Data ◽  
Complex Environments ◽  
Middle Stage ◽  
Weight Assignment ◽  
Instance Segmentation

Recent advancements in environmental perception for autonomous vehicles have been driven by deep learning-based approaches. However, effective traffic target detection in complex environments remains a challenging task. This paper presents a novel dual-modal instance segmentation deep neural network (DM-ISDNN) by merging camera and LIDAR data, which can be used to deal with the problem of target detection in complex environments efficiently based on multi-sensor data fusion. Due to the sparseness of the LIDAR point cloud data, we propose a weight assignment function that assigns different weight coefficients to different feature pyramid convolutional layers for the LIDAR sub-network. We compare and analyze the adaptations of early-, middle-, and late-stage fusion architectures in depth. By comprehensively considering the detection accuracy and detection speed, the middle-stage fusion architecture with a weight assignment mechanism, with the best performance, is selected. This work has great significance for exploring the best feature fusion scheme of a multi-modal neural network. In addition, we apply a mask distribution function to improve the quality of the predicted mask. A dual-modal traffic object instance segmentation dataset is established using a 7481 camera and LIDAR data pairs from the KITTI dataset, with 79,118 manually annotated instance masks. To the best of our knowledge, there is no existing instance annotation for the KITTI dataset with such quality and volume. A novel dual-modal dataset, composed of 14,652 camera and LIDAR data pairs, is collected using our own developed autonomous vehicle under different environmental conditions in real driving scenarios, for which a total of 62,579 instance masks are obtained using semi-automatic annotation method. This dataset can be used to validate the detection performance under complex environmental conditions of instance segmentation networks. Experimental results on the dual-modal KITTI Benchmark demonstrate that DM-ISDNN using middle-stage data fusion and the weight assignment mechanism has better detection performance than single- and dual-modal networks with other data fusion strategies, which validates the robustness and effectiveness of the proposed method. Meanwhile, compared to the state-of-the-art instance segmentation networks, our method shows much better detection performance, in terms of AP and F1 score, on the dual-modal dataset collected under complex environmental conditions, which further validates the superiority of our method.

scholarly journals Robust Mouse Tracking in Complex Environments using Neural Networks

2018 ◽  
Author(s):  
Brian Q. Geuther ◽  
Sean P. Deats ◽  
Kai J. Fox ◽  
Steve A. Murray ◽  
Robert E. Braun ◽  
...  
Keyword(s):  
Neural Network ◽  
Environmental Conditions ◽  
State Of The Art ◽  
High Accuracy ◽  
General Purpose ◽  
Training Data ◽  
Network Architectures ◽  
Complex Environments ◽  
Behavioral Experiments ◽  
Modern Machine

AbstractThe ability to track animals accurately is critical for behavioral experiments. For video-based assays, this is often accomplished by manipulating environmental conditions to increase contrast between the animal and the background, in order to achieve proper foreground/background detection (segmentation). However, as behavioral paradigms become more sophisticated with ethologically relevant environments, the approach of modifying environmental conditions offers diminishing returns, particularly for scalable experiments. Currently, there is a need for methods to monitor behaviors over long periods of time, under dynamic environmental conditions, and in animals that are genetically and behaviorally heterogeneous. To address this need, we developed a state-of-the-art neural network-based tracker for mice, using modern machine vision techniques. We test three different neural network architectures to determine their performance on genetically diverse mice under varying environmental conditions. We find that an encoder-decoder segmentation neural network achieves high accuracy and speed with minimal training data. Furthermore, we provide a labeling interface, labeled training data, tuned hyperparameters, and a pre-trained network for the mouse behavior and neuroscience communities. This general-purpose neural network tracker can be easily extended to other experimental paradigms and even to other animals, through transfer learning, thus providing a robust, generalizable solution for biobehavioral research.

Road segmentation with image-LiDAR data fusion in deep neural network

2019 ◽  
Vol 79 (47-48) ◽  
pp. 35503-35518 ◽  
Author(s):  
Huafeng Liu ◽  
Yazhou Yao ◽  
Zeren Sun ◽  
Xiangrui Li ◽  
Ke Jia ◽  
...  
Keyword(s):  
Neural Network ◽  
Data Fusion ◽  
Deep Neural Network ◽  
Lidar Data ◽  
Road Segmentation

Exploiting spectral and polarimetric data fusion to enhance target detection performance

2010 ◽  
Author(s):  
Brian M. Flusche ◽  
Michael G. Gartley ◽  
John R. Schott
Keyword(s):  
Data Fusion ◽  
Target Detection ◽  
Detection Performance

scholarly journals Infrared small target detection based on region proposal and CNN classifier

2021 ◽  
Author(s):  
Mingming Fan ◽  
Shaoqing Tian ◽  
Kai Liu ◽  
Jiaxin Zhao ◽  
Yunsong Li
Keyword(s):  
Neural Network ◽  
False Alarm ◽  
False Alarm Rate ◽  
Target Detection ◽  
Detection Rate ◽  
Corner Detection ◽  
Small Target ◽  
Small Target Detection ◽  
Potential Target ◽  
Infrared Small Target

AbstractInfrared small target detection has been a challenging task due to the weak radiation intensity of targets and the complexity of the background. Traditional methods using hand-designed features are usually effective for specific background and have the problems of low detection rate and high false alarm rate in complex infrared scene. In order to fully exploit the features of infrared image, this paper proposes an infrared small target detection method based on region proposal and convolution neural network. Firstly, the small target intensity is enhanced according to the local intensity characteristics. Then, potential target regions are proposed by corner detection to ensure high detection rate of the method. Finally, the potential target regions are fed into the classifier based on convolutional neural network to eliminate the non-target regions, which can effectively suppress the complex background clutter. Extensive experiments demonstrate that the proposed method can effectively reduce the false alarm rate, and outperform other state-of-the-art methods in terms of subjective visual impression and quantitative evaluation metrics.

scholarly journals Unsupervised Damage Detection for Offshore Jacket Wind Turbine Foundations Based on an Autoencoder Neural Network

Sensors ◽  
2021 ◽  
Vol 21 (10) ◽  
pp. 3333
Author(s):  
Maria del Cisne Feijóo ◽  
Yovana Zambrano ◽  
Yolanda Vidal ◽  
Christian Tutivén
Keyword(s):  
Neural Network ◽  
Wind Turbine ◽  
Environmental Conditions ◽  
Wind Farms ◽  
Offshore Wind ◽  
Offshore Wind Turbine ◽  
Small Subset ◽  
Scaled Model ◽  
Vibration Data ◽  
Large Water

Structural health monitoring for offshore wind turbine foundations is paramount to the further development of offshore fixed wind farms. At present time there are a limited number of foundation designs, the jacket type being the preferred one in large water depths. In this work, a jacket-type foundation damage diagnosis strategy is stated. Normally, most or all the available data are of regular operation, thus methods that focus on the data leading to failures end up using only a small subset of the available data. Furthermore, when there is no historical precedent of a type of fault, those methods cannot be used. In addition, offshore wind turbines work under a wide variety of environmental conditions and regions of operation involving unknown input excitation given by the wind and waves. Taking into account the aforementioned difficulties, the stated strategy in this work is based on an autoencoder neural network model and its contribution is two-fold: (i) the proposed strategy is based only on healthy data, and (ii) it works under different operating and environmental conditions based only on the output vibration data gathered by accelerometer sensors. The proposed strategy has been tested through experimental laboratory tests on a scaled model.

scholarly journals Implementation of a Modified Faster R-CNN for Target Detection Technology of Coastal Defense Radar

2021 ◽  
Vol 13 (9) ◽  
pp. 1703
Author(s):  
He Yan ◽  
Chao Chen ◽  
Guodong Jin ◽  
Jindong Zhang ◽  
Xudong Wang ◽  
...  
Keyword(s):  
False Alarm ◽  
False Alarm Rate ◽  
Target Detection ◽  
Real Data ◽  
Detection Performance ◽  
Detection Accuracy ◽  
Constant False Alarm Rate ◽  
Data Set ◽  
Detection Technology ◽  
Coastal Defense

The traditional method of constant false-alarm rate detection is based on the assumption of an echo statistical model. The target recognition accuracy rate and the high false-alarm rate under the background of sea clutter and other interferences are very low. Therefore, computer vision technology is widely discussed to improve the detection performance. However, the majority of studies have focused on the synthetic aperture radar because of its high resolution. For the defense radar, the detection performance is not satisfactory because of its low resolution. To this end, we herein propose a novel target detection method for the coastal defense radar based on faster region-based convolutional neural network (Faster R-CNN). The main processing steps are as follows: (1) the Faster R-CNN is selected as the sea-surface target detector because of its high target detection accuracy; (2) a modified Faster R-CNN based on the characteristics of sparsity and small target size in the data set is employed; and (3) soft non-maximum suppression is exploited to eliminate the possible overlapped detection boxes. Furthermore, detailed comparative experiments based on a real data set of coastal defense radar are performed. The mean average precision of the proposed method is improved by 10.86% compared with that of the original Faster R-CNN.

scholarly journals Clutter Cancellation and Long Time Integration for GNSS-Based Passive Bistatic Radar

2021 ◽  
Vol 13 (4) ◽  
pp. 701 ◽  
Author(s):  
Binbin Wang ◽  
Hao Cha ◽  
Zibo Zhou ◽  
Bin Tian
Keyword(s):  
Fourier Transform ◽  
Target Detection ◽  
Time Integration ◽  
Detection Performance ◽  
Bistatic Radar ◽  
Detection Range ◽  
Robust Detection ◽  
Coherent Integration ◽  
Long Time ◽  
Long Time Integration

Clutter cancellation and long time integration are two vital steps for global navigation satellite system (GNSS)-based bistatic radar target detection. The former eliminates the influence of direct and multipath signals on the target detection performance, and the latter improves the radar detection range. In this paper, the extensive cancellation algorithm (ECA), which projects the surveillance channel signal in the subspace orthogonal to the clutter subspace, is first applied in GNSS-based bistatic radar. As a result, the clutter has been removed from the surveillance channel effectively. For long time integration, a modified version of the Fourier transform (FT), called long-time integration Fourier transform (LIFT), is proposed to obtain a high coherent processing gain. Relative acceleration (RA) is defined to describe the Doppler variation results from the motion of the target and long integration time. With the estimated RA, the Doppler frequency shift compensation is carried out in the LIFT. This method achieves a better and robust detection performance when comparing with the traditional coherent integration method. The simulation results demonstrate the effectiveness and advantages of the proposed processing method.

Sign in / Sign up

Export Citation Format

Share Document