M2R-Net: deep network for arbitrary oriented vehicle detection in MiniSAR images

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Zishuo Han ◽  
Chunping Wang ◽  
Qiang Fu
Keyword(s):  
Feature Fusion ◽  
Learning Algorithm ◽  
Phase Error ◽  
Class Imbalance ◽  
Sampling Strategy ◽  
Vehicle Detection ◽  
Traffic Monitoring ◽  
Detection Performance ◽  
Feature Maps ◽  
Content Type

Purpose The purpose of this paper is to use the most popular deep learning algorithm to complete the vehicle detection in the urban area of MiniSAR image, and provide reliable means for ground monitoring. Design/methodology/approach An accurate detector called the rotation region-based convolution neural networks (CNN) with multilayer fusion and multidimensional attention (M2R-Net) is proposed in this paper. Specifically, M2R-Net adopts the multilayer feature fusion strategy to extract feature maps with more extensive information. Next, the authors implement the multidimensional attention network to highlight target areas. Furthermore, a novel balanced sampling strategy for hard and easy positive-negative samples and a global balanced loss function are applied to deal with spatial imbalance and objective imbalance. Finally, rotation anchors are used to predict and calibrate the minimum circumscribed rectangle of vehicles. Findings By analyzing many groups of experiments, the validity and universality of the proposed model are verified. More importantly, comparisons with SSD, LRTDet, RFCN, DFPN, CMF-RCNN, R3Det, SCRDet demonstrate that M2R-Net has state-of-the-art detection performance. Research limitations/implications The progress in the field of MiniSAR application has been slow due to strong speckle noise, phase error, complex environments and a low signal-to-noise ratio. In addition, four kinds of imbalances, i.e. spatial imbalance, scale imbalance, class imbalance and objective imbalance, in object detection based on the CNN greatly inhibit the optimization of detection performance. Originality/value This research can not only enrich the means of daily traffic monitoring but also be used for enemy intelligence reconnaissance in wartime.

Real-time pixel-wise grasp affordance prediction based on multi-scale context information fusion

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Yongxiang Wu ◽  
Yili Fu ◽  
Shuguo Wang
Keyword(s):  
Information Fusion ◽  
Real World ◽  
Feature Fusion ◽  
Detection Performance ◽  
Context Information ◽  
Data Sets ◽  
Robotic Grasping ◽  
Content Type ◽  
Scale Feature ◽  
Multi Scale

Purpose This paper aims to use fully convolutional network (FCN) to predict pixel-wise antipodal grasp affordances for unknown objects and improve the grasp detection performance through multi-scale feature fusion. Design/methodology/approach A modified FCN network is used as the backbone to extract pixel-wise features from the input image, which are further fused with multi-scale context information gathered by a three-level pyramid pooling module to make more robust predictions. Based on the proposed unify feature embedding framework, two head networks are designed to implement different grasp rotation prediction strategies (regression and classification), and their performances are evaluated and compared with a defined point metric. The regression network is further extended to predict the grasp rectangles for comparisons with previous methods and real-world robotic grasping of unknown objects. Findings The ablation study of the pyramid pooling module shows that the multi-scale information fusion significantly improves the model performance. The regression approach outperforms the classification approach based on same feature embedding framework on two data sets. The regression network achieves a state-of-the-art accuracy (up to 98.9%) and speed (4 ms per image) and high success rate (97% for household objects, 94.4% for adversarial objects and 95.3% for objects in clutter) in the unknown object grasping experiment. Originality/value A novel pixel-wise grasp affordance prediction network based on multi-scale feature fusion is proposed to improve the grasp detection performance. Two prediction approaches are formulated and compared based on the proposed framework. The proposed method achieves excellent performances on three benchmark data sets and real-world robotic grasping experiment.

scholarly journals Glassboxing Deep Learning to Enhance Aircraft Detection from SAR Imagery

2021 ◽  
Vol 13 (18) ◽  
pp. 3650
Author(s):  
Ru Luo ◽  
Jin Xing ◽  
Lifu Chen ◽  
Zhouhao Pan ◽  
Xingmin Cai ◽  
...  
Keyword(s):  
Deep Learning ◽  
Feature Fusion ◽  
Detection Performance ◽  
Great Success ◽  
Sar Image ◽  
Feature Maps ◽  
Multi Scale ◽  
Learning Techniques ◽  
Sar Imagery ◽  
Aircraft Detection

Although deep learning has achieved great success in aircraft detection from SAR imagery, its blackbox behavior has been criticized for low comprehensibility and interpretability. Such challenges have impeded the trustworthiness and wide application of deep learning techniques in SAR image analytics. In this paper, we propose an innovative eXplainable Artificial Intelligence (XAI) framework to glassbox deep neural networks (DNN) by using aircraft detection as a case study. This framework is composed of three parts: hybrid global attribution mapping (HGAM) for backbone network selection, path aggregation network (PANet), and class-specific confidence scores mapping (CCSM) for visualization of the detector. HGAM integrates the local and global XAI techniques to evaluate the effectiveness of DNN feature extraction; PANet provides advanced feature fusion to generate multi-scale prediction feature maps; while CCSM relies on visualization methods to examine the detection performance with given DNN and input SAR images. This framework can select the optimal backbone DNN for aircraft detection and map the detection performance for better understanding of the DNN. We verify its effectiveness with experiments using Gaofen-3 imagery. Our XAI framework offers an explainable approach to design, develop, and deploy DNN for SAR image analytics.

scholarly journals Multi-Scale and Occlusion Aware Network for Vehicle Detection and Segmentation on UAV Aerial Images

2020 ◽  
Vol 12 (11) ◽  
pp. 1760 ◽  
Author(s):  
Wang Zhang ◽  
Chunsheng Liu ◽  
Faliang Chang ◽  
Ye Song
Keyword(s):  
Feature Fusion ◽  
Vehicle Detection ◽  
Aerial Images ◽  
Feature Maps ◽  
Vehicle Monitoring ◽  
Multi Scale ◽  
Adaptive Fusion ◽  
Feature Pyramid ◽  
Vehicle Information ◽  
Vehicle Segmentation

With the advantage of high maneuverability, Unmanned Aerial Vehicles (UAVs) have been widely deployed in vehicle monitoring and controlling. However, processing the images captured by UAV for the extracting vehicle information is hindered by some challenges including arbitrary orientations, huge scale variations and partial occlusion. In seeking to address these challenges, we propose a novel Multi-Scale and Occlusion Aware Network (MSOA-Net) for UAV based vehicle segmentation, which consists of two parts including a Multi-Scale Feature Adaptive Fusion Network (MSFAF-Net) and a Regional Attention based Triple Head Network (RATH-Net). In MSFAF-Net, a self-adaptive feature fusion module is proposed, which can adaptively aggregate hierarchical feature maps from multiple levels to help Feature Pyramid Network (FPN) deal with the scale change of vehicles. The RATH-Net with a self-attention mechanism is proposed to guide the location-sensitive sub-networks to enhance the vehicle of interest and suppress background noise caused by occlusions. In this study, we release a large comprehensive UAV based vehicle segmentation dataset (UVSD), which is the first public dataset for UAV based vehicle detection and segmentation. Experiments are conducted on the challenging UVSD dataset. Experimental results show that the proposed method is efficient in detecting and segmenting vehicles, and outperforms the compared state-of-the-art works.

scholarly journals Cascaded Cross-Layer Fusion Network for Pedestrian Detection

Mathematics ◽  
2022 ◽  
Vol 10 (1) ◽  
pp. 139
Author(s):  
Zhifeng Ding ◽  
Zichen Gu ◽  
Yanpeng Sun ◽  
Xinguang Xiang
Keyword(s):  
Feature Fusion ◽  
Pedestrian Detection ◽  
Detection Performance ◽  
Feature Representation ◽  
Cross Layer ◽  
Feature Maps ◽  
Level Information ◽  
High Level ◽  
Center Map ◽  
The Impact

The detection method based on anchor-free not only reduces the training cost of object detection, but also avoids the imbalance problem caused by an excessive number of anchors. However, these methods only pay attention to the impact of the detection head on the detection performance, thus ignoring the impact of feature fusion on the detection performance. In this article, we take pedestrian detection as an example and propose a one-stage network Cascaded Cross-layer Fusion Network (CCFNet) based on anchor-free. It consists of Cascaded Cross-layer Fusion module (CCF) and novel detection head. Among them, CCF fully considers the distribution of high-level information and low-level information of feature maps under different stages in the network. First, the deep network is used to remove a large amount of noise in the shallow features, and finally, the high-level features are reused to obtain a more complete feature representation. Secondly, for the pedestrian detection task, a novel detection head is designed, which uses the global smooth map (GSMap) to provide global information for the center map to obtain a more accurate center map. Finally, we verified the feasibility of CCFNet on the Caltech and CityPersons datasets.

scholarly journals A Fast and Lightweight Detection Network for Multi-Scale SAR Ship Detection under Complex Backgrounds

2021 ◽  
Vol 14 (1) ◽  
pp. 31
Author(s):  
Jimin Yu ◽  
Guangyu Zhou ◽  
Shangbo Zhou ◽  
Maowei Qin
Keyword(s):  
Feature Fusion ◽  
Detection Performance ◽  
Synthetic Aperture ◽  
Detection Methods ◽  
Constant False Alarm Rate ◽  
Feature Maps ◽  
Sar Images ◽  
Ship Detection ◽  
Multi Scale ◽  
Information Focus

It is very difficult to detect multi-scale synthetic aperture radar (SAR) ships, especially under complex backgrounds. Traditional constant false alarm rate methods are cumbersome in manual design and weak in migration capabilities. Based on deep learning, researchers have introduced methods that have shown good performance in order to get better detection results. However, the majority of these methods have a huge network structure and many parameters which greatly restrict the application and promotion. In this paper, a fast and lightweight detection network, namely FASC-Net, is proposed for multi-scale SAR ship detection under complex backgrounds. The proposed FASC-Net is mainly composed of ASIR-Block, Focus-Block, SPP-Block, and CAPE-Block. Specifically, without losing information, Focus-Block is placed at the forefront of FASC-Net for the first down-sampling of input SAR images at first. Then, ASIR-Block continues to down-sample the feature maps and use a small number of parameters for feature extraction. After that, the receptive field of the feature maps is increased by SPP-Block, and then CAPE-Block is used to perform feature fusion and predict targets of different scales on different feature maps. Based on this, a novel loss function is designed in the present paper in order to train the FASC-Net. The detection performance and generalization ability of FASC-Net have been demonstrated by a series of comparative experiments on the SSDD dataset, SAR-Ship-Dataset, and HRSID dataset, from which it is obvious that FASC-Net has outstanding detection performance on the three datasets and is superior to the existing excellent ship detection methods.

scholarly journals An Approach to Improve SSD through Skip Connection of Multiscale Feature Maps

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Xiaoguo Zhang ◽  
Ye Gao ◽  
Fei Ye ◽  
Qihan Liu ◽  
Kaixin Zhang
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Semantic Information ◽  
Feature Fusion ◽  
Poor Performance ◽  
Detection Performance ◽  
Single Shot ◽  
Small Object ◽  
Feature Maps ◽  
Input Size

SSD (Single Shot MultiBox Detector) is one of the best object detection algorithms and is able to provide high accurate object detection performance in real time. However, SSD shows relatively poor performance on small object detection because its shallow prediction layer, which is responsible for detecting small objects, lacks enough semantic information. To overcome this problem, SKIPSSD, an improved SSD with a novel skip connection of multiscale feature maps, is proposed in this paper to enhance the semantic information and the details of the prediction layers through skippingly fusing high-level and low-level feature maps. For the detail of the fusion methods, we design two feature fusion modules and multiple fusion strategies to improve the SSD detector’s sensitivity and perception ability. Experimental results on the PASCAL VOC2007 test set demonstrate that SKIPSSD significantly improves the detection performance and outperforms lots of state-of-the-art object detectors. With an input size of 300 × 300, SKIPSSD achieves 79.0% mAP (mean average precision) at 38.7 FPS (frame per second) on a single 1080 GPU, 1.8% higher than the mAP of SSD while still keeping the real-time detection speed.

Proposed Concept for Specifying Vehicle Detection Performance

2009 ◽  
Vol 2128 (1) ◽  
pp. 161-172 ◽  
Author(s):  
Dan Middleton ◽  
Ryan Longmire ◽  
Darcy M. Bullock ◽  
James R. Sturdevant
Keyword(s):  
Vehicle Detection ◽  
Detection Performance

scholarly journals Spectral Convolution Feature-Based SPD Matrix Representation for Signal Detection Using a Deep Neural Network

Entropy ◽  
2020 ◽  
Vol 22 (9) ◽  
pp. 949
Author(s):  
Jiangyi Wang ◽  
Min Liu ◽  
Xinwu Zeng ◽  
Xiaoqiang Hua
Keyword(s):  
Neural Network ◽  
Signal Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Network ◽  
Detection Method ◽  
Learning Algorithm ◽  
Simulated Data ◽  
Data Sets ◽  
Feature Maps ◽  
Simulated Data Sets

Convolutional neural networks have powerful performances in many visual tasks because of their hierarchical structures and powerful feature extraction capabilities. SPD (symmetric positive definition) matrix is paid attention to in visual classification, because it has excellent ability to learn proper statistical representation and distinguish samples with different information. In this paper, a deep neural network signal detection method based on spectral convolution features is proposed. In this method, local features extracted from convolutional neural network are used to construct the SPD matrix, and a deep learning algorithm for the SPD matrix is used to detect target signals. Feature maps extracted by two kinds of convolutional neural network models are applied in this study. Based on this method, signal detection has become a binary classification problem of signals in samples. In order to prove the availability and superiority of this method, simulated and semi-physical simulated data sets are used. The results show that, under low SCR (signal-to-clutter ratio), compared with the spectral signal detection method based on the deep neural network, this method can obtain a gain of 0.5–2 dB on simulated data sets and semi-physical simulated data sets.

scholarly journals High-Resolution SAR Image Classification Using Multi-Scale Deep Feature Fusion and Covariance Pooling Manifold Network

2021 ◽  
Vol 13 (2) ◽  
pp. 328
Author(s):  
Wenkai Liang ◽  
Yan Wu ◽  
Ming Li ◽  
Yice Cao ◽  
Xin Hu
Keyword(s):  
High Resolution ◽  
Image Classification ◽  
Feature Fusion ◽  
Representation Learning ◽  
Sar Image ◽  
Gabor Filtering ◽  
Feature Maps ◽  
Sar Images ◽  
Multi Scale ◽  
Deep Feature

The classification of high-resolution (HR) synthetic aperture radar (SAR) images is of great importance for SAR scene interpretation and application. However, the presence of intricate spatial structural patterns and complex statistical nature makes SAR image classification a challenging task, especially in the case of limited labeled SAR data. This paper proposes a novel HR SAR image classification method, using a multi-scale deep feature fusion network and covariance pooling manifold network (MFFN-CPMN). MFFN-CPMN combines the advantages of local spatial features and global statistical properties and considers the multi-feature information fusion of SAR images in representation learning. First, we propose a Gabor-filtering-based multi-scale feature fusion network (MFFN) to capture the spatial pattern and get the discriminative features of SAR images. The MFFN belongs to a deep convolutional neural network (CNN). To make full use of a large amount of unlabeled data, the weights of each layer of MFFN are optimized by unsupervised denoising dual-sparse encoder. Moreover, the feature fusion strategy in MFFN can effectively exploit the complementary information between different levels and different scales. Second, we utilize a covariance pooling manifold network to extract further the global second-order statistics of SAR images over the fusional feature maps. Finally, the obtained covariance descriptor is more distinct for various land covers. Experimental results on four HR SAR images demonstrate the effectiveness of the proposed method and achieve promising results over other related algorithms.

Unmanned vehicle dynamic obstacle detection, tracking and recognition method based on laser sensor

2021 ◽  
Vol 14 (2) ◽  
pp. 239-251
Author(s):  
Hualei Zhang ◽  
Mohammad Asif Ikbal
Keyword(s):  
Feature Fusion ◽  
Obstacle Detection ◽  
Geometric Features ◽  
Recognition Method ◽  
Content Type ◽  
Detection And Tracking ◽  
Spatio Temporal ◽  
Dynamic Obstacle ◽  
Real Vehicle ◽  
Temporal Feature

PurposeIn response to these shortcomings, this paper proposes a dynamic obstacle detection and tracking method based on multi-feature fusion and a dynamic obstacle recognition method based on spatio-temporal feature vectors.Design/methodology/approachThe existing dynamic obstacle detection and tracking methods based on geometric features have a high false detection rate. The recognition methods based on the geometric features and motion status of dynamic obstacles are greatly affected by distance and scanning angle, and cannot meet the requirements of real traffic scene applications.FindingsFirst, based on the geometric features of dynamic obstacles, the obstacles are considered The echo pulse width feature is used to improve the accuracy of obstacle detection and tracking; second, the space-time feature vector is constructed based on the time dimension and space dimension information of the obstacle, and then the support vector machine method is used to realize the recognition of dynamic obstacles to improve the obstacle The accuracy of object recognition. Finally, the accuracy and effectiveness of the proposed method are verified by real vehicle tests.Originality/valueThe paper proposes a dynamic obstacle detection and tracking method based on multi-feature fusion and a dynamic obstacle recognition method based on spatio-temporal feature vectors. The accuracy and effectiveness of the proposed method are verified by real vehicle tests.

Sign in / Sign up

Export Citation Format

Share Document