A new multi-scale backbone network for object detection based on asymmetric convolutions

Real-time object detection on mobile platforms is a crucial but challenging computer vision task. However, it is widely recognized that although the lightweight object detectors have a high detection speed, the detection accuracy is relatively low. In order to improve detecting accuracy, it is beneficial to extract complete multi-scale image features in visual cognitive tasks. Asymmetric convolutions have a useful quality, that is, they have different aspect ratios, which can be used to exact image features of objects, especially objects with multi-scale characteristics. In this paper, we exploit three different asymmetric convolutions in parallel and propose a new multi-scale asymmetric convolution unit, namely MAC block to enhance multi-scale representation ability of CNNs. In addition, MAC block can adaptively merge the features with different scales by allocating learnable weighted parameters to three different asymmetric convolution branches. The proposed MAC blocks can be inserted into the state-of-the-art backbone such as ResNet-50 to form a new multi-scale backbone network of object detectors. To evaluate the performance of MAC block, we conduct experiments on CIFAR-100, PASCAL VOC 2007, PASCAL VOC 2012 and MS COCO 2014 datasets. Experimental results show that the detection precision can be greatly improved while a fast detection speed is guaranteed as well.

Download Full-text

GC-YOLOv3: You Only Look Once with Global Context Block

Electronics ◽

10.3390/electronics9081235 ◽

2020 ◽

Vol 9 (8) ◽

pp. 1235

Author(s):

Yang Yang ◽

Hongmin Deng

Keyword(s):

Object Detection ◽

Irrelevant Information ◽

Detection Algorithm ◽

Visual Object ◽

Detection Accuracy ◽

Feature Maps ◽

Average Precision ◽

Global Context ◽

Pascal Voc ◽

Feature Pyramid

In order to make the classification and regression of single-stage detectors more accurate, an object detection algorithm named Global Context You-Only-Look-Once v3 (GC-YOLOv3) is proposed based on the You-Only-Look-Once (YOLO) in this paper. Firstly, a better cascading model with learnable semantic fusion between a feature extraction network and a feature pyramid network is designed to improve detection accuracy using a global context block. Secondly, the information to be retained is screened by combining three different scaling feature maps together. Finally, a global self-attention mechanism is used to highlight the useful information of feature maps while suppressing irrelevant information. Experiments show that our GC-YOLOv3 reaches a maximum of 55.5 object detection mean Average Precision (mAP)@0.5 on Common Objects in Context (COCO) 2017 test-dev and that the mAP is 5.1% higher than that of the YOLOv3 algorithm on Pascal Visual Object Classes (PASCAL VOC) 2007 test set. Therefore, experiments indicate that the proposed GC-YOLOv3 model exhibits optimal performance on the PASCAL VOC and COCO datasets.

Download Full-text

Object Detection Based on Region Decomposition and Assembly

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33018094 ◽

2019 ◽

Vol 33 ◽

pp. 8094-8101 ◽

Cited By ~ 4

Author(s):

Seung-Hwan Bae

Keyword(s):

Neural Networks ◽

Object Detection ◽

Performance Improvement ◽

Semantic Relations ◽

Detection Accuracy ◽

Semantic Features ◽

Multi Scale ◽

Object Proposals ◽

Object Region ◽

High Level

Region-based object detection infers object regions for one or more categories in an image. Due to the recent advances in deep learning and region proposal methods, object detectors based on convolutional neural networks (CNNs) have been flourishing and provided the promising detection results. However, the detection accuracy is degraded often because of the low discriminability of object CNN features caused by occlusions and inaccurate region proposals. In this paper, we therefore propose a region decomposition and assembly detector (R-DAD) for more accurate object detection.In the proposed R-DAD, we first decompose an object region into multiple small regions. To capture an entire appearance and part details of the object jointly, we extract CNN features within the whole object region and decomposed regions. We then learn the semantic relations between the object and its parts by combining the multi-region features stage by stage with region assembly blocks, and use the combined and high-level semantic features for the object classification and localization. In addition, for more accurate region proposals, we propose a multi-scale proposal layer that can generate object proposals of various scales. We integrate the R-DAD into several feature extractors, and prove the distinct performance improvement on PASCAL07/12 and MSCOCO18 compared to the recent convolutional detectors.

Download Full-text

Optimized loss functions for object detection and application on nighttime vehicle detection

Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering ◽

10.1177/09544070211036366 ◽

2021 ◽

pp. 095440702110363

Author(s):

Shang Jiang ◽

Haoran Qin ◽

Bingli Zhang ◽

Jieyu Zheng

Keyword(s):

Object Detection ◽

Loss Function ◽

Mahalanobis Distance ◽

Vehicle Detection ◽

Detection Task ◽

Loss Functions ◽

Detection Accuracy ◽

Localization Accuracy ◽

Detection Speed

The loss function is a crucial factor that affects the detection precision in the object detection task. In this paper, we optimize both two loss functions for classification and localization simultaneously. Firstly, we reconstruct the classification loss function by combining the prediction results of localization, aiming to establish the correlation between localization and classification subnetworks. Compared to the existing studies, in which the correlation is only established among the positive samples and applied to improve the localization accuracy of predicted boxes, this paper utilizes the correlation to define the hard negative samples and then puts emphasis on the classification of them. Thus the whole misclassified rate for negative samples can be reduced. Besides, a novel localization loss named MIoU is proposed by incorporating a Mahalanobis distance between the predicted box and target box, eliminating the gradients inconsistency problem in the DIoU loss, further improving the localization accuracy. Finally, the proposed methods are applied to train the networks for nighttime vehicle detection. Experimental results show that the detection accuracy can be outstandingly improved with our proposed loss functions without hurting the detection speed.

Download Full-text

Object Detection Network Based on Feature Fusion and Attention Mechanism

Future Internet ◽

10.3390/fi11010009 ◽

2019 ◽

Vol 11 (1) ◽

pp. 9 ◽

Cited By ~ 6

Author(s):

Ying Zhang ◽

Yimin Chen ◽

Chen Huang ◽

Mingke Gao

Keyword(s):

Object Detection ◽

Feature Fusion ◽

Empirical Evaluation ◽

Attention Mechanism ◽

Detection Accuracy ◽

Small Object ◽

Art Object ◽

Pascal Voc ◽

Almost All ◽

The Impact

In recent years, almost all of the current top-performing object detection networks use CNN (convolutional neural networks) features. State-of-the-art object detection networks depend on CNN features. In this work, we add feature fusion in the object detection network to obtain a better CNN feature, which incorporates well deep, but semantic, and shallow, but high-resolution, CNN features, thus improving the performance of a small object. Also, the attention mechanism was applied to our object detection network, AF R-CNN (attention mechanism and convolution feature fusion based object detection), to enhance the impact of significant features and weaken background interference. Our AF R-CNN is a single end to end network. We choose the pre-trained network, VGG-16, to extract CNN features. Our detection network is trained on the dataset, PASCAL VOC 2007 and 2012. Empirical evaluation of the PASCAL VOC 2007 dataset demonstrates the effectiveness and improvement of our approach. Our AF R-CNN achieves an object detection accuracy of 75.9% on PASCAL VOC 2007, six points higher than Faster R-CNN.

Download Full-text

PSANet: Pyramid Splitting and Aggregation Network for 3D Object Detection in Point Cloud

Sensors ◽

10.3390/s21010136 ◽

2020 ◽

Vol 21 (1) ◽

pp. 136

Author(s):

Fangyu Li ◽

Weizheng Jin ◽

Cien Fan ◽

Lian Zou ◽

Qingsheng Chen ◽

...

Keyword(s):

Object Detection ◽

Point Clouds ◽

Autonomous Driving ◽

Feature Maps ◽

3D Object ◽

Multi Scale ◽

Backbone Network ◽

3D Object Detection ◽

Different Levels ◽

Fine Branch

3D object detection in LiDAR point clouds has been extensively used in autonomous driving, intelligent robotics, and augmented reality. Although the one-stage 3D detector has satisfactory training and inference speed, there are still some performance problems due to insufficient utilization of bird’s eye view (BEV) information. In this paper, a new backbone network is proposed to complete the cross-layer fusion of multi-scale BEV feature maps, which makes full use of various information for detection. Specifically, our proposed backbone network can be divided into a coarse branch and a fine branch. In the coarse branch, we use the pyramidal feature hierarchy (PFH) to generate multi-scale BEV feature maps, which retain the advantages of different levels and serves as the input of the fine branch. In the fine branch, our proposed pyramid splitting and aggregation (PSA) module deeply integrates different levels of multi-scale feature maps, thereby improving the expressive ability of the final features. Extensive experiments on the challenging KITTI-3D benchmark show that our method has better performance in both 3D and BEV object detection compared with some previous state-of-the-art methods. Experimental results with average precision (AP) prove the effectiveness of our network.

Download Full-text

Real-Time Water Surface Object Detection Based on Improved Faster R-CNN

Sensors ◽

10.3390/s19163523 ◽

2019 ◽

Vol 19 (16) ◽

pp. 3523 ◽

Cited By ~ 4

Author(s):

Lili Zhang ◽

Yi Zhang ◽

Zhen Zhang ◽

Jie Shen ◽

Huibin Wang

Keyword(s):

Object Detection ◽

Real Time ◽

Water Surface ◽

Detection Methods ◽

Detection Accuracy ◽

Natural Scenes ◽

The North ◽

Aspect Ratios ◽

High Level ◽

Surface Object

In this paper, we consider water surface object detection in natural scenes. Generally, background subtraction and image segmentation are the classical object detection methods. The former is highly susceptible to variable scenes, so its accuracy will be greatly reduced when detecting water surface objects due to the changing of the sunlight and waves. The latter is more sensitive to the selection of object features, which will lead to poor generalization as a result, so it cannot be applied widely. Consequently, methods based on deep learning have recently been proposed. The River Chief System has been implemented in China recently, and one of the important requirements is to detect and deal with the water surface floats in a timely fashion. In response to this case, we propose a real-time water surface object detection method in this paper which is based on the Faster R-CNN. The proposed network model includes two modules and integrates low-level features with high-level features to improve detection accuracy. Moreover, we propose to set the different scales and aspect ratios of anchors by analyzing the distribution of object scales in our dataset, so our method has good robustness and high detection accuracy for multi-scale objects in complex natural scenes. We utilized the proposed method to detect the floats on the water surface via a three-day video surveillance stream of the North Canal in Beijing, and validated its performance. The experiments show that the mean average precision (MAP) of the proposed method was 83.7%, and the detection speed was 13 frames per second. Therefore, our method can be applied in complex natural scenes and mostly meets the requirements of accuracy and speed of water surface object detection online.

Download Full-text

Small Object Detection in Traffic Scenes Based on Attention Feature Fusion

Sensors ◽

10.3390/s21093031 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3031

Author(s):

Jing Lian ◽

Yuhang Yin ◽

Linhui Li ◽

Zhenghao Wang ◽

Yafu Zhou

Keyword(s):

Object Detection ◽

Feature Fusion ◽

Contextual Information ◽

Detection Accuracy ◽

Small Object ◽

Limited Information ◽

Feature Maps ◽

Multi Scale ◽

Validation Set ◽

Small Object Detection

There are many small objects in traffic scenes, but due to their low resolution and limited information, their detection is still a challenge. Small object detection is very important for the understanding of traffic scene environments. To improve the detection accuracy of small objects in traffic scenes, we propose a small object detection method in traffic scenes based on attention feature fusion. First, a multi-scale channel attention block (MS-CAB) is designed, which uses local and global scales to aggregate the effective information of the feature maps. Based on this block, an attention feature fusion block (AFFB) is proposed, which can better integrate contextual information from different layers. Finally, the AFFB is used to replace the linear fusion module in the object detection network and obtain the final network structure. The experimental results show that, compared to the benchmark model YOLOv5s, this method has achieved a higher mean Average Precison (mAP) under the premise of ensuring real-time performance. It increases the mAP of all objects by 0.9 percentage points on the validation set of the traffic scene dataset BDD100K, and at the same time, increases the mAP of small objects by 3.5%.

Download Full-text

Subtask Attention Based Object Detection in Remote Sensing Images

Remote Sensing ◽

10.3390/rs13101925 ◽

2021 ◽

Vol 13 (10) ◽

pp. 1925

Author(s):

Shengzhou Xiong ◽

Yihua Tan ◽

Yansheng Li ◽

Cai Wen ◽

Pei Yan

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Feature Fusion ◽

Detection Task ◽

Feature Representation ◽

Detection Accuracy ◽

Remote Sensing Images ◽

Attention Network ◽

Multi Scale ◽

Automatic Interpretation

Object detection in remote sensing images (RSIs) is one of the basic tasks in the field of remote sensing image automatic interpretation. In recent years, the deep object detection frameworks of natural scene images (NSIs) have been introduced into object detection on RSIs, and the detection performance has improved significantly because of the powerful feature representation. However, there are still many challenges concerning the particularities of remote sensing objects. One of the main challenges is the missed detection of small objects which have less than five percent of the pixels of the big objects. Generally, the existing algorithms choose to deal with this problem by multi-scale feature fusion based on a feature pyramid. However, the benefits of this strategy are limited, considering that the location of small objects in the feature map will disappear when the detection task is processed at the end of the network. In this study, we propose a subtask attention network (StAN), which handles the detection task directly on the shallow layer of the network. First, StAN contains one shared feature branch and two subtask attention branches of a semantic auxiliary subtask and a detection subtask based on the multi-task attention network (MTAN). Second, the detection branch uses only low-level features considering small objects. Third, the attention map guidance mechanism is put forward to optimize the network for keeping the identification ability. Fourth, the multi-dimensional sampling module (MdS), global multi-view channel weights (GMulW) and target-guided pixel attention (TPA) are designed for further improvement of the detection accuracy in complex scenes. The experimental results on the NWPU VHR-10 dataset and DOTA dataset demonstrated that the proposed algorithm achieved the SOTA performance, and the missed detection of small objects decreased. On the other hand, ablation experiments also proved the effects of MdS, GMulW and TPA.

Download Full-text

Small Object Detection in Traffic Scenes Based on YOLO-MXANet

Sensors ◽

10.3390/s21217422 ◽

2021 ◽

Vol 21 (21) ◽

pp. 7422

Author(s):

Xiaowei He ◽

Rao Cheng ◽

Zhonglong Zheng ◽

Zeji Wang

Keyword(s):

Object Detection ◽

Spatial Attention ◽

Training Model ◽

Activation Function ◽

Model Complexity ◽

Detection Accuracy ◽

Small Object ◽

Enhancement Method ◽

Detection Speed ◽

Improved Algorithm

In terms of small objects in traffic scenes, general object detection algorithms have low detection accuracy, high model complexity, and slow detection speed. To solve the above problems, an improved algorithm (named YOLO-MXANet) is proposed in this paper. Complete-Intersection over Union (CIoU) is utilized to improve loss function for promoting the positioning accuracy of the small object. In order to reduce the complexity of the model, we present a lightweight yet powerful backbone network (named SA-MobileNeXt) that incorporates channel and spatial attention. Our approach can extract expressive features more effectively by applying the Shuffle Channel and Spatial Attention (SCSA) module into the SandGlass Block (SGBlock) module while increasing the parameters by a small number. In addition, the data enhancement method combining Mosaic and Mixup is employed to improve the robustness of the training model. The Multi-scale Feature Enhancement Fusion (MFEF) network is proposed to fuse the extracted features better. In addition, the SiLU activation function is utilized to optimize the Convolution-Batchnorm-Leaky ReLU (CBL) module and the SGBlock module to accelerate the convergence of the model. The ablation experiments on the KITTI dataset show that each improved method is effective. The improved algorithm reduces the complexity and detection speed of the model while improving the object detection accuracy. The comparative experiments on the KITTY dataset and CCTSDB dataset with other algorithms show that our algorithm also has certain advantages.

Download Full-text

Immature Apple Detection Method Based on Improved Yolov3

ASP Transactions on Internet of Things ◽

10.52810/10.52810/tiot.2021.100028 ◽

2021 ◽

Vol 1 (1) ◽

pp. 9-13

Author(s):

Zhongqiang Huang ◽

Ping Zhang ◽

Ruigang Liu ◽

Dongxu Li

Keyword(s):

Decision Making ◽

Feasible Solution ◽

Detection Method ◽

Detection Accuracy ◽

Data Set ◽

Apple Industry ◽

Backbone Network ◽

Target Frame ◽

Detection Speed ◽

Detection Effect

The identification of immature apples is a key technical link to realize automatic real-time monitoring of orchards, expert decision-making, and realization of orchard output prediction. In the orchard scene, the reflection caused by light and the color of immature apples are highly similar to the leaves, especially the obscuration and overlap of fruits by leaves and branches, which brings great challenges to the detection of immature apples. This paper proposes an improved YOLOv3 detection method for immature apples in the orchard scene. Use CSPDarknet53 as the backbone network of the model, introduce the CIOU target frame regression mechanism, and combine with the Mosaic algorithm to improve the detection accuracy. For the data set with severely occluded fruits, the F1 and mAP of the immature apple recognition model proposed in this article are 0.652 and 0.675, respectively. The inference speed for a single 416×416 picture is 12 ms, the detection speed can reach 83 frames/s on 1080ti, and the inference speed is 8.6 ms. Therefore, for the severely occluded immature apple data set, the method proposed in this article has a significant detection effect, and provides a feasible solution for the automation and mechanization of the apple industry.

Download Full-text