scholarly journals Attention Fusion for One-Stage Multispectral Pedestrian Detection

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4184
Author(s):  
Zhiwei Cao ◽  
Huihua Yang ◽  
Juan Zhao ◽  
Shuhong Guo ◽  
Lingqiao Li
Keyword(s):  
Feature Fusion ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Complementary Information ◽  
Deep Convolutional Neural Networks ◽  
One Stage ◽  
Current State ◽  
Fusion Methods ◽  
Feature Information ◽  
Bounding Boxes

Multispectral pedestrian detection, which consists of a color stream and thermal stream, is essential under conditions of insufficient illumination because the fusion of the two streams can provide complementary information for detecting pedestrians based on deep convolutional neural networks (CNNs). In this paper, we introduced and adapted a simple and efficient one-stage YOLOv4 to replace the current state-of-the-art two-stage fast-RCNN for multispectral pedestrian detection and to directly predict bounding boxes with confidence scores. To further improve the detection performance, we analyzed the existing multispectral fusion methods and proposed a novel multispectral channel feature fusion (MCFF) module for integrating the features from the color and thermal streams according to the illumination conditions. Moreover, several fusion architectures, such as Early Fusion, Halfway Fusion, Late Fusion, and Direct Fusion, were carefully designed based on the MCFF to transfer the feature information from the bottom to the top at different stages. Finally, the experimental results on the KAIST and Utokyo pedestrian benchmarks showed that Halfway Fusion was used to obtain the best performance of all architectures and the MCFF could adapt fused features in the two modalities. The log-average miss rate (MR) for the two modalities with reasonable settings were 4.91% and 23.14%, respectively.

scholarly journals Adaptive Weighted Multi-Level Fusion of Multi-Scale Features: A New Approach to Pedestrian Detection

2021 ◽  
Vol 13 (2) ◽  
pp. 38
Author(s):  
Yao Xu ◽  
Qin Yu
Keyword(s):  
Deep Learning ◽  
Feature Fusion ◽  
Pedestrian Detection ◽  
Feature Maps ◽  
Scale Feature ◽  
Multi Scale ◽  
One Stage ◽  
Current State ◽  
Multi Level ◽  
Feature Utilization

Great achievements have been made in pedestrian detection through deep learning. For detectors based on deep learning, making better use of features has become the key to their detection effect. While current pedestrian detectors have made efforts in feature utilization to improve their detection performance, the feature utilization is still inadequate. To solve the problem of inadequate feature utilization, we proposed the Multi-Level Feature Fusion Module (MFFM) and its Multi-Scale Feature Fusion Unit (MFFU) sub-module, which connect feature maps of the same scale and different scales by using horizontal and vertical connections and shortcut structures. All of these connections are accompanied by weights that can be learned; thus, they can be used as adaptive multi-level and multi-scale feature fusion modules to fuse the best features. Then, we built a complete pedestrian detector, the Adaptive Feature Fusion Detector (AFFDet), which is an anchor-free one-stage pedestrian detector that can make full use of features for detection. As a result, compared with other methods, our method has better performance on the challenging Caltech Pedestrian Detection Benchmark (Caltech) and has quite competitive speed. It is the current state-of-the-art one-stage pedestrian detection method.

scholarly journals A Robust Context-Based Deep Learning Approach for Highly Imbalanced Hyperspectral Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Juan F. Ramirez Rochac ◽  
Nian Zhang ◽  
Lara A. Thompson ◽  
Tolessa Deksissa
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Mineral Exploration ◽  
Classification Models ◽  
Noise Resistance ◽  
Deep Convolutional Neural Networks ◽  
Current State ◽  
Feature Augmentation ◽  
Active Research ◽  
Hyperspectral Classification

Hyperspectral imaging is an area of active research with many applications in remote sensing, mineral exploration, and environmental monitoring. Deep learning and, in particular, convolution-based approaches are the current state-of-the-art classification models. However, in the presence of noisy hyperspectral datasets, these deep convolutional neural networks underperform. In this paper, we proposed a feature augmentation approach to increase noise resistance in imbalanced hyperspectral classification. Our method calculates context-based features, and it uses a deep convolutional neuronet (DCN). We tested our proposed approach on the Pavia datasets and compared three models, DCN, PCA + DCN, and our context-based DCN, using the original datasets and the datasets plus noise. Our experimental results show that DCN and PCA + DCN perform well on the original datasets but not on the noisy datasets. Our robust context-based DCN was able to outperform others in the presence of noise and was able to maintain a comparable classification accuracy on clean hyperspectral images.

scholarly journals AI-driven deep CNN approach for multi-label pathology classification using chest X-Rays

2021 ◽  
Vol 7 ◽  
pp. e495
Author(s):  
Saleh Albahli ◽  
Hafiz Tayyab Rauf ◽  
Abdulelah Algosaibi ◽  
Valentina Emilia Balas
Keyword(s):  
Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Synthetic Data ◽  
X Rays ◽  
Deep Convolutional Neural Networks ◽  
Current State ◽  
Pathology Classification ◽  
Wide Range ◽  
Multi Class Classification

Artificial intelligence (AI) has played a significant role in image analysis and feature extraction, applied to detect and diagnose a wide range of chest-related diseases. Although several researchers have used current state-of-the-art approaches and have produced impressive chest-related clinical outcomes, specific techniques may not contribute many advantages if one type of disease is detected without the rest being identified. Those who tried to identify multiple chest-related diseases were ineffective due to insufficient data and the available data not being balanced. This research provides a significant contribution to the healthcare industry and the research community by proposing a synthetic data augmentation in three deep Convolutional Neural Networks (CNNs) architectures for the detection of 14 chest-related diseases. The employed models are DenseNet121, InceptionResNetV2, and ResNet152V2; after training and validation, an average ROC-AUC score of 0.80 was obtained competitive as compared to the previous models that were trained for multi-class classification to detect anomalies in x-ray images. This research illustrates how the proposed model practices state-of-the-art deep neural networks to classify 14 chest-related diseases with better accuracy.

scholarly journals Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

2018 ◽  
Vol 232 ◽  
pp. 01061
Author(s):  
Danhua Li ◽  
Xiaofeng Di ◽  
Xuan Qu ◽  
Yunfei Zhao ◽  
Honggang Kong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Deep Convolutional Neural Network ◽  
High Quality ◽  
Low Level ◽  
Features Fusion ◽  
Current State ◽  
High Level

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

scholarly journals Exploring the Benefits of Depth Information in Object Pixel Masking (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13833-13834
Author(s):  
Anish Kachinthaya ◽  
Yi Ding ◽  
Tobias Hollerer
Keyword(s):  
State Of The Art ◽  
Depth Information ◽  
Spatial Awareness ◽  
Multiple Objects ◽  
Depth Data ◽  
Current State ◽  
Complex Scenes ◽  
Bounding Boxes

In this paper, we look at how depth data can benefit existing object masking methods applied in occluded scenes. Masking the pixel locations of objects within scenes helps computers get a spatial awareness of where objects are within images. The current state-of-the-art algorithm for masking objects in images is Mask R-CNN, which builds on the Faster R-CNN network to mask object pixels rather than just detecting their bounding boxes. This paper examines the weaknesses Mask R-CNN has in masking people when they are occluded in a frame. It then looks at how depth data gathered from an RGB-D sensor can be used. We provide a case study to show how simply applying thresholding methods on the depth information can aid in distinguishing occluded persons. The intention of our research is to examine how features from depth data can benefit object pixel masking methods in an explainable manner, especially in complex scenes with multiple objects.

Efficient Hair Rendering with a GPU Cone Tracing Approach

2017 ◽  
Vol 8 (1) ◽  
pp. 1-19
Author(s):  
Jorge R. Martins ◽  
Vasco S. Costa ◽  
João M. Pereira
Keyword(s):  
Human Hair ◽  
State Of The Art ◽  
Sampling Rate ◽  
Current State ◽  
Bounding Volume ◽  
Bounding Boxes ◽  
Bounding Volume Hierarchies ◽  
Bounding Volume Hierarchy

Rendering human hair can be a hard task because of the required high super-sampling rate to render thin hair fibers without noticeable aliasing. Additionally, the current state-of-the-art bounding volume hierarchies (BVHs) are not suitable to hair rendering. In fact, the axis-aligned bounding boxes (AABBs) do not tightly bind hair primitives which impacts negatively the intersection tests activity. Both limitations can degrade severely the rendering performance so described in this article, a cone tracing GPU approach coupled with a hybrid bounding volume hierarchy to tackle these problems. The hybrid BVH makes use of both oriented and axis aligned bounding boxes. It is shown that the experiment is able to drastically reduce the super-sampling required to produce aliasing free images while minimizing the number of intersection tests and achieving speedups of up to 4, depending on the scene.

scholarly journals Occluded Pedestrian Detection Techniques by Deformable Attention-Guided Network (DAGN)

2021 ◽  
Vol 11 (13) ◽  
pp. 6025
Author(s):  
Han Xie ◽  
Wenqi Zheng ◽  
Hyunchul Shin
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Tone Mapping ◽  
Context Information ◽  
Global Context ◽  
Detection Techniques ◽  
Low Visibility ◽  
Bounding Boxes ◽  
Preprocessing Technique

Although many deep-learning-based methods have achieved considerable detection performance for pedestrians with high visibility, their overall performances are still far from satisfactory, especially when heavily occluded instances are included. In this research, we have developed a novel pedestrian detector using a deformable attention-guided network (DAGN). Considering that pedestrians may be deformed with occlusions or under diverse poses, we have designed a deformable convolution with an attention module (DCAM) to sample from non-rigid locations, and obtained the attention feature map by aggregating global context information. Furthermore, the loss function was optimized to get accurate detection bounding boxes, by adopting complete-IoU loss for regression, and the distance IoU-NMS was used to refine the predicted boxes. Finally, a preprocessing technique based on tone mapping was applied to cope with the low visibility cases due to poor illumination. Extensive evaluations were conducted on three popular traffic datasets. Our method could decrease the log-average miss rate (MR−2) by 12.44% and 7.8%, respectively, for the heavy occlusion and overall cases, when compared to the published state-of-the-art results of the Caltech pedestrian dataset. Of the CityPersons and EuroCity Persons datasets, our proposed method outperformed the current best results by about 5% in MR−2 for the heavy occlusion cases.

scholarly journals A Lightweight YOLOv4-Based Forestry Pest Detection Method Using Coordinate Attention and Feature Fusion

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1587
Author(s):  
Mingfeng Zha ◽  
Wenbin Qian ◽  
Wenlong Yi ◽  
Jing Hua
Keyword(s):  
Detection Method ◽  
Feature Fusion ◽  
State Of The Art ◽  
Detection Methods ◽  
Model Parameters ◽  
Symmetric Structure ◽  
Proposed Model ◽  
Pest Detection ◽  
Feature Information ◽  
Small Targets

Traditional pest detection methods are challenging to use in complex forestry environments due to their low accuracy and speed. To address this issue, this paper proposes the YOLOv4_MF model. The YOLOv4_MF model utilizes MobileNetv2 as the feature extraction block and replaces the traditional convolution with depth-wise separated convolution to reduce the model parameters. In addition, the coordinate attention mechanism was embedded in MobileNetv2 to enhance feature information. A symmetric structure consisting of a three-layer spatial pyramid pool is presented, and an improved feature fusion structure was designed to fuse the target information. For the loss function, focal loss was used instead of cross-entropy loss to enhance the network’s learning of small targets. The experimental results showed that the YOLOv4_MF model has 4.24% higher mAP, 4.37% higher precision, and 6.68% higher recall than the YOLOv4 model. The size of the proposed model was reduced to 1/6 of that of YOLOv4. Moreover, the proposed algorithm achieved 38.62% mAP with respect to some state-of-the-art algorithms on the COCO dataset.

scholarly journals Video Face Super-Resolution with Motion-Adaptive Feedback Cell

2020 ◽  
Vol 34 (07) ◽  
pp. 12468-12475
Author(s):  
Jingwei Xin ◽  
Nannan Wang ◽  
Jie Li ◽  
Xinbo Gao ◽  
Zhifeng Li
Keyword(s):  
Motion Estimation ◽  
State Of The Art ◽  
Super Resolution ◽  
Complex Motion ◽  
Deep Convolutional Neural Networks ◽  
Adaptive Feedback ◽  
Current State ◽  
Model Complex ◽  
Frame Motion ◽  
Inter Frame

Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN). Current state-of-the-art CNN methods usually treat the VSR problem as a large number of separate multi-frame super-resolution tasks, at which a batch of low resolution (LR) frames is utilized to generate a single high resolution (HR) frame, and running a slide window to select LR frames over the entire video would obtain a series of HR frames. However, duo to the complex temporal dependency between frames, with the number of LR input frames increase, the performance of the reconstructed HR frames become worse. The reason is in that these methods lack the ability to model complex temporal dependencies and hard to give an accurate motion estimation and compensation for VSR process. Which makes the performance degrade drastically when the motion in frames is complex. In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way. Our approach efficiently utilizes the information of the inter-frame motion, the dependence of the network on motion estimation and compensation method can be avoid. In addition, benefiting from the excellent nature of MAFC, the network can achieve better performance in the case of extremely complex motion scenarios. Extensive evaluations and comparisons validate the strengths of our approach, and the experimental results demonstrated that the proposed framework is outperform the state-of-the-art methods.

scholarly journals M2Det: A Single-Shot Object Detector Based on Multi-Level Feature Pyramid Network

2019 ◽  
Vol 33 ◽  
pp. 9259-9266 ◽  
Author(s):  
Qijie Zhao ◽  
Tao Sheng ◽  
Yongtao Wang ◽  
Zhi Tang ◽  
Ying Chen ◽  
...  
Keyword(s):  
Feature Fusion ◽  
State Of The Art ◽  
Single Shot ◽  
Multi Scale ◽  
One Stage ◽  
Single Scale ◽  
Feature Pyramid ◽  
Multi Level ◽  
Multiple Levels ◽  
Inference Strategy

Feature pyramids are widely exploited by both the state-of-the-art one-stage object detectors (e.g., DSSD, RetinaNet, RefineDet) and the two-stage object detectors (e.g., Mask RCNN, DetNet) to alleviate the problem arising from scale variation across object instances. Although these object detectors with feature pyramids achieve encouraging results, they have some limitations due to that they only simply construct the feature pyramid according to the inherent multiscale, pyramidal architecture of the backbones which are originally designed for object classification task. Newly, in this work, we present Multi-Level Feature Pyramid Network (MLFPN) to construct more effective feature pyramids for detecting objects of different scales. First, we fuse multi-level features (i.e. multiple layers) extracted by backbone as the base feature. Second, we feed the base feature into a block of alternating joint Thinned U-shape Modules and Feature Fusion Modules and exploit the decoder layers of each Ushape module as the features for detecting objects. Finally, we gather up the decoder layers with equivalent scales (sizes) to construct a feature pyramid for object detection, in which every feature map consists of the layers (features) from multiple levels. To evaluate the effectiveness of the proposed MLFPN, we design and train a powerful end-to-end one-stage object detector we call M2Det by integrating it into the architecture of SSD, and achieve better detection performance than state-of-the-art one-stage detectors. Specifically, on MSCOCO benchmark, M2Det achieves AP of 41.0 at speed of 11.8 FPS with single-scale inference strategy and AP of 44.2 with multi-scale inference strategy, which are the new stateof-the-art results among one-stage detectors. The code will be made available on https://github.com/qijiezhao/M2Det.

Sign in / Sign up

Export Citation Format

Share Document