scholarly journals RGDiNet: Efficient Onboard Object Detection with Faster R-CNN for Air-to-Ground Surveillance

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1677
Author(s):  
Jongwon Kim ◽  
Jeongho Cho
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Environmental Changes ◽  
Depth Map ◽  
Operating Conditions ◽  
Superior Performance ◽  
Aerial Image ◽  
Detection Accuracy ◽  
Time Data ◽  
Cloud Data

An essential component for the autonomous flight or air-to-ground surveillance of a UAV is an object detection device. It must possess a high detection accuracy and requires real-time data processing to be employed for various tasks such as search and rescue, object tracking and disaster analysis. With the recent advancements in multimodal data-based object detection architectures, autonomous driving technology has significantly improved, and the latest algorithm has achieved an average precision of up to 96%. However, these remarkable advances may be unsuitable for the image processing of UAV aerial data directly onboard for object detection because of the following major problems: (1) Objects in aerial views generally have a smaller size than in an image and they are uneven and sparsely distributed throughout an image; (2) Objects are exposed to various environmental changes, such as occlusion and background interference; and (3) The payload weight of a UAV is limited. Thus, we propose employing a new real-time onboard object detection architecture, an RGB aerial image and a point cloud data (PCD) depth map image network (RGDiNet). A faster region-based convolutional neural network was used as the baseline detection network and an RGD, an integration of the RGB aerial image and the depth map reconstructed by the light detection and ranging PCD, was utilized as an input for computational efficiency. Performance tests and evaluation of the proposed RGDiNet were conducted under various operating conditions using hand-labeled aerial datasets. Consequently, it was shown that the proposed method has a superior performance for the detection of vehicles and pedestrians than conventional vision-based methods.

scholarly journals Multiple-Oriented and Small Object Detection with Convolutional Neural Networks for Aerial Image

2019 ◽  
Vol 11 (18) ◽  
pp. 2176 ◽  
Author(s):  
Chen ◽  
Zhong ◽  
Tan
Keyword(s):  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Networks ◽  
Aerial Images ◽  
Superior Performance ◽  
Aerial Image ◽  
Detection Accuracy ◽  
Small Object ◽  
Data Set ◽  
Orientation Information

Detecting objects in aerial images is a challenging task due to multiple orientations and relatively small size of the objects. Although many traditional detection models have demonstrated an acceptable performance by using the imagery pyramid and multiple templates in a sliding-window manner, such techniques are inefficient and costly. Recently, convolutional neural networks (CNNs) have successfully been used for object detection, and they have demonstrated considerably superior performance than that of traditional detection methods; however, this success has not been expanded to aerial images. To overcome such problems, we propose a detection model based on two CNNs. One of the CNNs is designed to propose many object-like regions that are generated from the feature maps of multi scales and hierarchies with the orientation information. Based on such a design, the positioning of small size objects becomes more accurate, and the generated regions with orientation information are more suitable for the objects arranged with arbitrary orientations. Furthermore, another CNN is designed for object recognition; it first extracts the features of each generated region and subsequently makes the final decisions. The results of the extensive experiments performed on the vehicle detection in aerial imagery (VEDAI) and overhead imagery research data set (OIRDS) datasets indicate that the proposed model performs well in terms of not only the detection accuracy but also the detection speed.

scholarly journals Exploring a Multimodal Mixture-Of-YOLOs Framework for Advanced Real-Time Object Detection

2020 ◽  
Vol 10 (2) ◽  
pp. 612
Author(s):  
Jinsoo Kim ◽  
Jeongho Cho
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Environmental Changes ◽  
Detection System ◽  
Dynamic Environment ◽  
Autonomous Driving ◽  
Detection Performance ◽  
Weighted Mean ◽  
Cloud Data ◽  
Individual Object

To construct a safe and sound autonomous driving system, object detection is essential, and research on fusion of sensors is being actively conducted to increase the detection rate of objects in a dynamic environment in which safety must be secured. Recently, considerable performance improvements in object detection have been achieved with the advent of the convolutional neural network (CNN) structure. In particular, the YOLO (You Only Look Once) architecture, which is suitable for real-time object detection by simultaneously predicting and classifying bounding boxes of objects, is receiving great attention. However, securing the robustness of object detection systems in various environments still remains a challenge. In this paper, we propose a weighted mean-based adaptive object detection strategy that enhances detection performance through convergence of individual object detection results based on an RGB camera and a LiDAR (Light Detection and Ranging) for autonomous driving. The proposed system utilizes the YOLO framework to perform object detection independently based on image data and point cloud data (PCD). Each detection result is united to reduce the number of objects not detected at the decision level by the weighted mean scheme. To evaluate the performance of the proposed object detection system, tests on vehicles and pedestrians were carried out using the KITTI Benchmark Suite. Test results demonstrated that the proposed strategy can achieve detection performance with a higher mean average precision (mAP) for targeted objects than an RGB camera and is also robust against external environmental changes.

scholarly journals A Set of Single YOLO Modalities to Detect Occluded Entities via Viewpoint Conversion

2021 ◽  
Vol 11 (13) ◽  
pp. 6016
Author(s):  
Jinsoo Kim ◽  
Jeongho Cho
Keyword(s):  
Object Detection ◽  
Autonomous Vehicles ◽  
Autonomous Driving ◽  
Detection Algorithm ◽  
Detection Accuracy ◽  
Cloud Data ◽  
Detection Techniques ◽  
Bounding Boxes ◽  
Partially Occluded ◽  
Rgb Image

For autonomous vehicles, it is critical to be aware of the driving environment to avoid collisions and drive safely. The recent evolution of convolutional neural networks has contributed significantly to accelerating the development of object detection techniques that enable autonomous vehicles to handle rapid changes in various driving environments. However, collisions in an autonomous driving environment can still occur due to undetected obstacles and various perception problems, particularly occlusion. Thus, we propose a robust object detection algorithm for environments in which objects are truncated or occluded by employing RGB image and light detection and ranging (LiDAR) bird’s eye view (BEV) representations. This structure combines independent detection results obtained in parallel through “you only look once” networks using an RGB image and a height map converted from the BEV representations of LiDAR’s point cloud data (PCD). The region proposal of an object is determined via non-maximum suppression, which suppresses the bounding boxes of adjacent regions. A performance evaluation of the proposed scheme was performed using the KITTI vision benchmark suite dataset. The results demonstrate the detection accuracy in the case of integration of PCD BEV representations is superior to when only an RGB camera is used. In addition, robustness is improved by significantly enhancing detection accuracy even when the target objects are partially occluded when viewed from the front, which demonstrates that the proposed algorithm outperforms the conventional RGB-based model.

Real-Time Thermodynamic Performance Monitoring and Optimum Thermoeconomic Operation of Power Plants

2011 ◽  
Author(s):  
G. Hariharan ◽  
B. Kosanovic
Keyword(s):  
Linear Programming ◽  
Power Plant ◽  
Real Time ◽  
Power Plants ◽  
Operating Conditions ◽  
Recursive Least Squares ◽  
Time Data ◽  
Real Time Data ◽  
One Step ◽  
Inputs And Outputs

The ability of modern power plant data acquisition systems to provide a continuous real-time data feed can be exploited to carry out interesting research studies. In the first part of this study, real-time data from a power plant is used to carry out a comprehensive heat balance calculation. The calculation involves application of the first law of thermodynamics to each powerhouse component. Stoichiometric combustion principles are applied to calculate emissions from fossil fuel consuming components. Exergy analysis is carried out for all components by the combined application of the first and second laws of thermodynamics. In the second part of this study, techniques from the field of System Identification and Linear Programming are brought together in finding thermoeconomically optimum plant operating conditions one step ahead in time. This is done by first using autoregressive models to make short-term predictions of plant inputs and outputs. Then, parameter estimation using recursive least squares is used to determine the relations between the predicted inputs and outputs. The estimated parameters are used in setting up a linear programming problem which is solved using the simplex method. The end result is knowledge of thermoeconomically optimum plant inputs and outputs one step ahead in time.

scholarly journals Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Electronics ◽  
2020 ◽  
Vol 9 (3) ◽  
pp. 451 ◽  
Author(s):  
Limin Guan ◽  
Yi Chen ◽  
Guiping Wang ◽  
Xu Lei
Keyword(s):  
Real Time ◽  
Color Image ◽  
Evidence Theory ◽  
Depth Map ◽  
Vehicle Detection ◽  
Detection Accuracy ◽  
Distance Information ◽  
Detection Model ◽  
Detection Mode ◽  
Single Sensor

Vehicle detection is essential for driverless systems. However, the current single sensor detection mode is no longer sufficient in complex and changing traffic environments. Therefore, this paper combines camera and light detection and ranging (LiDAR) to build a vehicle-detection framework that has the characteristics of multi adaptability, high real-time capacity, and robustness. First, a multi-adaptive high-precision depth-completion method was proposed to convert the 2D LiDAR sparse depth map into a dense depth map, so that the two sensors are aligned with each other at the data level. Then, the You Only Look Once Version 3 (YOLOv3) real-time object detection model was used to detect the color image and the dense depth map. Finally, a decision-level fusion method based on bounding box fusion and improved Dempster–Shafer (D–S) evidence theory was proposed to merge the two results of the previous step and obtain the final vehicle position and distance information, which not only improves the detection accuracy but also improves the robustness of the whole framework. We evaluated our method using the KITTI dataset and the Waymo Open Dataset, and the results show the effectiveness of the proposed depth completion method and multi-sensor fusion strategy.

Test and Analysis of Exhaust Pollutants from Automobiles Based on GPS Technology

2010 ◽  
Vol 40-41 ◽  
pp. 675-681
Author(s):  
Ming Li Xian ◽  
Qing Huang Yong
Keyword(s):  
Real Time ◽  
Data Gathering ◽  
Operating Conditions ◽  
Vehicle Emission ◽  
Testing System ◽  
Time Data ◽  
Gps Technology ◽  
Urban Road ◽  
Real Time Data ◽  
Object Of Study

Taking the actual running vehicles on the urban roads of Ningpo City as the object of study, by using the brand-new on-vehicle automobile exhaust real-time testing system, and through actual testing by tracking the running vehicles and real-time data gathering, The paper analyzed urban road operating conditions, the vehicle emission situation on the actual roads, obtained the relations between the operating conditions, the speed and emissions and the law by which the automobile operating conditions affect the automobile exhausts.

scholarly journals Motion-Print: A Biometric for Real-Time Pilot Identification using Hierarchical Temporal Memory

2020 ◽  
Author(s):  
Sam Heiserman ◽  
Kirill Zaychik ◽  
Timothy Miller
Keyword(s):  
Real Time ◽  
Superior Performance ◽  
Time Warping ◽  
Control Task ◽  
Time Data ◽  
Temporal Memory ◽  
Hierarchical Temporal Memory ◽  
Motion Simulator ◽  
Monitoring Applications ◽  
Dynamic Time

<p>This study presents a novel biometric approach to identify operators, given only streams of their control movements within a manual control task setting. In the present task subjects control a simulated, remotely operated robotic arm, attempting to dock onto a satellite in orbit. The proposed methodology utilizes the Hierarchical Temporal Memory (HTM) algorithm to distinguish operators by their unique control behaviors. Results presented compare the identification performance of HTM with Dynamic Time Warping (DTW) and Edit Distance on Real Sequences (EDR), in both static and real-time data settings. The HTM method outperformed both DTW and EDR in the real- time setting, and matched DTW in the static setting. Observed superior performance of the HTM algorithm lays the foundation for the extension of the proposed methodology to other motion- monitoring applications, such as real-time workload assessment, motion/simulator sickness onset or distraction detection.</p><p><br></p><p>The data gathered in the study was posted to IEEE-dataport, DOI: <a href="http://dx.doi.org/10.21227/wpyf-r927" target="_blank">10.21227/wpyf-r927</a><br></p><div><br></div>

scholarly journals Ship Detection Based on YOLOv2 for SAR Imagery

2019 ◽  
Vol 11 (7) ◽  
pp. 786 ◽  
Author(s):  
Yang-Lang Chang ◽  
Amare Anagaw ◽  
Lena Chang ◽  
Yi Wang ◽  
Chih-Yu Hsiao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Real Time ◽  
Experimental Results ◽  
Detection Methods ◽  
Computational Time ◽  
Detection Accuracy ◽  
Single Shot ◽  
Ship Detection ◽  
Sar Imagery

Synthetic aperture radar (SAR) imagery has been used as a promising data source for monitoring maritime activities, and its application for oil and ship detection has been the focus of many previous research studies. Many object detection methods ranging from traditional to deep learning approaches have been proposed. However, majority of them are computationally intensive and have accuracy problems. The huge volume of the remote sensing data also brings a challenge for real time object detection. To mitigate this problem a high performance computing (HPC) method has been proposed to accelerate SAR imagery analysis, utilizing the GPU based computing methods. In this paper, we propose an enhanced GPU based deep learning method to detect ship from the SAR images. The You Only Look Once version 2 (YOLOv2) deep learning framework is proposed to model the architecture and training the model. YOLOv2 is a state-of-the-art real-time object detection system, which outperforms Faster Region-Based Convolutional Network (Faster R-CNN) and Single Shot Multibox Detector (SSD) methods. Additionally, in order to reduce computational time with relatively competitive detection accuracy, we develop a new architecture with less number of layers called YOLOv2-reduced. In the experiment, we use two types of datasets: A SAR ship detection dataset (SSDD) dataset and a Diversified SAR Ship Detection Dataset (DSSDD). These two datasets were used for training and testing purposes. YOLOv2 test results showed an increase in accuracy of ship detection as well as a noticeable reduction in computational time compared to Faster R-CNN. From the experimental results, the proposed YOLOv2 architecture achieves an accuracy of 90.05% and 89.13% on the SSDD and DSSDD datasets respectively. The proposed YOLOv2-reduced architecture has a similarly competent detection performance as YOLOv2, but with less computational time on a NVIDIA TITAN X GPU. The experimental results shows that the deep learning can make a big leap forward in improving the performance of SAR image ship detection.

scholarly journals Real-Time Water Surface Object Detection Based on Improved Faster R-CNN

Sensors ◽  
2019 ◽  
Vol 19 (16) ◽  
pp. 3523 ◽  
Author(s):  
Lili Zhang ◽  
Yi Zhang ◽  
Zhen Zhang ◽  
Jie Shen ◽  
Huibin Wang
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Water Surface ◽  
Detection Methods ◽  
Detection Accuracy ◽  
Natural Scenes ◽  
The North ◽  
Aspect Ratios ◽  
High Level ◽  
Surface Object

In this paper, we consider water surface object detection in natural scenes. Generally, background subtraction and image segmentation are the classical object detection methods. The former is highly susceptible to variable scenes, so its accuracy will be greatly reduced when detecting water surface objects due to the changing of the sunlight and waves. The latter is more sensitive to the selection of object features, which will lead to poor generalization as a result, so it cannot be applied widely. Consequently, methods based on deep learning have recently been proposed. The River Chief System has been implemented in China recently, and one of the important requirements is to detect and deal with the water surface floats in a timely fashion. In response to this case, we propose a real-time water surface object detection method in this paper which is based on the Faster R-CNN. The proposed network model includes two modules and integrates low-level features with high-level features to improve detection accuracy. Moreover, we propose to set the different scales and aspect ratios of anchors by analyzing the distribution of object scales in our dataset, so our method has good robustness and high detection accuracy for multi-scale objects in complex natural scenes. We utilized the proposed method to detect the floats on the water surface via a three-day video surveillance stream of the North Canal in Beijing, and validated its performance. The experiments show that the mean average precision (MAP) of the proposed method was 83.7%, and the detection speed was 13 frames per second. Therefore, our method can be applied in complex natural scenes and mostly meets the requirements of accuracy and speed of water surface object detection online.

Sign in / Sign up

Export Citation Format

Share Document