Computer Vision for Autonomous UAV Flight Safety: An Overview and a Vision-based Safe Landing Pipeline Example

Recent years have seen an unprecedented spread of Unmanned Aerial Vehicles (UAVs, or “drones”), which are highly useful for both civilian and military applications. Flight safety is a crucial issue in UAV navigation, having to ensure accurate compliance with recently legislated rules and regulations. The emerging use of autonomous drones and UAV swarms raises additional issues, making it necessary to transfuse safety- and regulations-awareness to relevant algorithms and architectures. Computer vision plays a pivotal role in such autonomous functionalities. Although the main aspects of autonomous UAV technologies (e.g., path planning, navigation control, landing control, mapping and localization, target detection/tracking) are already mature and well-covered, ensuring safe flying in the vicinity of crowds, avoidance of passing over persons, or guaranteed emergency landing capabilities in case of malfunctions, are generally treated as an afterthought when designing autonomous UAV platforms for unstructured environments. This fact is reflected in the fragmentary coverage of the above issues in current literature. This overview attempts to remedy this situation, from the point of view of computer vision. It examines the field from multiple aspects, including regulations across the world and relevant current technologies. Finally, since very few attempts have been made so far towards a complete UAV safety flight and landing pipeline, an example computer vision-based UAV flight safety pipeline is introduced, taking into account all issues present in current autonomous drones. The content is relevant to any kind of autonomous drone flight (e.g., for movie/TV production, news-gathering, search and rescue, surveillance, inspection, mapping, wildlife monitoring, crowd monitoring/management), making this a topic of broad interest.

A Social Distance Estimation and Crowd Monitoring System for Surveillance Cameras

Social distancing is crucial to restrain the spread of diseases such as COVID-19, but complete adherence to safety guidelines is not guaranteed. Monitoring social distancing through mass surveillance is paramount to develop appropriate mitigation plans and exit strategies. Nevertheless, it is a labor-intensive task that is prone to human error and tainted with plausible breaches of privacy. This paper presents a privacy-preserving adaptive social distance estimation and crowd monitoring solution for camera surveillance systems. We develop a novel person localization strategy through pose estimation, build a privacy-preserving adaptive smoothing and tracking model to mitigate occlusions and noisy/missing measurements, compute inter-personal distances in the real-world coordinates, detect social distance infractions, and identify overcrowded regions in a scene. Performance evaluation is carried out by testing the system’s ability in person detection, localization, density estimation, anomaly recognition, and high-risk areas identification. We compare the proposed system to the latest techniques and examine the performance gain delivered by the localization and smoothing/tracking algorithms. Experimental results indicate a considerable improvement, across different metrics, when utilizing the developed system. In addition, they show its potential and functionality for applications other than social distancing.

Unmanned Aerial Vehicles for Crowd Monitoring and Analysis

Crowd monitoring and analysis has become increasingly used for unmanned aerial vehicle applications. From preventing stampede in high concentration crowds to estimating crowd density and to surveilling crowd movements, crowd monitoring and analysis have long been employed in the past by authorities and regulatory bodies to tackle challenges posed by large crowds. Conventional methods of crowd analysis using static cameras are limited due to their low coverage area and non-flexible perspectives and features. Unmanned aerial vehicles have tremendously increased the quality of images obtained for crowd analysis reasons, relieving the relevant authorities of the venues’ inadequacies and of concerns of inaccessible locations and situation. This paper reviews existing literature sources regarding the use of aerial vehicles for crowd monitoring and analysis purposes. Vehicle specifications, onboard sensors, power management, and an analysis algorithm are critically reviewed and discussed. In addition, ethical and privacy issues surrounding the use of this technology are presented.

Background Invariant Faster Motion Modeling for Drone Action Recognition

Visual data collected from drones has opened a new direction for surveillance applications and has recently attracted considerable attention among computer vision researchers. Due to the availability and increasing use of the drone for both public and private sectors, it is a critical futuristic technology to solve multiple surveillance problems in remote areas. One of the fundamental challenges in recognizing crowd monitoring videos’ human action is the precise modeling of an individual’s motion feature. Most state-of-the-art methods heavily rely on optical flow for motion modeling and representation, and motion modeling through optical flow is a time-consuming process. This article underlines this issue and provides a novel architecture that eliminates the dependency on optical flow. The proposed architecture uses two sub-modules, FMFM (faster motion feature modeling) and AAR (accurate action recognition), to accurately classify the aerial surveillance action. Another critical issue in aerial surveillance is a deficiency of the dataset. Out of few datasets proposed recently, most of them have multiple humans performing different actions in the same scene, such as a crowd monitoring video, and hence not suitable for directly applying to the training of action recognition models. Given this, we have proposed a novel dataset captured from top view aerial surveillance that has a good variety in terms of actors, daytime, and environment. The proposed architecture has shown the capability to be applied in different terrain as it removes the background before using the action recognition model. The proposed architecture is validated through the experiment with varying investigation levels and achieves a remarkable performance of 0.90 validation accuracy in aerial action recognition.

Estimation of Crowd Density from UAV Images based on Deep Learning

Crowd monitoring is necessary to improve safety and controllable movements to minimize risk, especially in high crowded events, such as Kumbh Mela, political rallies, sports event etc. In this current digital age mostly crowd monitoring still relies on outdated methods such as keeping records, using people counters manually, and using sensors to count people at the entrance. These approaches are futile in situations where people's movements are completely unpredictable, highly variable, and complex. Crowd surveillance using unmanned aerial vehicles (UAVs), can help us solve these problems. The proposed paper uses a UAV on which an IP Camera will be attached to get media, we then use a convolutional neural network to learn a regression model for crowd counting, the model will be trained extensively by using three widely used crowd counting datasets, ShanghaiTech part A and part B, UCF-CC 50 and UCF-QNRF.

Smart System for Crowd Monitoring and Detection to Prevent Covid19

It is essential to maintain social distance and avoid large mob gatherings at one place to break the chain of corona virus infection, but maintaining these things is not that much easy. People knowingly or unknowingly, gather, roam on the streets & break the rules. Hence Keeping an eye on all these activities is not an easy job. The proposed system is an automatic method for controlling crowd in this pandemic situation, where crowd gatherings should be avoided on large basis. We have proposed a system which will keep a watchful eye on crowd gathering with help of RPi camera, as crowd is detected the system will give a alert to authorities that they will take actions against the crowd gatherings and restrict the public from areas where crowds are restricted to be gather. The main aim of the survey is to be found how to avoid unnecessary gatherings where crowd is restricted! and if crowd gathers unnecessarily, system can alert the respective authorities and crowd can be minimized.