GAN-Holo: Generative Adversarial Networks-Based Generated Holography Using Deep Learning

Current development in a deep neural network (DNN) has given an opportunity to a novel framework for the reconstruction of a holographic image and a phase recovery method with real-time performance. There are many deep learning-based techniques that have been proposed for the holographic image reconstruction, but these deep learning-based methods can still lack in performance, time complexity, accuracy, and real-time performance. Due to iterative calculation, the generation of a CGH requires a long computation time. A novel deep generative adversarial network holography (GAN-Holo) framework is proposed for hologram reconstruction. This novel framework consists of two phases. In phase one, we used the Fresnel-based method to make the dataset. In the second phase, we trained the raw input image and holographic label image data from phase one acquired images. Our method has the capability of the noniterative process of computer-generated holograms (CGHs). The experimental results have demonstrated that the proposed method outperforms the existing methods.

Download Full-text

Audio-Based Drone Detection and Identification Using Deep Learning Techniques with Dataset Enhancement through Generative Adversarial Networks

Sensors ◽

10.3390/s21154953 ◽

2021 ◽

Vol 21 (15) ◽

pp. 4953

Author(s):

Sara Al-Emadi ◽

Abdulla Al-Ali ◽

Abdulaziz Al-Ali

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recurrent Neural Network ◽

Learning Algorithms ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Adversarial Networks ◽

Detection And Identification ◽

Learning Techniques ◽

The Impact

Drones are becoming increasingly popular not only for recreational purposes but in day-to-day applications in engineering, medicine, logistics, security and others. In addition to their useful applications, an alarming concern in regard to the physical infrastructure security, safety and privacy has arisen due to the potential of their use in malicious activities. To address this problem, we propose a novel solution that automates the drone detection and identification processes using a drone’s acoustic features with different deep learning algorithms. However, the lack of acoustic drone datasets hinders the ability to implement an effective solution. In this paper, we aim to fill this gap by introducing a hybrid drone acoustic dataset composed of recorded drone audio clips and artificially generated drone audio samples using a state-of-the-art deep learning technique known as the Generative Adversarial Network. Furthermore, we examine the effectiveness of using drone audio with different deep learning algorithms, namely, the Convolutional Neural Network, the Recurrent Neural Network and the Convolutional Recurrent Neural Network in drone detection and identification. Moreover, we investigate the impact of our proposed hybrid dataset in drone detection. Our findings prove the advantage of using deep learning techniques for drone detection and identification while confirming our hypothesis on the benefits of using the Generative Adversarial Networks to generate real-like drone audio clips with an aim of enhancing the detection of new and unfamiliar drones.

Download Full-text

Generative Adversarial Networks-Based Semi-Supervised Automatic Modulation Recognition for Cognitive Radio Networks

Sensors ◽

10.3390/s18113913 ◽

2018 ◽

Vol 18 (11) ◽

pp. 3913 ◽

Cited By ~ 6

Author(s):

Mingxuan Li ◽

Ou Li ◽

Guangyi Liu ◽

Ce Zhang

Keyword(s):

Deep Learning ◽

Cognitive Radio ◽

Supervised Learning ◽

Rapid Development ◽

Generative Adversarial Networks ◽

Modulation Recognition ◽

Learning Methods ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Automatic Modulation Recognition

With the recently explosive growth of deep learning, automatic modulation recognition has undergone rapid development. Most of the newly proposed methods are dependent on large numbers of labeled samples. We are committed to using fewer labeled samples to perform automatic modulation recognition in the cognitive radio domain. Here, a semi-supervised learning method based on adversarial training is proposed which is called signal classifier generative adversarial network. Most of the prior methods based on this technology involve computer vision applications. However, we improve the existing network structure of a generative adversarial network by adding the encoder network and a signal spatial transform module, allowing our framework to address radio signal processing tasks more efficiently. These two technical improvements effectively avoid nonconvergence and mode collapse problems caused by the complexity of the radio signals. The results of simulations show that compared with well-known deep learning methods, our method improves the classification accuracy on a synthetic radio frequency dataset by 0.1% to 12%. In addition, we verify the advantages of our method in a semi-supervised scenario and obtain a significant increase in accuracy compared with traditional semi-supervised learning methods.

Download Full-text

INFRASTRUCTURE DEGRADATION AND POST-DISASTER DAMAGE DETECTION USING ANOMALY DETECTING GENERATIVE ADVERSARIAL NETWORKS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-2-2020-573-2020 ◽

2020 ◽

Vol V-2-2020 ◽

pp. 573-582 ◽

Cited By ~ 1

Author(s):

S. M. Tilon ◽

F. Nex ◽

D. Duarte ◽

N. Kerle ◽

G. Vosselman

Keyword(s):

Deep Learning ◽

Damage Detection ◽

Training Data ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Essential Information ◽

Urban Scenes ◽

Adversarial Network ◽

Collapsed Buildings ◽

Post Disaster

Abstract. Degradation and damage detection provides essential information to maintenance workers in routine monitoring and to first responders in post-disaster scenarios. Despite advance in Earth Observation (EO), image analysis and deep learning techniques, the quality and quantity of training data for deep learning is still limited. As a result, no robust method has been found yet that can transfer and generalize well over a variety of geographic locations and typologies of damages. Since damages can be seen as anomalies, occurring sparingly over time and space, we propose to use an anomaly detecting Generative Adversarial Network (GAN) to detect damages. The main advantages of using GANs are that only healthy unannotated images are needed, and that a variety of damages, including the never before seen damage, can be detected. In this study we aimed to investigate 1) the ability of anomaly detecting GANs to detect degradation (potholes and cracks) in asphalt road infrastructures using Mobile Mapper imagery and building damage (collapsed buildings, rubble piles) using post-disaster aerial imagery, and 2) the sensitivity of this method against various types of pre-processing. Our results show that we can detect damages in urban scenes at satisfying levels but not on asphalt roads. Future work will investigate how to further classify the found damages and how to improve damage detection for asphalt roads.

Download Full-text

Upsampling Real-Time, Low-Resolution CCTV Videos Using Generative Adversarial Networks

Electronics ◽

10.3390/electronics9081312 ◽

2020 ◽

Vol 9 (8) ◽

pp. 1312

Author(s):

Debapriya Hazra ◽

Yung-Cheol Byun

Keyword(s):

High Resolution ◽

Real Time ◽

Super Resolution ◽

Poor Quality ◽

Generative Adversarial Networks ◽

Temporal Consistency ◽

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Benchmark Datasets

Video super-resolution has become an emerging topic in the field of machine learning. The generative adversarial network is a framework that is widely used to develop solutions for low-resolution videos. Video surveillance using closed-circuit television (CCTV) is significant in every field, all over the world. A common problem with CCTV videos is sudden video loss or poor quality. In this paper, we propose a generative adversarial network that implements spatio-temporal generators and discriminators to enhance real-time low-resolution CCTV videos to high-resolution. The proposed model considers both foreground and background motion of a CCTV video and effectively models the spatial and temporal consistency from low-resolution video frames to generate high-resolution videos. Quantitative and qualitative experiments on benchmark datasets, including Kinetics-700, UCF101, HMDB51 and IITH_Helmet2, showed that our model outperforms the existing GAN models for video super-resolution.

Download Full-text

A Hyperspectral Image Classification Method Based on Multi-Discriminator Generative Adversarial Networks

Sensors ◽

10.3390/s19153269 ◽

2019 ◽

Vol 19 (15) ◽

pp. 3269 ◽

Cited By ~ 8

Author(s):

Hongmin Gao ◽

Dan Yao ◽

Mingxia Wang ◽

Chenming Li ◽

Haiyun Liu ◽

...

Keyword(s):

Deep Learning ◽

Hyperspectral Image ◽

Hyperspectral Data ◽

Generative Adversarial Networks ◽

Data Sets ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

The One ◽

Hyperspectral Remote Sensing Images

Hyperspectral remote sensing images (HSIs) have great research and application value. At present, deep learning has become an important method for studying image processing. The Generative Adversarial Network (GAN) model is a typical network of deep learning developed in recent years and the GAN model can also be used to classify HSIs. However, there are still some problems in the classification of HSIs. On the one hand, due to the existence of different objects with the same spectrum phenomenon, if only according to the original GAN model to generate samples from spectral samples, it will produce the wrong detailed characteristic information. On the other hand, the gradient disappears in the original GAN model and the scoring ability of a single discriminator limits the quality of the generated samples. In order to solve the above problems, we introduce the scoring mechanism of multi-discriminator collaboration and complete semi-supervised classification on three hyperspectral data sets. Compared with the original GAN model with a single discriminator, the adjusted criterion is more rigorous and accurate and the generated samples can show more accurate characteristics. Aiming at the pattern collapse and diversity deficiency of the original GAN generated by single discriminator, this paper proposes a multi-discriminator generative adversarial networks (MDGANs) and studies the influence of the number of discriminators on the classification results. The experimental results show that the introduction of multi-discriminator improves the judgment ability of the model, ensures the effect of generating samples, solves the problem of noise in generating spectral samples and can improve the classification effect of HSIs. At the same time, the number of discriminators has different effects on different data sets.

Download Full-text

Lane Detection Algorithm for Intelligent Vehicles in Complex Road Conditions and Dynamic Environments

Sensors ◽

10.3390/s19143166 ◽

2019 ◽

Vol 19 (14) ◽

pp. 3166 ◽

Cited By ~ 8

Author(s):

Cao ◽

Song ◽

Xiao ◽

Peng

Keyword(s):

Deep Learning ◽

Real Time ◽

Processing Time ◽

Dynamic Environments ◽

Detection Algorithm ◽

Intelligent Vehicles ◽

Lane Detection ◽

Detection Accuracy ◽

Traditional Methods ◽

Time Performance

Lane detection is an important foundation in the development of intelligent vehicles. To address problems such as low detection accuracy of traditional methods and poor real-time performance of deep learning-based methodologies, a lane detection algorithm for intelligent vehicles in complex road conditions and dynamic environments was proposed. Firstly, converting the distorted image and using the superposition threshold algorithm for edge detection, an aerial view of the lane was obtained via region of interest extraction and inverse perspective transformation. Secondly, the random sample consensus algorithm was adopted to fit the curves of lane lines based on the third-order B-spline curve model, and fitting evaluation and curvature radius calculation were then carried out on the curve. Lastly, by using the road driving video under complex road conditions and the Tusimple dataset, simulation test experiments for lane detection algorithm were performed. The experimental results show that the average detection accuracy based on road driving video reached 98.49%, and the average processing time reached 21.5 ms. The average detection accuracy based on the Tusimple dataset reached 98.42%, and the average processing time reached 22.2 ms. Compared with traditional methods and deep learning-based methodologies, this lane detection algorithm had excellent accuracy and real-time performance, a high detection efficiency and a strong anti-interference ability. The accurate recognition rate and average processing time were significantly improved. The proposed algorithm is crucial in promoting the technological level of intelligent vehicle driving assistance and conducive to the further improvement of the driving safety of intelligent vehicles.

Download Full-text

Post-Disaster Building Damage Detection from Earth Observation Imagery Using Unsupervised and Transferable Anomaly Detecting Generative Adversarial Networks

Remote Sensing ◽

10.3390/rs12244193 ◽

2020 ◽

Vol 12 (24) ◽

pp. 4193

Author(s):

Sofia Tilon ◽

Francesco Nex ◽

Norman Kerle ◽

George Vosselman

Keyword(s):

Deep Learning ◽

Damage Detection ◽

Satellite Imagery ◽

Building Damage ◽

Training Data ◽

Training Dataset ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Custom Made ◽

Post Disaster

We present an unsupervised deep learning approach for post-disaster building damage detection that can transfer to different typologies of damage or geographical locations. Previous advances in this direction were limited by insufficient qualitative training data. We propose to use a state-of-the-art Anomaly Detecting Generative Adversarial Network (ADGAN) because it only requires pre-event imagery of buildings in their undamaged state. This approach aids the post-disaster response phase because the model can be developed in the pre-event phase and rapidly deployed in the post-event phase. We used the xBD dataset, containing pre- and post- event satellite imagery of several disaster-types, and a custom made Unmanned Aerial Vehicle (UAV) dataset, containing post-earthquake imagery. Results showed that models trained on UAV-imagery were capable of detecting earthquake-induced damage. The best performing model for European locations obtained a recall, precision and F1-score of 0.59, 0.97 and 0.74, respectively. Models trained on satellite imagery were capable of detecting damage on the condition that the training dataset was void of vegetation and shadows. In this manner, the best performing model for (wild)fire events yielded a recall, precision and F1-score of 0.78, 0.99 and 0.87, respectively. Compared to other supervised and/or multi-epoch approaches, our results are encouraging. Moreover, in addition to image classifications, we show how contextual information can be used to create detailed damage maps without the need of a dedicated multi-task deep learning framework. Finally, we formulate practical guidelines to apply this single-epoch and unsupervised method to real-world applications.

Download Full-text

Single-Image Depth Inference Using Generative Adversarial Networks

Sensors ◽

10.3390/s19071708 ◽

2019 ◽

Vol 19 (7) ◽

pp. 1708 ◽

Cited By ~ 1

Author(s):

Daniel Stanley Tan ◽

Chih-Yuan Yao ◽

Conrado Ruiz ◽

Kai-Lung Hua

Keyword(s):

Smart Cities ◽

Depth Map ◽

Depth Estimation ◽

Input Image ◽

Generative Adversarial Networks ◽

Depth Information ◽

Single Image ◽

Neural Network Models ◽

Generative Adversarial Network ◽

Depth Sensors

Depth has been a valuable piece of information for perception tasks such as robot grasping, obstacle avoidance, and navigation, which are essential tasks for developing smart homes and smart cities. However, not all applications have the luxury of using depth sensors or multiple cameras to obtain depth information. In this paper, we tackle the problem of estimating the per-pixel depths from a single image. Inspired by the recent works on generative neural network models, we formulate the task of depth estimation as a generative task where we synthesize an image of the depth map from a single Red, Green, and Blue (RGB) input image. We propose a novel generative adversarial network that has an encoder-decoder type generator with residual transposed convolution blocks trained with an adversarial loss. Quantitative and qualitative experimental results demonstrate the effectiveness of our approach over several depth estimation works.

Download Full-text

A Novel Deep Learning Model for Mechanical Rotating Parts Fault Diagnosis Based on Optimal Transport and Generative Adversarial Networks

Actuators ◽

10.3390/act10070146 ◽

2021 ◽

Vol 10 (7) ◽

pp. 146

Author(s):

Xuanquan Wang ◽

Xiongjun Liu ◽

Ping Song ◽

Yifan Li ◽

Youtian Qie

Keyword(s):

Fault Diagnosis ◽

Real Time ◽

Network Architecture ◽

Transport Theory ◽

Optimal Transport ◽

Transmission System ◽

Generative Adversarial Networks ◽

Time Performance ◽

Network Algorithm ◽

Public Data

To solve the poor real-time performance of the existing fault diagnosis algorithms on transmission system rotating components, this paper proposes a novel high-dimensional OT-Caps (Optimal Transport–Capsule Network) model. Based on the traditional capsule network algorithm, an auxiliary loss is introduced during the offline training process to improve the network architecture. Simultaneously, an optimal transport theory and a generative adversarial network are introduced into the auxiliary loss, which accurately depicts the error distribution of the fault characteristic. The proposed model solves the low real-time performance of the capsule network algorithm due to complex architecture, long calculation time, and oversized hardware resource consumption. Meanwhile, it ensures the high precision, early prediction, and transfer aptitude of fault diagnosis. Finally, the model’s effectiveness is verified by the public data sets and the actual faults data of the transmission system, which provide technical support for the application.

Download Full-text

Real-time recognition of spraying area for UAV sprayers using a deep learning approach

PLoS ONE ◽

10.1371/journal.pone.0249436 ◽

2021 ◽

Vol 16 (4) ◽

pp. e0249436 ◽

Cited By ~ 1

Author(s):

Shahbaz Khan ◽

Muhammad Tufail ◽

Muhammad Tahir Khan ◽

Zubair Ahmad Khan ◽

Javaid Iqbal ◽

...

Keyword(s):

Deep Learning ◽

Real Time ◽

Computation Time ◽

Recognition System ◽

Crop Productivity ◽

Learning System ◽

Average Computation Time ◽

Weed Infestation ◽

Human Operators ◽

The Stability

Agricultural production is vital for the stability of the country’s economy. Controlling weed infestation through agrochemicals is necessary for increasing crop productivity. However, its excessive use has severe repercussions on the environment (damaging the ecosystem) and the human operators exposed to it. The use of Unmanned Aerial Vehicles (UAVs) has been proposed by several authors in the literature for performing the desired spraying and is considered safer and more precise than the conventional methods. Therefore, the study’s objective was to develop an accurate real-time recognition system of spraying areas for UAVs, which is of utmost importance for UAV-based sprayers. A two-step target recognition system was developed by using deep learning for the images collected from a UAV. Agriculture cropland of coriander was considered for building a classifier for recognizing spraying areas. The developed deep learning system achieved an average F1 score of 0.955, while the classifier recognition average computation time was 3.68 ms. The developed deep learning system can be deployed in real-time to UAV-based sprayers for accurate spraying.

Download Full-text