scholarly journals Converting Optical Videos to Infrared Videos Using Attention GAN and Its Impact on Target Detection and Classification Performance

2021 ◽  
Vol 13 (16) ◽  
pp. 3257
Author(s):  
Mohammad Shahab Uddin ◽  
Reshad Hoque ◽  
Kazi Aminul Islam ◽  
Chiman Kwan ◽  
David Gribben ◽  
...  
Keyword(s):  
Deep Learning ◽  
Target Detection ◽  
High Performance ◽  
Classification Performance ◽  
Training Data ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Subjective Evaluations ◽  
Infrared Video ◽  
The Impact

To apply powerful deep-learning-based algorithms for object detection and classification in infrared videos, it is necessary to have more training data in order to build high-performance models. However, in many surveillance applications, one can have a lot more optical videos than infrared videos. This lack of IR video datasets can be mitigated if optical-to-infrared video conversion is possible. In this paper, we present a new approach for converting optical videos to infrared videos using deep learning. The basic idea is to focus on target areas using attention generative adversarial network (attention GAN), which will preserve the fidelity of target areas. The approach does not require paired images. The performance of the proposed attention GAN has been demonstrated using objective and subjective evaluations. Most importantly, the impact of attention GAN has been demonstrated in improved target detection and classification performance using real-infrared videos.

scholarly journals A deep learning segmentation strategy that minimizes the amount of manually annotated images

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 256
Author(s):  
Thierry Pécot ◽  
Alexander Alekseyenko ◽  
Kristin Wallace
Keyword(s):  
Deep Learning ◽  
Data Augmentation ◽  
Training Data ◽  
Artificial Data ◽  
Deep Convolutional Neural Networks ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Segmentation Strategy ◽  
Adversarial Network ◽  
The Impact

Deep learning has revolutionized the automatic processing of images. While deep convolutional neural networks have demonstrated astonishing segmentation results for many biological objects acquired with microscopy, this technology's good performance relies on large training datasets. In this paper, we present a strategy to minimize the amount of time spent in manually annotating images for segmentation. It involves using an efficient and open source annotation tool, the artificial increase of the training data set with data augmentation, the creation of an artificial data set with a conditional generative adversarial network and the combination of semantic and instance segmentations. We evaluate the impact of each of these approaches for the segmentation of nuclei in 2D widefield images of human precancerous polyp biopsies in order to define an optimal strategy.

Underwater Acoustic Target Recognition Based on Generative Adversarial Network Data Augmentation

2021 ◽  
Vol 263 (2) ◽  
pp. 4558-4564
Author(s):  
Minghong Zhang ◽  
Xinwei Luo
Keyword(s):  
Data Augmentation ◽  
Target Recognition ◽  
Training Data ◽  
Small Samples ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Underwater Acoustic ◽  
Adversarial Network ◽  
Acoustic Target ◽  
The Impact

Underwater acoustic target recognition is an important aspect of underwater acoustic research. In recent years, machine learning has been developed continuously, which is widely and effectively applied in underwater acoustic target recognition. In order to acquire good recognition results and reduce the problem of overfitting, Adequate data sets are essential. However, underwater acoustic samples are relatively rare, which has a certain impact on recognition accuracy. In this paper, in addition of the traditional audio data augmentation method, a new method of data augmentation using generative adversarial network is proposed, which uses generator and discriminator to learn the characteristics of underwater acoustic samples, so as to generate reliable underwater acoustic signals to expand the training data set. The expanded data set is input into the deep neural network, and the transfer learning method is applied to further reduce the impact caused by small samples by fixing part of the pre-trained parameters. The experimental results show that the recognition result of this method is better than the general underwater acoustic recognition method, and the effectiveness of this method is verified.

scholarly journals INFRASTRUCTURE DEGRADATION AND POST-DISASTER DAMAGE DETECTION USING ANOMALY DETECTING GENERATIVE ADVERSARIAL NETWORKS

2020 ◽  
Vol V-2-2020 ◽  
pp. 573-582 ◽  
Author(s):  
S. M. Tilon ◽  
F. Nex ◽  
D. Duarte ◽  
N. Kerle ◽  
G. Vosselman
Keyword(s):  
Deep Learning ◽  
Damage Detection ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Essential Information ◽  
Urban Scenes ◽  
Adversarial Network ◽  
Collapsed Buildings ◽  
Post Disaster

Abstract. Degradation and damage detection provides essential information to maintenance workers in routine monitoring and to first responders in post-disaster scenarios. Despite advance in Earth Observation (EO), image analysis and deep learning techniques, the quality and quantity of training data for deep learning is still limited. As a result, no robust method has been found yet that can transfer and generalize well over a variety of geographic locations and typologies of damages. Since damages can be seen as anomalies, occurring sparingly over time and space, we propose to use an anomaly detecting Generative Adversarial Network (GAN) to detect damages. The main advantages of using GANs are that only healthy unannotated images are needed, and that a variety of damages, including the never before seen damage, can be detected. In this study we aimed to investigate 1) the ability of anomaly detecting GANs to detect degradation (potholes and cracks) in asphalt road infrastructures using Mobile Mapper imagery and building damage (collapsed buildings, rubble piles) using post-disaster aerial imagery, and 2) the sensitivity of this method against various types of pre-processing. Our results show that we can detect damages in urban scenes at satisfying levels but not on asphalt roads. Future work will investigate how to further classify the found damages and how to improve damage detection for asphalt roads.

scholarly journals Generative Adversarial Networks for Visible to Infrared Video Conversion

2020 ◽  
Author(s):  
Mohammad Shahab Uddin ◽  
Jiang Li
Keyword(s):  
Deep Learning ◽  
Performance Metrics ◽  
Infrared Image ◽  
Image Databases ◽  
Learning Models ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Optical Images ◽  
Infrared Video ◽  
Image Pairs

Deep learning models are data driven. For example, the most popular convolutional neural network (CNN) model used for image classification or object detection requires large labeled databases for training to achieve competitive performances. This requirement is not difficult to be satisfied in the visible domain since there are lots of labeled video and image databases available nowadays. However, given the less popularity of infrared (IR) camera, the availability of labeled infrared videos or image databases is limited. Therefore, training deep learning models in infrared domain is still challenging. In this chapter, we applied the pix2pix generative adversarial network (Pix2Pix GAN) and cycle-consistent GAN (Cycle GAN) models to convert visible videos to infrared videos. The Pix2Pix GAN model requires visible-infrared image pairs for training while the Cycle GAN relaxes this constraint and requires only unpaired images from both domains. We applied the two models to an open-source database where visible and infrared videos provided by the signal multimedia and telecommunications laboratory at the Federal University of Rio de Janeiro. We evaluated conversion results by performance metrics including Inception Score (IS), Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Our experiments suggest that cycle-consistent GAN is more effective than pix2pix GAN for generating IR images from optical images.

scholarly journals Self-Attention-Based Conditional Variational Auto-Encoder Generative Adversarial Networks for Hyperspectral Classification

2021 ◽  
Vol 13 (16) ◽  
pp. 3316
Author(s):  
Zhitao Chen ◽  
Lei Tong ◽  
Bin Qian ◽  
Jing Yu ◽  
Chuangbai Xiao
Keyword(s):  
Classification Performance ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Classification Methods ◽  
Generative Adversarial Network ◽  
Model Stability ◽  
Adversarial Network ◽  
Variational Autoencoder ◽  
Virtual Samples ◽  
Hyperspectral Classification

Hyperspectral classification is an important technique for remote sensing image analysis. For the current classification methods, limited training data affect the classification results. Recently, Conditional Variational Autoencoder Generative Adversarial Network (CVAEGAN) has been used to generate virtual samples to augment the training data, which could improve the classification performance. To further improve the classification performance, based on the CVAEGAN, we propose a Self-Attention-Based Conditional Variational Autoencoder Generative Adversarial Network (SACVAEGAN). Compared with CVAEGAN, we first use random latent vectors to obtain more enhanced virtual samples, which can improve the generalization performance. Then, we introduce the self-attention mechanism into our model to force the training process to pay more attention to global information, which can achieve better classification accuracy. Moreover, we explore model stability by incorporating the WGAN-GP loss function into our model to reduce the mode collapse probability. Experiments on three data sets and a comparison of the state-of-art methods show that SACVAEGAN has great advantages in accuracy compared with state-of-the-art HSI classification methods.

scholarly journals Anomaly Detection Neural Network with Dual Auto-Encoders GAN and Its Industrial Inspection Applications

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3336 ◽  
Author(s):  
Ta-Wei Tang ◽  
Wei-Han Kuo ◽  
Jauh-Hsiang Lan ◽  
Chien-Fang Ding ◽  
Hakiem Hsu ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Training Data ◽  
Labor Costs ◽  
Generative Adversarial Network ◽  
Detection Ability ◽  
Adversarial Network ◽  
Automated Optical Inspection ◽  
High Level

Recently, researchers have been studying methods to introduce deep learning into automated optical inspection (AOI) systems to reduce labor costs. However, the integration of deep learning in the industry may encounter major challenges such as sample imbalance (defective products that only account for a small proportion). Therefore, in this study, an anomaly detection neural network, dual auto-encoder generative adversarial network (DAGAN), was developed to solve the problem of sample imbalance. With skip-connection and dual auto-encoder architecture, the proposed method exhibited excellent image reconstruction ability and training stability. Three datasets, namely public industrial detection training set, MVTec AD, with mobile phone screen glass and wood defect detection datasets, were used to verify the inspection ability of DAGAN. In addition, training with a limited amount of data was proposed to verify its detection ability. The results demonstrated that the areas under the curve (AUCs) of DAGAN were better than previous generative adversarial network-based anomaly detection models in 13 out of 17 categories in these datasets, especially in categories with high variability or noise. The maximum AUC improvement was 0.250 (toothbrush). Moreover, the proposed method exhibited better detection ability than the U-Net auto-encoder, which indicates the function of discriminator in this application. Furthermore, the proposed method had a high level of AUCs when using only a small amount of training data. DAGAN can significantly reduce the time and cost of collecting and labeling data when it is applied to industrial detection.

scholarly journals 4-Class MI-EEG Signal Generation and Recognition with CVAE-GAN

2021 ◽  
Vol 11 (4) ◽  
pp. 1798
Author(s):  
Jun Yang ◽  
Huijuan Yu ◽  
Tao Shen ◽  
Yaolian Song ◽  
Zhuangfei Chen
Keyword(s):  
Deep Learning ◽  
Data Augmentation ◽  
Feature Matching ◽  
Continuous Optimization ◽  
Training Data ◽  
Small Scale ◽  
Eeg Signal ◽  
Generative Adversarial Network ◽  
The Real ◽  
Adversarial Network

As the capability of an electroencephalogram’s (EEG) measurement of the real-time electrodynamics of the human brain is known to all, signal processing techniques, particularly deep learning, could either provide a novel solution for learning but also optimize robust representations from EEG signals. Considering the limited data collection and inadequate concentration of during subjects testing, it becomes essential to obtain sufficient training data and useful features with a potential end-user of a brain–computer interface (BCI) system. In this paper, we combined a conditional variational auto-encoder network (CVAE) with a generative adversarial network (GAN) for learning latent representations from EEG brain signals. By updating the fine-tuned parameter fed into the resulting generative model, we could synthetize the EEG signal under a specific category. We employed an encoder network to obtain the distributed samples of the EEG signal, and applied an adversarial learning mechanism to continuous optimization of the parameters of the generator, discriminator and classifier. The CVAE was adopted to adjust the synthetics more approximately to the real sample class. Finally, we demonstrated our approach take advantages of both statistic and feature matching to make the training process converge faster and more stable and address the problem of small-scale datasets in deep learning applications for motor imagery tasks through data augmentation. The augmented training datasets produced by our proposed CVAE-GAN method significantly enhance the performance of MI-EEG recognition.

GANDaLF: GAN for Data-Limited Fingerprinting

2021 ◽  
Vol 2021 (2) ◽  
pp. 305-322
Author(s):  
Se Eun Oh ◽  
Nate Mathews ◽  
Mohammad Saidur Rahman ◽  
Matthew Wright ◽  
Nicholas Hopper
Keyword(s):  
Deep Learning ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Large Set ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Closed World ◽  
Training Samples ◽  
A Site

Abstract We introduce Generative Adversarial Networks for Data-Limited Fingerprinting (GANDaLF), a new deep-learning-based technique to perform Website Fingerprinting (WF) on Tor traffic. In contrast to most earlier work on deep-learning for WF, GANDaLF is intended to work with few training samples, and achieves this goal through the use of a Generative Adversarial Network to generate a large set of “fake” data that helps to train a deep neural network in distinguishing between classes of actual training data. We evaluate GANDaLF in low-data scenarios including as few as 10 training instances per site, and in multiple settings, including fingerprinting of website index pages and fingerprinting of non-index pages within a site. GANDaLF achieves closed-world accuracy of 87% with just 20 instances per site (and 100 sites) in standard WF settings. In particular, GANDaLF can outperform Var-CNN and Triplet Fingerprinting (TF) across all settings in subpage fingerprinting. For example, GANDaLF outperforms TF by a 29% margin and Var-CNN by 38% for training sets using 20 instances per site.

Applying LSTM and GAN to build a deep learning model (TGAN-TEC) for global ionospheric TEC

2020 ◽  
Author(s):  
zhou chen ◽  
Yue deng ◽  
Jing-Song wang
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Training Data ◽  
Spatial And Temporal Variation ◽  
Electron Content ◽  
Generative Adversarial Network ◽  
Total Electron ◽  
Adversarial Network ◽  
Short Time ◽  
Deep Learning Model

<p>TEC is very important ionospheric parameter, which is commonly used observation for studying various ionospheric physical mechanism and other technological related to ionosphere (i.e. Global Positioning). However, the variation of global TEC is very dynamic, and its spatiotemporal variation is extremely complicated. So in this paper, we try to build a novel global ionospheric TEC (total electron content) predicting model based on two deep learning algorithms: generative adversarial network (GAN) and long short-term memory (LSTM). Training data is from 10-year IGS TEC data, which provide plenty of data for the GAN and LSTM algorithm to obtain the spatial and temporal variation of TEC respectively. The prediction accuracy of this model have been calculated under different levels of geomagnetic activity. The statistic result suggest that the proposed ionospheric model can be used as an efficient tool for ionospheric TEC short-time prediction.</p>

Role of General Adversarial Networks in Mammogram Analysis: A Review

2020 ◽  
Vol 16 (7) ◽  
pp. 863-877
Author(s):  
Annapoorani Gopal ◽  
Lathaselvi Gandhimaruthian ◽  
Javid Ali
Keyword(s):  
Breast Tumor ◽  
Deep Neural Networks ◽  
Training Data ◽  
Learning Technology ◽  
Breast Cancers ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Tumor Extraction

The Deep Neural Networks have gained prominence in the biomedical domain, becoming the most commonly used networks after machine learning technology. Mammograms can be used to detect breast cancers with high precision with the help of Convolutional Neural Network (CNN) which is deep learning technology. An exhaustive labeled data is required to train the CNN from scratch. This can be overcome by deploying Generative Adversarial Network (GAN) which comparatively needs lesser training data during a mammogram screening. In the proposed study, the application of GANs in estimating breast density, high-resolution mammogram synthesis for clustered microcalcification analysis, effective segmentation of breast tumor, analysis of the shape of breast tumor, extraction of features and augmentation of the image during mammogram classification have been extensively reviewed.

Sign in / Sign up

Export Citation Format

Share Document