LIME-Based Data Selection Method for SAR Images Generation Using GAN

Deep learning has obtained remarkable achievements in computer vision, especially image and video processing. However, in synthetic aperture radar (SAR) image recognition, the application of DNNs is usually restricted due to data insufficiency. To augment datasets, generative adversarial networks (GANs) are usually used to generate numerous photo-realistic SAR images. Although there are many pixel-level metrics to measure GAN’s performance from the quality of generated SAR images, there are few measurements to evaluate whether the generated SAR images include the most representative features of the target. In this case, the classifier probably categorizes a SAR image into the corresponding class based on “wrong” criterion, i.e., “Clever Hans”. In this paper, local interpretable model-agnostic explanation (LIME) is innovatively utilized to evaluate whether a generated SAR image possessed the most representative features of a specific kind of target. Firstly, LIME is used to visualize positive contributions of the input SAR image to the correct prediction of the classifier. Subsequently, these representative SAR images can be selected handily by evaluating how much the positive contribution region matches the target. Experimental results demonstrate that the proposed method can ally “Clever Hans” phenomenon greatly caused by the spurious relationship between generated SAR images and the corresponding classes.

Download Full-text

A Noisy SAR Image Fusion Method Based on NLM and GAN

Entropy ◽

10.3390/e23040410 ◽

2021 ◽

Vol 23 (4) ◽

pp. 410

Author(s):

Jing Fang ◽

Xiaole Ma ◽

Jingjing Wang ◽

Kai Qin ◽

Shaohai Hu ◽

...

Keyword(s):

Image Fusion ◽

Generative Adversarial Networks ◽

Fusion Method ◽

Sar Image ◽

Color Information ◽

Image Block ◽

Sar Images ◽

Adversarial Networks ◽

Fused Image ◽

Image Fusion Method

The unavoidable noise often present in synthetic aperture radar (SAR) images, such as speckle noise, negatively impacts the subsequent processing of SAR images. Further, it is not easy to find an appropriate application for SAR images, given that the human visual system is sensitive to color and SAR images are gray. As a result, a noisy SAR image fusion method based on nonlocal matching and generative adversarial networks is presented in this paper. A nonlocal matching method is applied to processing source images into similar block groups in the pre-processing step. Then, adversarial networks are employed to generate a final noise-free fused SAR image block, where the generator aims to generate a noise-free SAR image block with color information, and the discriminator tries to increase the spatial resolution of the generated image block. This step ensures that the fused image block contains high resolution and color information at the same time. Finally, a fused image can be obtained by aggregating all the image blocks. By extensive comparative experiments on the SEN1–2 datasets and source images, it can be found that the proposed method not only has better fusion results but is also robust to image noise, indicating the superiority of the proposed noisy SAR image fusion method over the state-of-the-art methods.

Download Full-text

Automatic Target Recognition for Low Resolution Foliage Penetrating SAR Images Using CNNs and GANs

Remote Sensing ◽

10.3390/rs13040596 ◽

2021 ◽

Vol 13 (4) ◽

pp. 596

Author(s):

David Vint ◽

Matthew Anderson ◽

Yuhao Yang ◽

Christos Ilioudis ◽

Gaetano Di Caterina ◽

...

Keyword(s):

Target Recognition ◽

Automatic Target Recognition ◽

Generative Adversarial Networks ◽

Low Resolution ◽

Sar Images ◽

Adversarial Networks ◽

Technological Advances ◽

Dataset Size ◽

Resolution Imaging ◽

High Level

In recent years, the technological advances leading to the production of high-resolution Synthetic Aperture Radar (SAR) images has enabled more and more effective target recognition capabilities. However, high spatial resolution is not always achievable, and, for some particular sensing modes, such as Foliage Penetrating Radars, low resolution imaging is often the only option. In this paper, the problem of automatic target recognition in Low Resolution Foliage Penetrating (FOPEN) SAR is addressed through the use of Convolutional Neural Networks (CNNs) able to extract both low and high level features of the imaged targets. Additionally, to address the issue of limited dataset size, Generative Adversarial Networks are used to enlarge the training set. Finally, a Receiver Operating Characteristic (ROC)-based post-classification decision approach is used to reduce classification errors and measure the capability of the classifier to provide a reliable output. The effectiveness of the proposed framework is demonstrated through the use of real SAR FOPEN data.

Download Full-text

Towards Accuracy Enhancement of Age Group Classification Using Generative Adversarial Networks

Journal of Integrated Design and Process Science ◽

10.3233/jid-210019 ◽

2021 ◽

pp. 1-17

Author(s):

Khaled ELKarazle ◽

Valliappan Raman ◽

Patrick Then

Keyword(s):

Age Estimation ◽

Super Resolution ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Original Dataset ◽

Age Group Classification ◽

Facial Images

Age estimation models can be employed in many applications, including soft biometrics, content access control, targeted advertising, and many more. However, as some facial images are taken in unrestrained conditions, the quality relegates, which results in the loss of several essential ageing features. This study investigates how introducing a new layer of data processing based on a super-resolution generative adversarial network (SRGAN) model can influence the accuracy of age estimation by enhancing the quality of both the training and testing samples. Additionally, we introduce a novel convolutional neural network (CNN) classifier to distinguish between several age classes. We train one of our classifiers on a reconstructed version of the original dataset and compare its performance with an identical classifier trained on the original version of the same dataset. Our findings reveal that the classifier which trains on the reconstructed dataset produces better classification accuracy, opening the door for more research into building data-centric machine learning systems.

Download Full-text

Fractional Wavelet-Based Generative Scattering Networks

Frontiers in Neurorobotics ◽

10.3389/fnbot.2021.752752 ◽

2021 ◽

Vol 15 ◽

Author(s):

Jiasong Wu ◽

Xiang Qiu ◽

Jing Zhang ◽

Fuzhi Wu ◽

Youyong Kong ◽

...

Keyword(s):

Dimensionality Reduction ◽

Reduction Method ◽

Gaussian White Noise ◽

Principal Component ◽

Experimental Results ◽

Generative Adversarial Networks ◽

Image Generation ◽

Adversarial Networks ◽

Dimensionality Reduction Method

Generative adversarial networks and variational autoencoders (VAEs) provide impressive image generation from Gaussian white noise, but both are difficult to train, since they need a generator (or encoder) and a discriminator (or decoder) to be trained simultaneously, which can easily lead to unstable training. To solve or alleviate these synchronous training problems of generative adversarial networks (GANs) and VAEs, researchers recently proposed generative scattering networks (GSNs), which use wavelet scattering networks (ScatNets) as the encoder to obtain features (or ScatNet embeddings) and convolutional neural networks (CNNs) as the decoder to generate an image. The advantage of GSNs is that the parameters of ScatNets do not need to be learned, while the disadvantage of GSNs is that their ability to obtain representations of ScatNets is slightly weaker than that of CNNs. In addition, the dimensionality reduction method of principal component analysis (PCA) can easily lead to overfitting in the training of GSNs and, therefore, affect the quality of generated images in the testing process. To further improve the quality of generated images while keeping the advantages of GSNs, this study proposes generative fractional scattering networks (GFRSNs), which use more expressive fractional wavelet scattering networks (FrScatNets), instead of ScatNets as the encoder to obtain features (or FrScatNet embeddings) and use similar CNNs of GSNs as the decoder to generate an image. Additionally, this study develops a new dimensionality reduction method named feature-map fusion (FMF) instead of performing PCA to better retain the information of FrScatNets,; it also discusses the effect of image fusion on the quality of the generated image. The experimental results obtained on the CIFAR-10 and CelebA datasets show that the proposed GFRSNs can lead to better generated images than the original GSNs on testing datasets. The experimental results of the proposed GFRSNs with deep convolutional GAN (DCGAN), progressive GAN (PGAN), and CycleGAN are also given.

Download Full-text

Multi-Turn Chatbot Based on Query-Context Attentions and Dual Wasserstein Generative Adversarial Networks

Applied Sciences ◽

10.3390/app9183908 ◽

2019 ◽

Vol 9 (18) ◽

pp. 3908 ◽

Cited By ~ 3

Author(s):

Jintae Kim ◽

Shinhyeok Oh ◽

Oh-Woog Kwon ◽

Harksoo Kim

Keyword(s):

Performance Measures ◽

State Of The Art ◽

Attention Mechanism ◽

Generative Adversarial Networks ◽

Training Method ◽

Adversarial Networks ◽

Proposed Model ◽

Previous State ◽

Vector Representations

To generate proper responses to user queries, multi-turn chatbot models should selectively consider dialogue histories. However, previous chatbot models have simply concatenated or averaged vector representations of all previous utterances without considering contextual importance. To mitigate this problem, we propose a multi-turn chatbot model in which previous utterances participate in response generation using different weights. The proposed model calculates the contextual importance of previous utterances by using an attention mechanism. In addition, we propose a training method that uses two types of Wasserstein generative adversarial networks to improve the quality of responses. In experiments with the DailyDialog dataset, the proposed model outperformed the previous state-of-the-art models based on various performance measures.

Download Full-text

A Novel Method for the Deblurring of Photogrammetric Images Using Conditional Generative Adversarial Networks

Remote Sensing ◽

10.3390/rs12162586 ◽

2020 ◽

Vol 12 (16) ◽

pp. 2586 ◽

Cited By ~ 1

Author(s):

Pawel Burdziakowski

Keyword(s):

Point Clouds ◽

Aerial Images ◽

Generative Adversarial Networks ◽

Camera Motion ◽

Data Set ◽

Adversarial Networks ◽

Blur Kernel ◽

Almost All ◽

Blurred Images

The visual data acquisition from small unmanned aerial vehicles (UAVs) may encounter a situation in which blur appears on the images. Image blurring caused by camera motion during exposure significantly impacts the images interpretation quality and consequently the quality of photogrammetric products. On blurred images, it is difficult to visually locate ground control points, and the number of identified feature points decreases rapidly together with an increasing blur kernel. The nature of blur can be non-uniform, which makes it hard to forecast for traditional deblurring methods. Due to the above, the author of this publication concluded that the neural methods developed in recent years were able to eliminate blur on UAV images with an unpredictable or highly variable blur nature. In this research, a new, rapid method based on generative adversarial networks (GANs) was applied for deblurring. A data set for neural network training was developed based on real aerial images collected over the last few years. More than 20 full sets of photogrammetric products were developed, including point clouds, orthoimages and digital surface models. The sets were generated from both blurred and deblurred images using the presented method. The results presented in the publication show that the method for improving blurred photo quality significantly contributed to an improvement in the general quality of typical photogrammetric products. The geometric accuracy of the products generated from deblurred photos was maintained despite the rising blur kernel. The quality of textures and input photos was increased. This research proves that the developed method based on neural networks can be used for deblur, even in highly blurred images, and it significantly increases the final geometric quality of the photogrammetric products. In practical cases, it will be possible to implement an additional feature in the photogrammetric software, which will eliminate unwanted blur and allow one to use almost all blurred images in the modelling process.

Download Full-text

PeaceGAN: A GAN-Based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier

Remote Sensing ◽

10.3390/rs13193939 ◽

2021 ◽

Vol 13 (19) ◽

pp. 3939

Author(s):

Jihyong Oh ◽

Munchurl Kim

Keyword(s):

Human Perception ◽

Speckle Noise ◽

Generative Adversarial Networks ◽

Image Generation ◽

Target Image ◽

Multiple Sources ◽

Sar Images ◽

Task Learning ◽

Adversarial Networks ◽

The One

Although generative adversarial networks (GANs) are successfully applied to diverse fields, training GANs on synthetic aperture radar (SAR) data is a challenging task due to speckle noise. On the one hand, in a learning perspective of human perception, it is natural to learn a task by using information from multiple sources. However, in the previous GAN works on SAR image generation, information on target classes has only been used. Due to the backscattering characteristics of SAR signals, the structures of SAR images are strongly dependent on their pose angles. Nevertheless, the pose angle information has not been incorporated into GAN models for SAR images. In this paper, we propose a novel GAN-based multi-task learning (MTL) method for SAR target image generation, called PeaceGAN, that has two additional structures, a pose estimator and an auxiliary classifier, at the side of its discriminator in order to effectively combine the pose and class information via MTL. Extensive experiments showed that the proposed MTL framework can help the PeaceGAN’s generator effectively learn the distributions of SAR images so that it can better generate the SAR target images more faithfully at intended pose angles for desired target classes in comparison with the recent state-of-the-art methods.

Download Full-text

Reconstruction of Generative Adversarial Networks in Cross Modal Image Generation with Canonical Polyadic Decomposition

Wireless Communications and Mobile Computing ◽

10.1155/2021/8868781 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Ruixin Ma ◽

Junying Lou ◽

Peng Li ◽

Jing Gao

Keyword(s):

Mobile Terminal ◽

Experimental Results ◽

Generative Adversarial Networks ◽

Image Generation ◽

Adversarial Networks ◽

Speed Up ◽

Canonical Polyadic Decomposition

Generating pictures from text is an interesting, classic, and challenging task. Benefited from the development of generative adversarial networks (GAN), the generation quality of this task has been greatly improved. Many excellent cross modal GAN models have been put forward. These models add extensive layers and constraints to get impressive generation pictures. However, complexity and computation of existing cross modal GANs are too high to be deployed in mobile terminal. To solve this problem, this paper designs a compact cross modal GAN based on canonical polyadic decomposition. We replace an original convolution layer with three small convolution layers and use an autoencoder to stabilize and speed up training. The experimental results show that our model achieves 20% times of compression in both parameters and FLOPs without loss of quality on generated images.

Download Full-text

Image-Adaptive GAN Based Reconstruction

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.5708 ◽

2020 ◽

Vol 34 (04) ◽

pp. 3121-3129 ◽

Cited By ~ 2

Author(s):

Shady Abu Hussein ◽

Tom Tirer ◽

Raja Giryes

Keyword(s):

Inverse Problems ◽

Compressed Sensing ◽

Super Resolution ◽

Generative Models ◽

Generative Adversarial Networks ◽

Adversarial Networks ◽

Image Super Resolution ◽

Human Faces

In the recent years, there has been a significant improvement in the quality of samples produced by (deep) generative models such as variational auto-encoders and generative adversarial networks. However, the representation capabilities of these methods still do not capture the full distribution for complex classes of images, such as human faces. This deficiency has been clearly observed in previous works that use pre-trained generative models to solve imaging inverse problems. In this paper, we suggest to mitigate the limited representation capabilities of generators by making them image-adaptive and enforcing compliance of the restoration with the observations via back-projections. We empirically demonstrate the advantages of our proposed approach for image super-resolution and compressed sensing.

Download Full-text

Utilizing Amari-Alpha Divergence to Stabilize the Training of Generative Adversarial Networks

Entropy ◽

10.3390/e22040410 ◽

2020 ◽

Vol 22 (4) ◽

pp. 410 ◽

Cited By ~ 2

Author(s):

Likun Cai ◽

Yanjie Chen ◽

Ning Cai ◽

Wei Cheng ◽

Hao Wang

Keyword(s):

State Of The Art ◽

Generative Adversarial Networks ◽

Image Generation ◽

Significant Progress ◽

Trade Off ◽

Adversarial Networks ◽

Leibler Divergence ◽

The Stability ◽

Hellinger Divergence

Generative Adversarial Nets (GANs) are one of the most popular architectures for image generation, which has achieved significant progress in generating high-resolution, diverse image samples. The normal GANs are supposed to minimize the Kullback–Leibler divergence between distributions of natural and generated images. In this paper, we propose the Alpha-divergence Generative Adversarial Net (Alpha-GAN) which adopts the alpha divergence as the minimization objective function of generators. The alpha divergence can be regarded as a generalization of the Kullback–Leibler divergence, Pearson χ 2 divergence, Hellinger divergence, etc. Our Alpha-GAN employs the power function as the form of adversarial loss for the discriminator with two-order indexes. These hyper-parameters make our model more flexible to trade off between the generated and target distributions. We further give a theoretical analysis of how to select these hyper-parameters to balance the training stability and the quality of generated images. Extensive experiments of Alpha-GAN are performed on SVHN and CelebA datasets, and evaluation results show the stability of Alpha-GAN. The generated samples are also competitive compared with the state-of-the-art approaches.

Download Full-text