Image Super Resolution Using Generative Adversarial Networks and non-Paired Strategy

Surveillance Systems ◽

Adversarial Network ◽

Deductive Method ◽

Analysis Of The Images

The quality of images obtained from video surveillance systems is a decisive aspect when performing investigations at the Forensic Science. Features such as scars, tattoos, and skin marks are great examples of details that allow to consolidate an investigation at certain scenarios in which there is the necessity to identify individuals captured in a video or image footage. However, the low quality of images could affect the results of the investigations. In this sense, this work proposes the study of a computational model to address the problem of increasing the resolution of Low-Resolution (LR) images, also known as the problem of super-resolution of images. The main idea is to train a Generative Adversarial Network (GAN) so that it can be able to enhance low-quality images. The hypothesis is that a variant model of a GAN, named Super-Resolution Generative Adversarial Network (SRGAN), is capable to produce High-Resolution (HR) images from LR ones. The proposed methodology is based on experimental research with the aid of the hypothetical deductive method, where two well-recognised state of art methods were used, which proposes the use of convolutional neural networks and deep learning. For the model validation, were conducted four different experiments: two to avail the capacity of the GAN to produce images with enhanced resolution and two other experiments to evaluate the quality of the results produced by the SRGAN. The quantitative results of our experiments are promising, with performances that are similar to those obtained by state-of-the-art approaches. Moreover, the qualitative results based on performing a visual analysis of the images produced by our approach suggest a interesting performance in terms of visual quality.

Towards Accuracy Enhancement of Age Group Classification Using Generative Adversarial Networks

Journal of Integrated Design and Process Science ◽

10.3233/jid-210019 ◽

2021 ◽

pp. 1-17

Author(s):

Khaled ELKarazle ◽

Valliappan Raman ◽

Patrick Then

Keyword(s):

Age Estimation ◽

Super Resolution ◽

Adversarial Network ◽

Adversarial Networks ◽

Original Dataset ◽

Age Group Classification ◽

Facial Images

Age estimation models can be employed in many applications, including soft biometrics, content access control, targeted advertising, and many more. However, as some facial images are taken in unrestrained conditions, the quality relegates, which results in the loss of several essential ageing features. This study investigates how introducing a new layer of data processing based on a super-resolution generative adversarial network (SRGAN) model can influence the accuracy of age estimation by enhancing the quality of both the training and testing samples. Additionally, we introduce a novel convolutional neural network (CNN) classifier to distinguish between several age classes. We train one of our classifiers on a reconstructed version of the original dataset and compare its performance with an identical classifier trained on the original version of the same dataset. Our findings reveal that the classifier which trains on the reconstructed dataset produces better classification accuracy, opening the door for more research into building data-centric machine learning systems.

TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution

ACM Transactions on Intelligent Systems and Technology ◽

10.1145/3456726 ◽

2021 ◽

Vol 12 (6) ◽

pp. 1-20

Author(s):

Fayaz Ali Dharejo ◽

Farah Deeba ◽

Yuanchun Zhou ◽

Bhagwan Das ◽

Munsif Ali Jatoi ◽

...

Keyword(s):

Remote Sensing ◽

Super Resolution ◽

Single Image ◽

Adversarial Network ◽

Adversarial Networks ◽

Image Super Resolution ◽

Spatio Temporal ◽

Single Image Super Resolution

Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR) . However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatio-temporal remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatio-temporal remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.

Upsampling Real-Time, Low-Resolution CCTV Videos Using Generative Adversarial Networks

Electronics ◽

10.3390/electronics9081312 ◽

2020 ◽

Vol 9 (8) ◽

pp. 1312

Author(s):

Debapriya Hazra ◽

Yung-Cheol Byun

Keyword(s):

High Resolution ◽

Real Time ◽

Super Resolution ◽

Poor Quality ◽

Temporal Consistency ◽

Low Resolution ◽

Adversarial Network ◽

Benchmark Datasets

Video super-resolution has become an emerging topic in the field of machine learning. The generative adversarial network is a framework that is widely used to develop solutions for low-resolution videos. Video surveillance using closed-circuit television (CCTV) is significant in every field, all over the world. A common problem with CCTV videos is sudden video loss or poor quality. In this paper, we propose a generative adversarial network that implements spatio-temporal generators and discriminators to enhance real-time low-resolution CCTV videos to high-resolution. The proposed model considers both foreground and background motion of a CCTV video and effectively models the spatial and temporal consistency from low-resolution video frames to generate high-resolution videos. Quantitative and qualitative experiments on benchmark datasets, including Kinetics-700, UCF101, HMDB51 and IITH_Helmet2, showed that our model outperforms the existing GAN models for video super-resolution.

Improvement of Learning Stability of Generative Adversarial Network Using Variational Learning

Applied Sciences ◽

10.3390/app10134528 ◽

2020 ◽

Vol 10 (13) ◽

pp. 4528

Author(s):

Je-Yeol Lee ◽

Sang-Il Choi

Keyword(s):

Generative Models ◽

Learning Processes ◽

Adversarial Network ◽

Adversarial Networks ◽

Secondary Network ◽

Variational Autoencoder ◽

Variational Learning

In this paper, we propose a new network model using variational learning to improve the learning stability of generative adversarial networks (GAN). The proposed method can be easily applied to improve the learning stability of GAN-based models that were developed for various purposes, given that the variational autoencoder (VAE) is used as a secondary network while the basic GAN structure is maintained. When the gradient of the generator vanishes in the learning process of GAN, the proposed method receives gradient information from the decoder of the VAE that maintains gradient stably, so that the learning processes of the generator and discriminator are not halted. The experimental results of the MNIST and the CelebA datasets verify that the proposed method improves the learning stability of the networks by overcoming the vanishing gradient problem of the generator, and maintains the excellent data quality of the conventional GAN-based generative models.

Empirical Analysis of Deep Convolutional Generative Adversarial Network for Ultrasound Image Synthesis

The Open Biomedical Engineering Journal ◽

10.2174/1874120702115010071 ◽

2021 ◽

Vol 15 (1) ◽

pp. 71-77

Author(s):

Dheeraj Kumar ◽

Mayuri A. Mehta ◽

Indranath Chatterjee

Keyword(s):

Empirical Analysis ◽

Ultrasound Imaging ◽

Ultrasound Image ◽

Ultrasound Images ◽

Adversarial Network ◽

Adversarial Networks ◽

Synthetic Images

Introduction: Recent research on Generative Adversarial Networks (GANs) in the biomedical field has proven the effectiveness in generating synthetic images of different modalities. Ultrasound imaging is one of the primary imaging modalities for diagnosis in the medical domain. In this paper, we present an empirical analysis of the state-of-the-art Deep Convolutional Generative Adversarial Network (DCGAN) for generating synthetic ultrasound images. Aims: This work aims to explore the utilization of deep convolutional generative adversarial networks for the synthesis of ultrasound images and to leverage its capabilities. Background: Ultrasound imaging plays a vital role in healthcare for timely diagnosis and treatment. Increasing interest in automated medical image analysis for precise diagnosis has expanded the demand for a large number of ultrasound images. Generative adversarial networks have been proven beneficial for increasing the size of data by generating synthetic images. Objective: Our main purpose in generating synthetic ultrasound images is to produce a sufficient amount of ultrasound images with varying representations of a disease. Methods: DCGAN has been used to generate synthetic ultrasound images. It is trained on two ultrasound image datasets, namely, the common carotid artery dataset and nerve dataset, which are publicly available on Signal Processing Lab and Kaggle, respectively. Results: Results show that good quality synthetic ultrasound images are generated within 100 epochs of training of DCGAN. The quality of synthetic ultrasound images is evaluated using Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM). We have also presented some visual representations of the slices of generated images for qualitative comparison. Conclusion: Our empirical analysis reveals that synthetic ultrasound image generation using DCGAN is an efficient approach. Other: In future work, we plan to compare the quality of images generated through other adversarial methods such as conditional GAN, progressive GAN.

International Journal of Innovative Technology and Exploring Engineering - Special Issue ◽

CBNWI-50: A Deep Learning Bird Dataset for Image Translation and Resolution Improvement using Generative Adversarial Network

10.35940/ijitee.i1015.0789s19 ◽

2019 ◽

Vol 8 (9S) ◽

pp. 91-102

Keyword(s):

Deep Learning ◽

Super Resolution ◽

Western India ◽

Style Transfer ◽

Adversarial Network ◽

Image Translation ◽

Common Birds ◽

Single Image Super Resolution

Generative Adversarial Networks have gained prominence in a short span of time as they can synthesize images from latent noise by minimizing the adversarial cost function. New variants of GANs have been developed to perform specific tasks using state-of-the-art GAN models, like image translation, single image super resolution, segmentation, classification, style transfer etc. However, a combination of two GANs to perform two different applications in one model has been sparsely explored. Hence, this paper concatenates two GANs and aims to perform Image Translation using Cycle GAN model on bird images and improve their resolution using SRGAN. During the extensive survey, it is observed that most of the deep learning databases on Aves were built using the new world species (i.e. species found in North America). Hence, to bridge this gap, a new Ave database, 'Common Birds of North - Western India' (CBNWI-50), is also proposed in this work.

Single Image De-Raining Using Spinning Detail Perceptual Generative Adversarial Networks

Journal of Advanced Computational Intelligence and Intelligent Informatics ◽

10.20965/jaciii.2020.p0811 ◽

2020 ◽

Vol 24 (7) ◽

pp. 811-819

Author(s):

Kaizheng Chen ◽

◽

Yaping Dai ◽

Zhiyang Jia ◽

Kaoru Hirota

Keyword(s):

Perceptual Quality ◽

Training Procedure ◽

Single Image ◽

Adversarial Network ◽

Adversarial Networks ◽

Real World Datasets ◽

High Level

In this paper, Spinning Detail Perceptual Generative Adversarial Networks (SDP-GAN) is proposed for single image de-raining. The proposed method adopts the Generative Adversarial Network (GAN) framework and consists of two following networks: the rain streaks generative network G and the discriminative network D. To reduce the background interference, we propose a rain streaks generative network which not only focuses on the high frequency detail map of rainy image, but also directly reduces the mapping range from input to output. To further improve the perceptual quality of generated images, we modify the perceptual loss by extracting high-level features from discriminative network D, rather than pre-trained networks. Furthermore, we introduce a new training procedure based on the notion of self spinning to improve the final de-raining performance. Extensive experiments on the synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods.

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

GENERATIVE ADVERSARIAL NETWORKS TO GENERALISE URBAN AREAS IN TOPOGRAPHIC MAPS

10.5194/isprs-archives-xliii-b4-2021-15-2021 ◽

2021 ◽

Vol XLIII-B4-2021 ◽

pp. 15-22

Author(s):

A. Courtial ◽

G. Touya ◽

X. Zhang

Keyword(s):

Urban Areas ◽

Topographic Maps ◽

Topographic Map ◽

Inner Cities ◽

Adversarial Network ◽

Adversarial Networks ◽

The Impact

Abstract. This article presents how a generative adversarial network (GAN) can be employed to produce a generalised map that combines several cartographic themes in the dense context of urban areas. We use as input detailed buildings, roads, and rivers from topographic datasets produced by the French national mapping agency (IGN), and we expect as output of the GAN a legible map of these elements at a target scale of 1:50,000. This level of detail requires to reduce the amount of information while preserving patterns; covering dense inner cities block by a unique polygon is also necessary because these blocks cannot be represented with enlarged individual buildings. The target map has a style similar to the topographic map produced by IGN. This experiment succeeded in producing image tiles that look like legible maps. It also highlights the impact of data and representation choices on the quality of predicted images, and the challenge of learning geographic relationships.

MapGAN: An Intelligent Generation Model for Network Tile Maps

Sensors ◽

10.3390/s20113119 ◽

2020 ◽

Vol 20 (11) ◽

pp. 3119 ◽

Cited By ~ 1

Author(s):

Jingtao Li ◽

Zhanlong Chen ◽

Xiaozhen Zhao ◽

Lijia Shao

Keyword(s):

Image Inpainting ◽

Super Resolution ◽

Image Synthesis ◽

Great Success ◽

Generation Model ◽

Adversarial Network ◽

Image Translation ◽

Map Generation

In recent years, the generative adversarial network (GAN)-based image translation model has achieved great success in image synthesis, image inpainting, image super-resolution, and other tasks. However, the images generated by these models often have problems such as insufficient details and low quality. Especially for the task of map generation, the generated electronic map cannot achieve effects comparable to industrial production in terms of accuracy and aesthetics. This paper proposes a model called Map Generative Adversarial Networks (MapGAN) for generating multitype electronic maps accurately and quickly based on both remote sensing images and render matrices. MapGAN improves the generator architecture of Pix2pixHD and adds a classifier to enhance the model, enabling it to learn the characteristics and style differences of different types of maps. Using the datasets of Google Maps, Baidu maps, and Map World maps, we compare MapGAN with some recent image translation models in the fields of one-to-one map generation and one-to-many domain map generation. The results show that the quality of the electronic maps generated by MapGAN is optimal in terms of both intuitive vision and classic evaluation indicators.

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

Tag Disentangled Generative Adversarial Network for Object Image Re-rendering

10.24963/ijcai.2017/404 ◽

2017 ◽

Cited By ~ 18

Author(s):

Chaoyue Wang ◽

Chaohui Wang ◽

Chang Xu ◽

Dacheng Tao

Keyword(s):

Input Image ◽

Single Image ◽

Training Strategy ◽

Adversarial Network ◽

Adversarial Networks ◽

Adversarial Training ◽

Realistic Images

In this paper, we propose a principled Tag Disentangled Generative Adversarial Networks (TD-GAN) for re-rendering new images for the object of interest from a single image of it by specifying multiple scene properties (such as viewpoint, illumination, expression, etc.). The whole framework consists of a disentangling network, a generative network, a tag mapping net, and a discriminative network, which are trained jointly based on a given set of images that are completely/partially tagged (i.e., supervised/semi-supervised setting). Given an input image, the disentangling network extracts disentangled and interpretable representations, which are then used to generate images by the generative network. In order to boost the quality of disentangled representations, the tag mapping net is integrated to explore the consistency between the image and its tags. Furthermore, the discriminative network is introduced to implement the adversarial training strategy for generating more realistic images. Experiments on two challenging datasets demonstrate the state-of-the-art performance of the proposed framework in the problem of interest.