A novel quadruple generative adversarial network for semi-supervised categorization of low-resolution images

Activity recognition is a fundamental and crucial task in computer vision. Impressive results have been achieved for activity recognition in high-resolution videos, but for extreme low-resolution videos, which capture the action information at a distance and are vital for preserving privacy, the performance of activity recognition algorithms is far from satisfactory. The reason is that extreme low-resolution (e.g., 12 × 16 pixels) images lack adequate scene and appearance information, which is needed for efficient recognition. To address this problem, we propose a super-resolution-driven generative adversarial network for activity recognition. To fully take advantage of the latent information in low-resolution images, a powerful network module is employed to super-resolve the extremely low-resolution images with a large scale factor. Then, a general activity recognition network is applied to analyze the super-resolved video clips. Extensive experiments on two public benchmarks were conducted to evaluate the effectiveness of our proposed method. The results demonstrate that our method outperforms several state-of-the-art low-resolution activity recognition approaches.

Download Full-text

Upsampling Real-Time, Low-Resolution CCTV Videos Using Generative Adversarial Networks

Electronics ◽

10.3390/electronics9081312 ◽

2020 ◽

Vol 9 (8) ◽

pp. 1312

Author(s):

Debapriya Hazra ◽

Yung-Cheol Byun

Keyword(s):

High Resolution ◽

Real Time ◽

Super Resolution ◽

Poor Quality ◽

Generative Adversarial Networks ◽

Temporal Consistency ◽

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Benchmark Datasets

Video super-resolution has become an emerging topic in the field of machine learning. The generative adversarial network is a framework that is widely used to develop solutions for low-resolution videos. Video surveillance using closed-circuit television (CCTV) is significant in every field, all over the world. A common problem with CCTV videos is sudden video loss or poor quality. In this paper, we propose a generative adversarial network that implements spatio-temporal generators and discriminators to enhance real-time low-resolution CCTV videos to high-resolution. The proposed model considers both foreground and background motion of a CCTV video and effectively models the spatial and temporal consistency from low-resolution video frames to generate high-resolution videos. Quantitative and qualitative experiments on benchmark datasets, including Kinetics-700, UCF101, HMDB51 and IITH_Helmet2, showed that our model outperforms the existing GAN models for video super-resolution.

Download Full-text

Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy

Sensors ◽

10.3390/s21062164 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2164

Author(s):

Md. Shahinur Alam ◽

Ki-Chul Kwon ◽

Munkh-Uchral Erdenebat ◽

Mohammed Y. Abbass ◽

Md. Ashraful Alam ◽

...

Keyword(s):

High Resolution ◽

Resolution Enhancement ◽

Three Dimensional ◽

Super Resolution ◽

Input Image ◽

Low Resolution ◽

Generative Adversarial Network ◽

Microscopic Object ◽

Integral Imaging ◽

Adversarial Network

The integral imaging microscopy system provides a three-dimensional visualization of a microscopic object. However, it has a low-resolution problem due to the fundamental limitation of the F-number (the aperture stops) by using micro lens array (MLA) and a poor illumination environment. In this paper, a generative adversarial network (GAN)-based super-resolution algorithm is proposed to enhance the resolution where the directional view image is directly fed as input. In a GAN network, the generator regresses the high-resolution output from the low-resolution input image, whereas the discriminator distinguishes between the original and generated image. In the generator part, we use consecutive residual blocks with the content loss to retrieve the photo-realistic original image. It can restore the edges and enhance the resolution by ×2, ×4, and even ×8 times without seriously hampering the image quality. The model is tested with a variety of low-resolution microscopic sample images and successfully generates high-resolution directional view images with better illumination. The quantitative analysis shows that the proposed model performs better for microscopic images than the existing algorithms.

Download Full-text

Low Resolution Face Recognition Using Generative Adversarial Network (GAN)

10.21203/rs.3.rs-36971/v2 ◽

2020 ◽

Author(s):

Howard Martin ◽

Suharjito

Keyword(s):

Face Recognition ◽

Super Resolution ◽

Image Data ◽

Face Image ◽

Image Resolution ◽

Low Resolution ◽

Generative Adversarial Network ◽

Resolution Image ◽

Surveillance Camera ◽

Adversarial Network

Abstract Face recognition has a lot of use on smartphone authentication, finding people, etc. Nowadays, face recognition with a constrained environment has achieved very good performance on accuracy. However, the accuracy of existing face recognition methods will gradually decrease when using a dataset with an unconstrained environment. Face image with an unconstrained environment is usually taken from a surveillance camera. In general, surveillance cameras will be placed on the corner of a room or even on the street. So, the image resolution will be low. Low-resolution image will cause the face very hard to be recognized and the accuracy will eventually decrease. That is the main reason why increasing the accuracy of the Low-Resolution Face Recognition (LRFR) problem is still challenging. This research aimed to solve the Low-Resolution Face Recognition (LRFR) problem. The datasets are YouTube Faces Database (YTF) and Labelled Faces in The Wild (LFW). In this research, face image resolution would be decreased using bicubic linear and became the low-resolution image data. Then super resolution methods as the preprocessing step would increase the image resolution. Super resolution methods used in this research are Super resolution GAN (SRGAN) [1] and Enhanced Super resolution GAN (ESRGAN) [2]. These methods would be compared to reach a better accuracy on solving LRFR problem. After increased the image resolution, the image would be recognized using FaceNet. This research concluded that using super resolution as the preprocessing step for LRFR problem has achieved a higher accuracy compared to [3]. The highest accuracy achieved by using ESRGAN as the preprocessing and FaceNet for face recognition with accuracy of 98.96 % and Validation rate 96.757 %.

Download Full-text

Unpaired Remote Sensing Image Super-Resolution with Multi-Stage Aggregation Networks

Remote Sensing ◽

10.3390/rs13163167 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3167

Author(s):

Lize Zhang ◽

Wen Lu ◽

Yuanfei Huang ◽

Xiaopeng Sun ◽

Hongyi Zhang

Keyword(s):

Remote Sensing ◽

Super Resolution ◽

Perceptual Content ◽

Remote Sensing Images ◽

Low Resolution ◽

Generative Adversarial Network ◽

Imaging Device ◽

Adversarial Network ◽

Multi Stage ◽

Image Super Resolution

Mainstream image super-resolution (SR) methods are generally based on paired training samples. As the high-resolution (HR) remote sensing images are difficult to collect with a limited imaging device, most of the existing remote sensing super-resolution methods try to down-sample the collected original images to generate an auxiliary low-resolution (LR) image and form a paired pseudo HR-LR dataset for training. However, the distribution of the generated LR images is generally inconsistent with the real images due to the limitation of remote sensing imaging devices. In this paper, we propose a perceptually unpaired super-resolution method by constructing a multi-stage aggregation network (MSAN). The optimization of the network depends on consistency losses. In particular, the first phase is to preserve the contents of the super-resolved results, by constraining the content consistency between the down-scaled SR results and the low-quality low-resolution inputs. The second stage minimizes perceptual feature loss between the current result and LR input to constrain perceptual-content consistency. The final phase employs the generative adversarial network (GAN) to adding photo-realistic textures by constraining perceptual-distribution consistency. Numerous experiments on synthetic remote sensing datasets and real remote sensing images show that our method obtains more plausible results than other SR methods quantitatively and qualitatively. The PSNR of our network is 0.06dB higher than the SOTA method—HAN on the UC Merced test set with complex degradation.

Download Full-text

HiCSR: a Hi-C super-resolution framework for producing highly realistic contact maps

10.1101/2020.02.24.961714 ◽

2020 ◽

Author(s):

Michael C. Dimmick ◽

Leo J. Lee ◽

Brendan J. Frey

Keyword(s):

High Resolution ◽

Super Resolution ◽

Cell Types ◽

Supplementary Information ◽

Low Resolution ◽

Generative Adversarial Network ◽

High Resolution Data ◽

Contact Maps ◽

Adversarial Network ◽

And Function

AbstractMotivationHi-C data has enabled the genome-wide study of chromatin folding and architecture, and has led to important discoveries in the structure and function of chromatin conformation. Here, high resolution data plays a particularly important role as many chromatin substructures such as Topologically Associating Domains (TADs) and chromatin loops cannot be adequately studied with low resolution contact maps. However, the high sequencing costs associated with the generation of high resolution Hi-C data has become an experimental barrier. Data driven machine learning models, which allow low resolution Hi-C data to be computationally enhanced, offer a promising avenue to address this challenge.ResultsBy carefully examining the properties of Hi-C maps and integrating various recent advances in deep learning, we developed a Hi-C Super-Resolution (HiCSR) framework capable of accurately recovering the fine details, textures, and substructures found in high resolution contact maps. This was achieved using a novel loss function tailored to the Hi-C enhancement problem which optimizes for an adversarial loss from a Generative Adversarial Network (GAN), a feature reconstruction loss derived from the latent representation of a denoising autoencoder, and a pixel-wise loss. Not only can the resulting framework generate enhanced Hi-C maps more visually similar to the original high resolution maps, it also excels on a suite of reproducibility metrics produced by members of the ENCODE Consortium compared to existing approaches, including HiCPlus, HiCNN, hicGAN and DeepHiC. Finally, we demonstrate that HiCSR is capable of enhancing Hi-C data across sequencing depth, cell types, and species, recovering biologically significant contact domain boundaries.AvailabilityWe make our implementation available for download at: https://github.com/PSI-Lab/[email protected] informationAvailable Online

Download Full-text

D2SC-GAN: Dual Deep-Shallow Channeled Generative Adversarial Network, for Resolving Low-Resolution Faces for Recognition in Classroom Scenarios

IEEE Transactions on Biometrics Behavior and Identity Science ◽

10.1109/tbiom.2020.2983524 ◽

2020 ◽

Vol 2 (3) ◽

pp. 223-234

Author(s):

Avishek Bhattacharjee ◽

Sukhendu Das

Keyword(s):

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network

Download Full-text

Accurate Segmentation for Low Resolution Satellite images by Discriminative Generative Adversarial Network for Identifying Agriculture Fields

Journal of Innovative Image Processing - December 2019 ◽

10.36548/jiip.2021.4.002 ◽

2021 ◽

Vol 3 (4) ◽

pp. 298-310

Author(s):

S. Iwin Thanakumar Joseph

Keyword(s):

Satellite Images ◽

Agricultural Field ◽

Low Resolution ◽

Sar Images ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Crop Density ◽

Agricultural Applications ◽

Crop Condition ◽

Agricultural Regions

Agricultural field identification is still a difficult issue because of the poor resolution of satellite imagery. Monitoring remote harvest and determining the condition of farmlands rely on the digital approach agricultural applications. Therefore, high-resolution photographs have obtained much more attention since they are more efficient in detecting land cover components. In contrast, because of low-resolution repositories of past satellite images used for time series analysis, wavelet decomposition filter-based analysis, free availability, and economic concerns, low-resolution images are still essential. Using low-resolution Synthetic Aperture Radar (SAR) satellite photos, this study proposes a GAN strategy for locating agricultural regions and determining the crop's cultivation state, linked to the initial or harvesting time. An object detector is used in the preprocessing step of training, followed by a transformation technique for extracting feature information and then the GAN strategy for classifying the crop segmented picture. After testing, the suggested algorithm is applied to the database's SAR images, which are further processed and categorized based on the training results. Using this information, the density between the crops is calculated. After zooming in on SAR photos, the crop condition may be categorized based on crop density and crop distance. The Euclidean distance formula is used to calculate the distance. Finally, the findings are compared to other existing approaches to determine the proposed technique's performance using reliable measures.

Download Full-text

Target Detection Method for Low-Resolution Remote Sensing Image Based on ESRGAN and ReDet

Photonics ◽

10.3390/photonics8100431 ◽

2021 ◽

Vol 8 (10) ◽

pp. 431

Author(s):

Yuwu Wang ◽

Guobing Sun ◽

Shengwei Guo

Keyword(s):

Remote Sensing ◽

Target Detection ◽

Detection Method ◽

Recognition Rate ◽

Super Resolution ◽

Remote Sensing Image ◽

Remote Sensing Images ◽

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network

With the widespread use of remote sensing images, low-resolution target detection in remote sensing images has become a hot research topic in the field of computer vision. In this paper, we propose a Target Detection on Super-Resolution Reconstruction (TDoSR) method to solve the problem of low target recognition rates in low-resolution remote sensing images under foggy conditions. The TDoSR method uses the Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) to perform defogging and super-resolution reconstruction of foggy low-resolution remote sensing images. In the target detection part, the Rotation Equivariant Detector (ReDet) algorithm, which has a higher recognition rate at this stage, is used to identify and classify various types of targets. While a large number of experiments have been carried out on the remote sensing image dataset DOTA-v1.5, the results of this paper suggest that the proposed method achieves good results in the target detection of low-resolution foggy remote sensing images. The principal result of this paper demonstrates that the recognition rate of the TDoSR method increases by roughly 20% when compared with low-resolution foggy remote sensing images.

Download Full-text

Fine-tuned Siamese Network with Modified Enhanced Super-Resolution GAN Plus Based on Low Quality Chest X-ray Images for COVID-19 Identification: Algorithm Development and Validation (Preprint)

10.2196/preprints.32915 ◽

2021 ◽

Author(s):

Grace Ugochi Nneji ◽

Jingye Cai ◽

Deng Jianhua ◽

Md Altab Hossin ◽

Happy Nkanta Monday ◽

...

Keyword(s):

Binary Classification ◽

Super Resolution ◽

Automated System ◽

Identification Algorithm ◽

Low Resolution ◽

Generative Adversarial Network ◽

X Ray ◽

Adversarial Network ◽

Chest X Ray ◽

Pooling Strategy

UNSTRUCTURED Background: Coronavirus disease has explosively spread globally since the early January of 2020. With the millions of the death rate of individuals, it is essential for an automated system to be utilized for aiding the clinical diagnosis and reduce time consumption for the image analysis. Objective: Our aim is to rapidly develop an automated AI model to diagnose COVID-19 in CXR images and differentiate COVID-19 from healthy and other pneumonia. Methods: This article presents a GAN-based deep learning application in precisely regaining high-resolution (HR) CXR images from low-resolution (LR) CXR correspondents for COVID-19 identification. Respectively, using the building block of generative adversarial network (GAN), we introduce a modified enhanced super-resolution with generative adversarial network plus (MESRGAN+) to inculcate a connected nonlinear mapping collected from noise-contaminated low-resolution input images to produce deblurred and denoised HR images. As opposed to the latest trend of increasing network elaboration and depth to advance imaging performance, we incorporated an enhanced VGG19 fine-tuned twin network with wavelet pooling strategy in order to extracts distinct features for COVID-19 identification. The qualitative results establish that the proposed model is robust and reliable for COVID-19 screening. Results: We demonstrate the proposed enhanced siamese fine-tuned model with wavelet pooling strategy and modified enhanced super-resolution GAN plus based on low quality images for COVID-19 identification on a publicly available dataset of 11,920 samples of chest x-ray images, each having 2,980 cases of COVID-19 CXR, healthy, viral and bacterial cases for our four-class classification. Furthermore, we performed binary classification of COVID-19 verse healthy cases. The proposed method achieves accuracy of 98.8%, precision of 98.6%, sensitivity of 97.5%, specificity of 98.9%, F1-score of 97.8% and ROC AUC of 98.8% for the multi- class task while for the binary class, the model achieved accuracy of 99.7%, precision of 98.9%, sensitivity of 98.7%, specificity of 99.3%, F1-score of 98.2% and ROC AUC of 99.7%. Conclusions: Our method obtained state-of-the-art (SOTA) performance, according to experimental results, which is helpful for COVID-19 screening. This new conceptual framework is proposed to play an influential task in the issue facing COVID-19 examination and other ailments, using CXR datasets.

Download Full-text