Fine-tuned Siamese Network with Modified Enhanced Super-Resolution GAN Plus Based on Low Quality Chest X-ray Images for COVID-19 Identification: Algorithm Development and Validation (Preprint)

UNSTRUCTURED Background: Coronavirus disease has explosively spread globally since the early January of 2020. With the millions of the death rate of individuals, it is essential for an automated system to be utilized for aiding the clinical diagnosis and reduce time consumption for the image analysis. Objective: Our aim is to rapidly develop an automated AI model to diagnose COVID-19 in CXR images and differentiate COVID-19 from healthy and other pneumonia. Methods: This article presents a GAN-based deep learning application in precisely regaining high-resolution (HR) CXR images from low-resolution (LR) CXR correspondents for COVID-19 identification. Respectively, using the building block of generative adversarial network (GAN), we introduce a modified enhanced super-resolution with generative adversarial network plus (MESRGAN+) to inculcate a connected nonlinear mapping collected from noise-contaminated low-resolution input images to produce deblurred and denoised HR images. As opposed to the latest trend of increasing network elaboration and depth to advance imaging performance, we incorporated an enhanced VGG19 fine-tuned twin network with wavelet pooling strategy in order to extracts distinct features for COVID-19 identification. The qualitative results establish that the proposed model is robust and reliable for COVID-19 screening. Results: We demonstrate the proposed enhanced siamese fine-tuned model with wavelet pooling strategy and modified enhanced super-resolution GAN plus based on low quality images for COVID-19 identification on a publicly available dataset of 11,920 samples of chest x-ray images, each having 2,980 cases of COVID-19 CXR, healthy, viral and bacterial cases for our four-class classification. Furthermore, we performed binary classification of COVID-19 verse healthy cases. The proposed method achieves accuracy of 98.8%, precision of 98.6%, sensitivity of 97.5%, specificity of 98.9%, F1-score of 97.8% and ROC AUC of 98.8% for the multi- class task while for the binary class, the model achieved accuracy of 99.7%, precision of 98.9%, sensitivity of 98.7%, specificity of 99.3%, F1-score of 98.2% and ROC AUC of 99.7%. Conclusions: Our method obtained state-of-the-art (SOTA) performance, according to experimental results, which is helpful for COVID-19 screening. This new conceptual framework is proposed to play an influential task in the issue facing COVID-19 examination and other ailments, using CXR datasets.

Download Full-text

A Hybrid Capsule Network for Pneumonia Detection Using Image Augmentation Based on Generative Adversarial Network

Traitement du signal ◽

10.18280/ts.380309 ◽

2021 ◽

Vol 38 (3) ◽

pp. 619-627

Author(s):

Kazim Firildak ◽

Muhammed Fatih Talu

Keyword(s):

Classification Accuracy ◽

Data Augmentation ◽

Model Parameters ◽

Classification Problems ◽

Generative Adversarial Network ◽

X Ray ◽

Discrimination Capability ◽

Adversarial Network ◽

Original Dataset ◽

Chest X Ray

Pneumonia, featured by inflammation of the air sacs in one or both lungs, is usually detected by examining chest X-ray images. This paper probes into the classification models that can distinguish between normal and pneumonia images. As is known, trained networks like AlexNet and GoogleNet are deep network architectures, which are widely adopted to solve many classification problems. They have been adapted to the target datasets, and employed to classify new data generated through transfer learning. However, the classical architectures are not accurate enough for the diagnosis of pneumonia. Therefore, this paper designs a capsule network with high discrimination capability, and trains the network on Kaggle’s online pneumonia dataset, which contains chest X-ray images of many adults and children. The original dataset consists of 1,583 normal images, and 4,273 pneumonia images. Then, two data augmentation approaches were applied to the dataset, and their effects on classification accuracy were compared in details. The model parameters were optimized through five different experiments. The results show that the highest classification accuracy (93.91% even on small images) was achieved by the capsule network, coupled with data augmentation by generative adversarial network (GAN), using optimized parameters. This network outperformed the classical strategies.

Download Full-text

Upsampling Real-Time, Low-Resolution CCTV Videos Using Generative Adversarial Networks

Electronics ◽

10.3390/electronics9081312 ◽

2020 ◽

Vol 9 (8) ◽

pp. 1312

Author(s):

Debapriya Hazra ◽

Yung-Cheol Byun

Keyword(s):

High Resolution ◽

Real Time ◽

Super Resolution ◽

Poor Quality ◽

Generative Adversarial Networks ◽

Temporal Consistency ◽

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Benchmark Datasets

Video super-resolution has become an emerging topic in the field of machine learning. The generative adversarial network is a framework that is widely used to develop solutions for low-resolution videos. Video surveillance using closed-circuit television (CCTV) is significant in every field, all over the world. A common problem with CCTV videos is sudden video loss or poor quality. In this paper, we propose a generative adversarial network that implements spatio-temporal generators and discriminators to enhance real-time low-resolution CCTV videos to high-resolution. The proposed model considers both foreground and background motion of a CCTV video and effectively models the spatial and temporal consistency from low-resolution video frames to generate high-resolution videos. Quantitative and qualitative experiments on benchmark datasets, including Kinetics-700, UCF101, HMDB51 and IITH_Helmet2, showed that our model outperforms the existing GAN models for video super-resolution.

Download Full-text

Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy

Sensors ◽

10.3390/s21062164 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2164

Author(s):

Md. Shahinur Alam ◽

Ki-Chul Kwon ◽

Munkh-Uchral Erdenebat ◽

Mohammed Y. Abbass ◽

Md. Ashraful Alam ◽

...

Keyword(s):

High Resolution ◽

Resolution Enhancement ◽

Three Dimensional ◽

Super Resolution ◽

Input Image ◽

Low Resolution ◽

Generative Adversarial Network ◽

Microscopic Object ◽

Integral Imaging ◽

Adversarial Network

The integral imaging microscopy system provides a three-dimensional visualization of a microscopic object. However, it has a low-resolution problem due to the fundamental limitation of the F-number (the aperture stops) by using micro lens array (MLA) and a poor illumination environment. In this paper, a generative adversarial network (GAN)-based super-resolution algorithm is proposed to enhance the resolution where the directional view image is directly fed as input. In a GAN network, the generator regresses the high-resolution output from the low-resolution input image, whereas the discriminator distinguishes between the original and generated image. In the generator part, we use consecutive residual blocks with the content loss to retrieve the photo-realistic original image. It can restore the edges and enhance the resolution by ×2, ×4, and even ×8 times without seriously hampering the image quality. The model is tested with a variety of low-resolution microscopic sample images and successfully generates high-resolution directional view images with better illumination. The quantitative analysis shows that the proposed model performs better for microscopic images than the existing algorithms.

Download Full-text

Extreme Low-Resolution Activity Recognition Using a Super-Resolution-Oriented Generative Adversarial Network

Micromachines ◽

10.3390/mi12060670 ◽

2021 ◽

Vol 12 (6) ◽

pp. 670

Author(s):

Mingzheng Hou ◽

Song Liu ◽

Jiliu Zhou ◽

Yi Zhang ◽

Ziliang Feng

Keyword(s):

Activity Recognition ◽

Large Scale ◽

Super Resolution ◽

General Activity ◽

Low Resolution ◽

Generative Adversarial Network ◽

Video Clips ◽

Adversarial Network ◽

Efficient Recognition ◽

Low Resolution Images

Activity recognition is a fundamental and crucial task in computer vision. Impressive results have been achieved for activity recognition in high-resolution videos, but for extreme low-resolution videos, which capture the action information at a distance and are vital for preserving privacy, the performance of activity recognition algorithms is far from satisfactory. The reason is that extreme low-resolution (e.g., 12 × 16 pixels) images lack adequate scene and appearance information, which is needed for efficient recognition. To address this problem, we propose a super-resolution-driven generative adversarial network for activity recognition. To fully take advantage of the latent information in low-resolution images, a powerful network module is employed to super-resolve the extremely low-resolution images with a large scale factor. Then, a general activity recognition network is applied to analyze the super-resolved video clips. Extensive experiments on two public benchmarks were conducted to evaluate the effectiveness of our proposed method. The results demonstrate that our method outperforms several state-of-the-art low-resolution activity recognition approaches.

Download Full-text

Low Resolution Face Recognition Using Generative Adversarial Network (GAN)

10.21203/rs.3.rs-36971/v2 ◽

2020 ◽

Author(s):

Howard Martin ◽

Suharjito

Keyword(s):

Face Recognition ◽

Super Resolution ◽

Image Data ◽

Face Image ◽

Image Resolution ◽

Low Resolution ◽

Generative Adversarial Network ◽

Resolution Image ◽

Surveillance Camera ◽

Adversarial Network

Abstract Face recognition has a lot of use on smartphone authentication, finding people, etc. Nowadays, face recognition with a constrained environment has achieved very good performance on accuracy. However, the accuracy of existing face recognition methods will gradually decrease when using a dataset with an unconstrained environment. Face image with an unconstrained environment is usually taken from a surveillance camera. In general, surveillance cameras will be placed on the corner of a room or even on the street. So, the image resolution will be low. Low-resolution image will cause the face very hard to be recognized and the accuracy will eventually decrease. That is the main reason why increasing the accuracy of the Low-Resolution Face Recognition (LRFR) problem is still challenging. This research aimed to solve the Low-Resolution Face Recognition (LRFR) problem. The datasets are YouTube Faces Database (YTF) and Labelled Faces in The Wild (LFW). In this research, face image resolution would be decreased using bicubic linear and became the low-resolution image data. Then super resolution methods as the preprocessing step would increase the image resolution. Super resolution methods used in this research are Super resolution GAN (SRGAN) [1] and Enhanced Super resolution GAN (ESRGAN) [2]. These methods would be compared to reach a better accuracy on solving LRFR problem. After increased the image resolution, the image would be recognized using FaceNet. This research concluded that using super resolution as the preprocessing step for LRFR problem has achieved a higher accuracy compared to [3]. The highest accuracy achieved by using ESRGAN as the preprocessing and FaceNet for face recognition with accuracy of 98.96 % and Validation rate 96.757 %.

Download Full-text

Unpaired Remote Sensing Image Super-Resolution with Multi-Stage Aggregation Networks

Remote Sensing ◽

10.3390/rs13163167 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3167

Author(s):

Lize Zhang ◽

Wen Lu ◽

Yuanfei Huang ◽

Xiaopeng Sun ◽

Hongyi Zhang

Keyword(s):

Remote Sensing ◽

Super Resolution ◽

Perceptual Content ◽

Remote Sensing Images ◽

Low Resolution ◽

Generative Adversarial Network ◽

Imaging Device ◽

Adversarial Network ◽

Multi Stage ◽

Image Super Resolution

Mainstream image super-resolution (SR) methods are generally based on paired training samples. As the high-resolution (HR) remote sensing images are difficult to collect with a limited imaging device, most of the existing remote sensing super-resolution methods try to down-sample the collected original images to generate an auxiliary low-resolution (LR) image and form a paired pseudo HR-LR dataset for training. However, the distribution of the generated LR images is generally inconsistent with the real images due to the limitation of remote sensing imaging devices. In this paper, we propose a perceptually unpaired super-resolution method by constructing a multi-stage aggregation network (MSAN). The optimization of the network depends on consistency losses. In particular, the first phase is to preserve the contents of the super-resolved results, by constraining the content consistency between the down-scaled SR results and the low-quality low-resolution inputs. The second stage minimizes perceptual feature loss between the current result and LR input to constrain perceptual-content consistency. The final phase employs the generative adversarial network (GAN) to adding photo-realistic textures by constraining perceptual-distribution consistency. Numerous experiments on synthetic remote sensing datasets and real remote sensing images show that our method obtains more plausible results than other SR methods quantitatively and qualitatively. The PSNR of our network is 0.06dB higher than the SOTA method—HAN on the UC Merced test set with complex degradation.

Download Full-text

HiCSR: a Hi-C super-resolution framework for producing highly realistic contact maps

10.1101/2020.02.24.961714 ◽

2020 ◽

Author(s):

Michael C. Dimmick ◽

Leo J. Lee ◽

Brendan J. Frey

Keyword(s):

High Resolution ◽

Super Resolution ◽

Cell Types ◽

Supplementary Information ◽

Low Resolution ◽

Generative Adversarial Network ◽

High Resolution Data ◽

Contact Maps ◽

Adversarial Network ◽

And Function

AbstractMotivationHi-C data has enabled the genome-wide study of chromatin folding and architecture, and has led to important discoveries in the structure and function of chromatin conformation. Here, high resolution data plays a particularly important role as many chromatin substructures such as Topologically Associating Domains (TADs) and chromatin loops cannot be adequately studied with low resolution contact maps. However, the high sequencing costs associated with the generation of high resolution Hi-C data has become an experimental barrier. Data driven machine learning models, which allow low resolution Hi-C data to be computationally enhanced, offer a promising avenue to address this challenge.ResultsBy carefully examining the properties of Hi-C maps and integrating various recent advances in deep learning, we developed a Hi-C Super-Resolution (HiCSR) framework capable of accurately recovering the fine details, textures, and substructures found in high resolution contact maps. This was achieved using a novel loss function tailored to the Hi-C enhancement problem which optimizes for an adversarial loss from a Generative Adversarial Network (GAN), a feature reconstruction loss derived from the latent representation of a denoising autoencoder, and a pixel-wise loss. Not only can the resulting framework generate enhanced Hi-C maps more visually similar to the original high resolution maps, it also excels on a suite of reproducibility metrics produced by members of the ENCODE Consortium compared to existing approaches, including HiCPlus, HiCNN, hicGAN and DeepHiC. Finally, we demonstrate that HiCSR is capable of enhancing Hi-C data across sequencing depth, cell types, and species, recovering biologically significant contact domain boundaries.AvailabilityWe make our implementation available for download at: https://github.com/PSI-Lab/[email protected] informationAvailable Online

Download Full-text

Target Detection Method for Low-Resolution Remote Sensing Image Based on ESRGAN and ReDet

Photonics ◽

10.3390/photonics8100431 ◽

2021 ◽

Vol 8 (10) ◽

pp. 431

Author(s):

Yuwu Wang ◽

Guobing Sun ◽

Shengwei Guo

Keyword(s):

Remote Sensing ◽

Target Detection ◽

Detection Method ◽

Recognition Rate ◽

Super Resolution ◽

Remote Sensing Image ◽

Remote Sensing Images ◽

Low Resolution ◽

Generative Adversarial Network ◽

Adversarial Network

With the widespread use of remote sensing images, low-resolution target detection in remote sensing images has become a hot research topic in the field of computer vision. In this paper, we propose a Target Detection on Super-Resolution Reconstruction (TDoSR) method to solve the problem of low target recognition rates in low-resolution remote sensing images under foggy conditions. The TDoSR method uses the Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) to perform defogging and super-resolution reconstruction of foggy low-resolution remote sensing images. In the target detection part, the Rotation Equivariant Detector (ReDet) algorithm, which has a higher recognition rate at this stage, is used to identify and classify various types of targets. While a large number of experiments have been carried out on the remote sensing image dataset DOTA-v1.5, the results of this paper suggest that the proposed method achieves good results in the target detection of low-resolution foggy remote sensing images. The principal result of this paper demonstrates that the recognition rate of the TDoSR method increases by roughly 20% when compared with low-resolution foggy remote sensing images.

Download Full-text

Generation of Synthetic Chest X-ray Images and Detection of COVID-19: A Deep Learning Based Approach

Diagnostics ◽

10.3390/diagnostics11050895 ◽

2021 ◽

Vol 11 (5) ◽

pp. 895

Author(s):

Yash Karbhari ◽

Arpan Basu ◽

Zong-Woo Geem ◽

Gi-Tae Han ◽

Ram Sarkar

Keyword(s):

Deep Learning ◽

Distress Syndrome ◽

Harmony Search ◽

Nasal Discharge ◽

X Rays ◽

Generative Adversarial Network ◽

Radiographic Images ◽

X Ray ◽

Adversarial Network ◽

Chest X Ray

COVID-19 is a disease caused by the SARS-CoV-2 virus. The COVID-19 virus spreads when a person comes into contact with an affected individual. This is mainly through drops of saliva or nasal discharge. Most of the affected people have mild symptoms while some people develop acute respiratory distress syndrome (ARDS), which damages organs like the lungs and heart. Chest X-rays (CXRs) have been widely used to identify abnormalities that help in detecting the COVID-19 virus. They have also been used as an initial screening procedure for individuals highly suspected of being infected. However, the availability of radiographic CXRs is still scarce. This can limit the performance of deep learning (DL) based approaches for COVID-19 detection. To overcome these limitations, in this work, we developed an Auxiliary Classifier Generative Adversarial Network (ACGAN), to generate CXRs. Each generated X-ray belongs to one of the two classes COVID-19 positive or normal. To ensure the goodness of the synthetic images, we performed some experimentation on the obtained images using the latest Convolutional Neural Networks (CNNs) to detect COVID-19 in the CXRs. We fine-tuned the models and achieved more than 98% accuracy. After that, we also performed feature selection using the Harmony Search (HS) algorithm, which reduces the number of features while retaining classification accuracy. We further release a GAN-generated dataset consisting of 500 COVID-19 radiographic images.

Download Full-text

RANDGAN: Randomized generative adversarial network for detection of COVID-19 in chest X-ray

Scientific Reports ◽

10.1038/s41598-021-87994-2 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Saman Motamed ◽

Patrik Rogalla ◽

Farzad Khalvati

Keyword(s):

Health Care ◽

Medical Images ◽

Health Care Systems ◽

Generative Adversarial Networks ◽

X Rays ◽

Generative Adversarial Network ◽

X Ray ◽

Adversarial Network ◽

Care Systems ◽

Chest X Ray

AbstractCOVID-19 spread across the globe at an immense rate and has left healthcare systems incapacitated to diagnose and test patients at the needed rate. Studies have shown promising results for detection of COVID-19 from viral bacterial pneumonia in chest X-rays. Automation of COVID-19 testing using medical images can speed up the testing process of patients where health care systems lack sufficient numbers of the reverse-transcription polymerase chain reaction tests. Supervised deep learning models such as convolutional neural networks need enough labeled data for all classes to correctly learn the task of detection. Gathering labeled data is a cumbersome task and requires time and resources which could further strain health care systems and radiologists at the early stages of a pandemic such as COVID-19. In this study, we propose a randomized generative adversarial network (RANDGAN) that detects images of an unknown class (COVID-19) from known and labelled classes (Normal and Viral Pneumonia) without the need for labels and training data from the unknown class of images (COVID-19). We used the largest publicly available COVID-19 chest X-ray dataset, COVIDx, which is comprised of Normal, Pneumonia, and COVID-19 images from multiple public databases. In this work, we use transfer learning to segment the lungs in the COVIDx dataset. Next, we show why segmentation of the region of interest (lungs) is vital to correctly learn the task of classification, specifically in datasets that contain images from different resources as it is the case for the COVIDx dataset. Finally, we show improved results in detection of COVID-19 cases using our generative model (RANDGAN) compared to conventional generative adversarial networks for anomaly detection in medical images, improving the area under the ROC curve from 0.71 to 0.77.

Download Full-text