StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

Deep learning applications on computer vision involve the use of large-volume and representative data to obtain state-of-the-art results due to the massive number of parameters to optimise in deep models. However, data are limited with asymmetric distributions in industrial applications due to rare cases, legal restrictions, and high image-acquisition costs. Data augmentation based on deep learning generative adversarial networks, such as StyleGAN, has arisen as a way to create training data with symmetric distributions that may improve the generalisation capability of built models. StyleGAN generates highly realistic images in a variety of domains as a data augmentation strategy but requires a large amount of data to build image generators. Thus, transfer learning in conjunction with generative models are used to build models with small datasets. However, there are no reports on the impact of pre-trained generative models, using transfer learning. In this paper, we evaluate a StyleGAN generative model with transfer learning on different application domains—training with paintings, portraits, Pokémon, bedrooms, and cats—to generate target images with different levels of content variability: bean seeds (low variability), faces of subjects between 5 and 19 years old (medium variability), and charcoal (high variability). We used the first version of StyleGAN due to the large number of publicly available pre-trained models. The Fréchet Inception Distance was used for evaluating the quality of synthetic images. We found that StyleGAN with transfer learning produced good quality images, being an alternative for generating realistic synthetic images in the evaluated domains.

Download Full-text

Copy-Move Forgery Detection (CMFD) Using Deep Learning for Image and Video Forensics

Journal of Imaging ◽

10.3390/jimaging7030059 ◽

2021 ◽

Vol 7 (3) ◽

pp. 59

Author(s):

Yohanna Rodriguez-Ortega ◽

Dora M. Ballesteros ◽

Diego Renza

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Training Data ◽

Video Editing ◽

Forgery Detection ◽

Copy Move Forgery Detection ◽

The Impact ◽

And Training ◽

Selection Of ◽

Traditional Image

With the exponential growth of high-quality fake images in social networks and media, it is necessary to develop recognition algorithms for this type of content. One of the most common types of image and video editing consists of duplicating areas of the image, known as the copy-move technique. Traditional image processing approaches manually look for patterns related to the duplicated content, limiting their use in mass data classification. In contrast, approaches based on deep learning have shown better performance and promising results, but they present generalization problems with a high dependence on training data and the need for appropriate selection of hyperparameters. To overcome this, we propose two approaches that use deep learning, a model by a custom architecture and a model by transfer learning. In each case, the impact of the depth of the network is analyzed in terms of precision (P), recall (R) and F1 score. Additionally, the problem of generalization is addressed with images from eight different open access datasets. Finally, the models are compared in terms of evaluation metrics, and training and inference times. The model by transfer learning of VGG-16 achieves metrics about 10% higher than the model by a custom architecture, however, it requires approximately twice as much inference time as the latter.

Download Full-text

Using Vehicle Synthesis Generative Adversarial Networks to Improve Vehicle Detection in Remote Sensing Images

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi8090390 ◽

2019 ◽

Vol 8 (9) ◽

pp. 390 ◽

Cited By ~ 1

Author(s):

Kun Zheng ◽

Mengfei Wei ◽

Guangmin Sun ◽

Bilal Anas ◽

Yu Li

Keyword(s):

Remote Sensing ◽

Traffic Control ◽

Data Augmentation ◽

Vehicle Detection ◽

Training Data ◽

Generative Adversarial Networks ◽

Remote Sensing Images ◽

Adversarial Networks ◽

Vehicle Data ◽

Augmentation Strategy

Vehicle detection based on very high-resolution (VHR) remote sensing images is beneficial in many fields such as military surveillance, traffic control, and social/economic studies. However, intricate details about the vehicle and the surrounding background provided by VHR images require sophisticated analysis based on massive data samples, though the number of reliable labeled training data is limited. In practice, data augmentation is often leveraged to solve this conflict. The traditional data augmentation strategy uses a combination of rotation, scaling, and flipping transformations, etc., and has limited capabilities in capturing the essence of feature distribution and proving data diversity. In this study, we propose a learning method named Vehicle Synthesis Generative Adversarial Networks (VS-GANs) to generate annotated vehicles from remote sensing images. The proposed framework has one generator and two discriminators, which try to synthesize realistic vehicles and learn the background context simultaneously. The method can quickly generate high-quality annotated vehicle data samples and greatly helps in the training of vehicle detectors. Experimental results show that the proposed framework can synthesize vehicles and their background images with variations and different levels of details. Compared with traditional data augmentation methods, the proposed method significantly improves the generalization capability of vehicle detectors. Finally, the contribution of VS-GANs to vehicle detection in VHR remote sensing images was proved in experiments conducted on UCAS-AOD and NWPU VHR-10 datasets using up-to-date target detection frameworks.

Download Full-text

Deep transfer learning and data augmentation improve glucose levels prediction in type 2 diabetes patients

npj Digital Medicine ◽

10.1038/s41746-021-00480-x ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Yixiang Deng ◽

Lu Lu ◽

Laura Aponte ◽

Angeliki M. Angelidi ◽

Vera Novak ◽

...

Keyword(s):

Type 2 Diabetes ◽

Deep Learning ◽

Blood Glucose ◽

Transfer Learning ◽

Data Augmentation ◽

Generative Models ◽

Patient Specific ◽

Learning Methods ◽

Glucose Levels

AbstractAccurate prediction of blood glucose variations in type 2 diabetes (T2D) will facilitate better glycemic control and decrease the occurrence of hypoglycemic episodes as well as the morbidity and mortality associated with T2D, hence increasing the quality of life of patients. Owing to the complexity of the blood glucose dynamics, it is difficult to design accurate predictive models in every circumstance, i.e., hypo/normo/hyperglycemic events. We developed deep-learning methods to predict patient-specific blood glucose during various time horizons in the immediate future using patient-specific every 30-min long glucose measurements by the continuous glucose monitoring (CGM) to predict future glucose levels in 5 min to 1 h. In general, the major challenges to address are (1) the dataset of each patient is often too small to train a patient-specific deep-learning model, and (2) the dataset is usually highly imbalanced given that hypo- and hyperglycemic episodes are usually much less common than normoglycemia. We tackle these two challenges using transfer learning and data augmentation, respectively. We systematically examined three neural network architectures, different loss functions, four transfer-learning strategies, and four data augmentation techniques, including mixup and generative models. Taken together, utilizing these methodologies we achieved over 95% prediction accuracy and 90% sensitivity for a time period within the clinically useful 1 h prediction horizon that would allow a patient to react and correct either hypoglycemia and/or hyperglycemia. We have also demonstrated that the same network architecture and transfer-learning methods perform well for the type 1 diabetes OhioT1DM public dataset.

Download Full-text

Generative Adversarial Networks for Pre-training of Medical Image Segmentation Networks

10.21203/rs.2.22269/v1 ◽

2020 ◽

Author(s):

Kun Chen ◽

Manning Wang ◽

Zhijian Song

Keyword(s):

Neural Networks ◽

Image Segmentation ◽

Medical Image ◽

Deep Neural Networks ◽

Data Augmentation ◽

Medical Image Segmentation ◽

Training Data ◽

Generative Adversarial Networks ◽

Adversarial Networks ◽

Synthetic Images

Abstract Background: Deep neural networks have been widely used in medical image segmentation and have achieved state-of-the-art performance in many tasks. However, different from the segmentation of natural images or video frames, the manual segmentation of anatomical structures in medical images needs high expertise so the scale of labeled training data is very small, which is a major obstacle for the improvement of deep neural networks performance in medical image segmentation. Methods: In this paper, we proposed a new end-to-end generation-segmentation framework by integrating Generative Adversarial Networks (GAN) and a segmentation network and train them simultaneously. The novelty is that during the training of the GAN, the intermediate synthetic images generated by the generator of the GAN are used to pre-train the segmentation network. As the advances of the training of the GAN, the synthetic images evolve gradually from being very coarse to containing more realistic textures, and these images help train the segmentation network gradually. After the training of GAN, the segmentation network is then fine-tuned by training with the real labeled images. Results: We evaluated the proposed framework on four different datasets, including 2D cardiac dataset and lung dataset, 3D prostate dataset and liver dataset. Compared with original U-net and CE-Net, our framework can achieve better segmentation performance. Our framework also can get better segmentation results than U-net on small datasets. In addition, our framework is more effective than the usual data augmentation methods. Conclusions: The proposed framework can be used as a pre-train method of segmentation network, which helps to get a better segmentation result. Our method can solve the shortcomings of current data augmentation methods to some extent.

Download Full-text

Plastic Gasket Defect Detection Based on Transfer Learning

Scientific Programming ◽

10.1155/2021/5990020 ◽

2021 ◽

Vol 2021 ◽

pp. 1-11

Author(s):

Xieyi Chen ◽

Dongyun Wang ◽

Jinjun Shao ◽

Jun Fan

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Defect Detection ◽

Data Augmentation ◽

Surface Defects ◽

Visual Detection ◽

Training Data

To automatically detect plastic gasket defects, a set of plastic gasket defect visual detection devices based on GoogLeNet Inception-V2 transfer learning was designed and established in this study. The GoogLeNet Inception-V2 deep convolutional neural network (DCNN) was adopted to extract and classify the defect features of plastic gaskets to solve the problem of their numerous surface defects and difficulty in extracting and classifying the features. Deep learning applications require a large amount of training data to avoid model overfitting, but there are few datasets of plastic gasket defects. To address this issue, data augmentation was applied to our dataset. Finally, the performance of the three convolutional neural networks was comprehensively compared. The results showed that the GoogLeNet Inception-V2 transfer learning model had a better performance in less time. It means it had higher accuracy, reliability, and efficiency on the dataset used in this paper.

Download Full-text

Novelty Detection via Contrastive Learning with Negative Data Augmentation

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/84 ◽

2021 ◽

Author(s):

Chengwei Chen ◽

Yuan Xie ◽

Shaohui Lin ◽

Ruizhi Qiao ◽

Jian Zhou ◽

...

Keyword(s):

Data Augmentation ◽

Novelty Detection ◽

Training Data ◽

Detection Methods ◽

Generative Adversarial Networks ◽

Negative Data ◽

Discriminative Ability ◽

Adversarial Networks ◽

Training Mode ◽

Augmentation Strategy

Novelty detection is the process of determining whether a query example differs from the learned training distribution. Previous generative adversarial networks based methods and self-supervised approaches suffer from instability training, mode dropping, and low discriminative ability. We overcome such problems by introducing a novel decoder-encoder framework. Firstly, a generative network (decoder) learns the representation by mapping the initialized latent vector to an image. In particular, this vector is initialized by considering the entire distribution of training data to avoid the problem of mode-dropping. Secondly, a contrastive network (encoder) aims to ``learn to compare'' through mutual information estimation, which directly helps the generative network to obtain a more discriminative representation by using a negative data augmentation strategy. Extensive experiments show that our model has significant superiority over cutting-edge novelty detectors and achieves new state-of-the-art results on various novelty detection benchmarks, e.g. CIFAR10 and DCASE. Moreover, our model is more stable for training in a non-adversarial manner, compared to other adversarial based novelty detection methods.

Download Full-text

Data Augmentation and Transfer Learning Strategies for Reaction Prediction in Low Chemical Data Regimes

10.26434/chemrxiv.13383275 ◽

2020 ◽

Author(s):

Yun Zhang ◽

Ling Wang ◽

Xinqiao Wang ◽

Chengyun Zhang ◽

Jiamin Ge ◽

...

Keyword(s):

Deep Learning ◽

Learning Strategies ◽

Transfer Learning ◽

Data Augmentation ◽

Learning Strategy ◽

Predictive Performance ◽

Learning Model ◽

Training Data ◽

Reaction Prediction ◽

Transformer Model

<p><b>Abstract:</b> Effective and rapid deep learning method to predict chemical reactions contributes to the research and development of organic chemistry and drug discovery. Despite the outstanding capability of deep learning in retrosynthesis and forward synthesis, predictions based on small chemical datasets generally result in low accuracy due to an insufficiency of reaction examples. Here, we introduce a new state art of method, which integrates transfer learning with transformer model to predict the outcomes of the Baeyer-Villiger reaction which is a representative small dataset reaction. The results demonstrate that introducing transfer learning strategy markedly improves the top-1 accuracy of the transformer-transfer learning model (81.8%) over that of the transformer-baseline model (58.4%). Moreover, we further introduce data augmentation to the input reaction SMILES, which allows for better performance and improves the accuracy of the transformer-transfer learning model (86.7%). In summary, both transfer learning and data augmentation methods significantly improve the predictive performance of transformer model, which are powerful methods used in chemistry field to eliminate the restriction of limited training data.</p>

Download Full-text

Data Augmentation and Transfer Learning Strategies for Reaction Prediction in Low Chemical Data Regimes

10.26434/chemrxiv.13383275.v1 ◽

2020 ◽

Author(s):

Yun Zhang ◽

Ling Wang ◽

Xinqiao Wang ◽

Chengyun Zhang ◽

Jiamin Ge ◽

...

Keyword(s):

Deep Learning ◽

Learning Strategies ◽

Transfer Learning ◽

Data Augmentation ◽

Learning Strategy ◽

Predictive Performance ◽

Learning Model ◽

Training Data ◽

Reaction Prediction ◽

Transformer Model

Download Full-text

Data Augmentation Using Generative Adversarial Networks (GANs) For GAN-Based Detection Of Pneumonia And COVID-19 In Chest X-Ray Images

10.21203/rs.3.rs-146161/v1 ◽

2021 ◽

Author(s):

Saman Motamed ◽

Patrik Rogalla ◽

Farzad Khalvati

Keyword(s):

Neural Networks ◽

Data Augmentation ◽

Generative Models ◽

Training Data ◽

Generative Adversarial Networks ◽

X Rays ◽

X Ray ◽

Standard Data ◽

Adversarial Networks ◽

Augmentation Techniques

Abstract Successful training of convolutional neural networks (CNNs) requires a substantial amount of data. With small datasets networks generalize poorly. Data Augmentation techniques improve the generalizability of neural networks by using existing training data more effectively. Standard data augmentation methods, however, produce limited plausible alternative data. Generative Adversarial Networks (GANs) have been utilized to generate new data and improve the performance of CNNs. Nevertheless, data augmentation techniques for training GANs are under-explored compared to CNNs. In this work, we propose a new GAN architecture for augmentation of chest X-rays for semi-supervised detection of pneumonia and COVID-19 using generative models. We show that the proposed GAN can be used to effectively augment data and improve classification accuracy of disease in chest X-rays for pneumonia and COVID-19. We compare our augmentation GAN model with Deep Convolutional GAN and traditional augmentation methods (rotate, zoom, etc) on two different X-ray datasets and show our GAN-based augmentation method surpasses other augmentation methods for training a GAN in detecting anomalies in X-ray images.

Download Full-text

A deep learning segmentation strategy that minimizes the amount of manually annotated images

F1000Research ◽

10.12688/f1000research.52026.1 ◽

2021 ◽

Vol 10 ◽

pp. 256

Author(s):

Thierry Pécot ◽

Alexander Alekseyenko ◽

Kristin Wallace

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Training Data ◽

Artificial Data ◽

Deep Convolutional Neural Networks ◽

Generative Adversarial Network ◽

Data Set ◽

Segmentation Strategy ◽

Adversarial Network ◽

The Impact

Deep learning has revolutionized the automatic processing of images. While deep convolutional neural networks have demonstrated astonishing segmentation results for many biological objects acquired with microscopy, this technology's good performance relies on large training datasets. In this paper, we present a strategy to minimize the amount of time spent in manually annotating images for segmentation. It involves using an efficient and open source annotation tool, the artificial increase of the training data set with data augmentation, the creation of an artificial data set with a conditional generative adversarial network and the combination of semantic and instance segmentations. We evaluate the impact of each of these approaches for the segmentation of nuclei in 2D widefield images of human precancerous polyp biopsies in order to define an optimal strategy.

Download Full-text