scholarly journals Normalization of HE-stained histological images using cycle consistent generative adversarial networks

2021 ◽  
Vol 16 (1) ◽  
Author(s):  
Marlen Runz ◽  
Daniel Rusche ◽  
Stefan Schmidt ◽  
Martin R. Weihrauch ◽  
Jürgen Hesser ◽  
...  
Keyword(s):  
Image Acquisition ◽  
Similarity Measures ◽  
Staining Intensity ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Inverse Mapping ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Link Type ◽  
Histological Images

Abstract Background Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these variances are called image normalization techniques. Methods In this paper, we investigate the potential of CycleGAN (cycle consistent Generative Adversarial Network) for color normalization in hematoxylin-eosin stained histological images using daily clinical data with consideration of the variability of internal staining protocol variations. The network consists of a generator network GB that learns to map an image X from a source domain A to a target domain B, i.e. GB:XA→XB. In addition, a discriminator network DB is trained to distinguish whether an image from domain B is real or generated. The same process is applied to another generator-discriminator pair (GA,DA), for the inverse mapping GA:XB→XA. Cycle consistency ensures that a generated image is close to its original when being mapped backwards (GA(GB(XA))≈XA and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma data set for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. In addition, we apply stain normalization on pathological lymph node data from our institute and test the gain from normalization on a ResNet classifier pre-trained on the Camelyon16 data set. Results Qualitative results of the images generated by our network are compared to original color distributions. Our evaluation indicates that by mapping images to a target domain, the similarity training images from that domain improves up to 96%. We also achieve a high cycle consistency for the generator networks by obtaining similarity indices greater than 0.9. When applying the CycleGAN normalization to HE-stain images from our institute the kappa-value of the ResNet-model that is only trained on Camelyon16 data is increased more than 50%. Conclusions CycleGANs have proven to efficiently normalize HE-stained images. The approach compensates for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions.The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch. The data set supporting the solutions is available at 10.11588/data/8LKEZF.

scholarly journals Normalization of HE-Stained Histological Images using Cycle Consistent Generative Adversarial Networks

2021 ◽  
Author(s):  
Marlen Runz ◽  
Daniel Rusche ◽  
Martin R Weihrauch ◽  
Jürgen Hesser ◽  
Cleo-Aron Weis
Keyword(s):  
Image Acquisition ◽  
Similarity Measures ◽  
Staining Intensity ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Inverse Mapping ◽  
Generative Adversarial Network ◽  
Source Domain ◽  
Adversarial Network ◽  
Histological Images

Abstract Background: Histological images show huge variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. The variance can impede many image analyzes such as staining intensity evaluation or classification. Methods to reduce these variances are gathered under the term image normalization. Methods: We present the application of CylceGAN - a cycle consistent Generative Adversarial Network for color normalization in hematoxylin-eosin stained histological images using typical clinical data including variability of internal staining. The network consists of a generator network GB that learns to map an image X from a source domain A to a target domain B, i.e. GB : XA → XB. In addition, a discriminator network DB is trained to distinguish whether an image from domain B is an original or generated one. The same process is applied to another generator-discriminator pair (GA, DA), for the inverse mapping GA : XB → XA. Cycle consistency ensures that the generated image is close to the original image when being mapped backwards (GA(GB(XA)) ≈ XA and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma dataset for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. Results: We present qualitative results of the images generated by our network compared to the original color distributions. Our evaluation shows that by mapping images from a source domain to a target domain, the similarity to original images from the target domain improve up to 96%. We also achieve a high cycle consistency for the inverse mapping by obtaining similarity indices bigger than 0.9. Conclusions: CycleGANs have proven to efficiently normalize HE-stained images. The approach enables to compensate for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions. The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch. The dataset supporting the solutions is available at https://heidata.uni-heidelberg. de/privateurl.xhtml?token=12493b50-1538-4bdf-aca5-03352a1399a8.

scholarly journals A Smart Surveillance System for Uncooperative Gait Recognition Using Cycle Consistent Generative Adversarial Networks (CCGANs)

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Wafaa Adnan Alsaggaf ◽  
Irfan Mehmood ◽  
Enas Fawai Khairullah ◽  
Samar Alhuraiji ◽  
Maha Farouk S. Sabir ◽  
...  
Keyword(s):  
Gait Recognition ◽  
Recognition System ◽  
Research Area ◽  
Generative Adversarial Networks ◽  
Normal Walking ◽  
Target Domain ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Adversarial Network ◽  
Smart Surveillance

Surveillance remains an important research area, and it has many applications. Smart surveillance requires a high level of accuracy even when persons are uncooperative. Gait Recognition is the study of recognizing people by the way they walk even when they are unwilling to cooperate. It is another form of a behavioral biometric system in which unique attributes of an individual’s gait are analyzed to determine their identity. On the other hand, one of the big limitations of the gait recognition system is uncooperative environments in which both gallery and probe sets are made under different and unknown walking conditions. In order to tackle this problem, we propose a deep learning-based method that is trained on individuals with the normal walking condition, and to deal with an uncooperative environment and recognize the individual with any dynamic walking conditions, a cycle consistent generative adversarial network is used. This method translates a GEI disturbed from different covariate factors to a normal GEI. It works like unsupervised learning, and during its training, a GEI disrupts from different covariate factors of each individual and acts as a source domain while the normal walking conditions of individuals are our target domain to which translation is required. The cycle consistent GANs automatically find an individual pair with the help of the Cycle Loss function and generate the required GEI, which is tested by the CNN model to predict the person ID. The proposed system is evaluated over a publicly available data set named CASIA-B, and it achieved excellent results. Moreover, this system can be implemented in sensitive areas, like banks, seminar halls (events), airports, embassies, shopping malls, police stations, military areas, and other public service areas for security purposes.

scholarly journals Generative adversarial networks for construction of virtual populations of mechanistic models: simulations to study Omecamtiv Mecarbil action

2021 ◽  
Author(s):  
Jaimit Parikh ◽  
Timothy Rumbell ◽  
Xenia Butova ◽  
Tatiana Myachina ◽  
Jorge Corral Acero ◽  
...  
Keyword(s):  
Calcium Sensitivity ◽  
Mechanistic Modeling ◽  
Hill Coefficient ◽  
Generative Adversarial Networks ◽  
Model Parameters ◽  
Inverse Mapping ◽  
Generative Adversarial Network ◽  
Complex Biological System ◽  
Omecamtiv Mecarbil

AbstractBiophysical models are increasingly used to gain mechanistic insights by fitting and reproducing experimental and clinical data. The inherent variability in the recorded datasets, however, presents a key challenge. In this study, we present a novel approach, which integrates mechanistic modeling and machine learning to analyze in vitro cardiac mechanics data and solve the inverse problem of model parameter inference. We designed a novel generative adversarial network (GAN) and employed it to construct virtual populations of cardiac ventricular myocyte models in order to study the action of Omecamtiv Mecarbil (OM), a positive cardiac inotrope. Populations of models were calibrated from mechanically unloaded myocyte shortening recordings obtained in experiments on rat myocytes in the presence and absence of OM. The GAN was able to infer model parameters while incorporating prior information about which model parameters OM targets. The generated populations of models reproduced variations in myocyte contraction recorded during in vitro experiments and provided improved understanding of OM’s mechanism of action. Inverse mapping of the experimental data using our approach suggests a novel action of OM, whereby it modifies interactions between myosin and tropomyosin proteins. To validate our approach, the inferred model parameters were used to replicate other in vitro experimental protocols, such as skinned preparations demonstrating an increase in calcium sensitivity and a decrease in the Hill coefficient of the force–calcium (F–Ca) curve under OM action. Our approach thereby facilitated the identification of the mechanistic underpinnings of experimental observations and the exploration of different hypotheses regarding variability in this complex biological system.

scholarly journals Imbalanced Fault Diagnosis of Rotating Machinery Based on Deep Generative Adversarial Networks with Gradient Penalty

Processes ◽  
2021 ◽  
Vol 9 (10) ◽  
pp. 1751
Author(s):  
Junqi Luo ◽  
Liucun Zhu ◽  
Quanfang Li ◽  
Daopeng Liu ◽  
Mingyou Chen
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Rotating Machinery ◽  
Generative Adversarial Networks ◽  
Support Vector ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Combination Methods ◽  
Generation Network ◽  
Noise Factors

In practical industrial application, the fault samples collected from rotating machinery are frequently unbalanced, which will create difficulties when it comes to diagnosis. Besides, the variation of working conditions and noise factors will further reduce the diagnosis’s accuracy and stability. Considering the above problems, we established a model based on deep Wasserstein generative adversarial network with gradient penalty (DWGANGP). In this model, the unbalanced fault data set will first be trained by the sample generation network to generate synthetic samples, which will be used to restore the balance. A one-dimensional convolutional neural network with a specific structure is then used as the fault diagnosis network to classify the reconstructed equilibrium samples. The experimental results show that the proposed sample generation network can generate high-quality synthetic samples under highly imbalanced data, and the diagnostic network has a fast training convergence. Compared to the combination methods of support vector machines, back propagation neural network and deep belief network, our method has a 74% average accuracy in all unbalanced experimental conditions, which has 64%, 69% and 87% averages leading, respectively.

scholarly journals 3D Design Using Generative Adversarial Networks and Physics-Based Validation

2019 ◽  
Vol 142 (7) ◽  
Author(s):  
Dule Shu ◽  
James Cunningham ◽  
Gary Stump ◽  
Simon W. Miller ◽  
Michael A. Yukish ◽  
...  
Keyword(s):  
Evaluation Process ◽  
3D Models ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
3D Design ◽  
Generative Adversarial Network ◽  
Data Set ◽  
High Performing ◽  
Adversarial Network

Abstract The authors present a generative adversarial network (GAN) model that demonstrates how to generate 3D models in their native format so that they can be either evaluated using complex simulation environments or realized using methods such as additive manufacturing. Once initially trained, the GAN can create additional training data itself by generating new designs, evaluating them in a physics-based virtual environment, and adding the high performing ones to the training set. A case study involving a GAN model that is initially trained on 4045 3D aircraft models is used for demonstration, where a training data set that has been updated with GAN-generated and evaluated designs results in enhanced model generation, in both the geometric feasibility and performance of the designs. Z-tests on the performance scores of the generated aircraft models indicate a statistically significant improvement in the functionality of the generated models after three iterations of the training-evaluation process. In the case study, a number of techniques are explored to structure the generate-evaluate process in order to balance the need to generate feasible designs with the need for innovative designs.

Ground-roll attenuation using generative adversarial networks

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA255-WA267 ◽  
Author(s):  
Yijun Yuan ◽  
Xu Si ◽  
Yue Zheng
Keyword(s):  
Seismic Data ◽  
Signal To Noise Ratio ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Coherent Noise ◽  
Traditional Methods ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Persistent Problem ◽  
Ground Roll

Ground roll is a persistent problem in land seismic data. This type of coherent noise often contaminates seismic signals and severely reduces the signal-to-noise ratio of seismic data. A variety of methods for addressing ground-roll attenuation have been developed. However, existing methods are limited, especially when using real land seismic data. For example, when ground roll and reflections overlap in the time or frequency domains, traditional methods cannot completely separate them and they often distort the signals during the suppression process. We have developed a generative adversarial network (GAN) to attenuate ground roll in seismic data. Unlike traditional methods for ground-roll attenuation dependent on various filters, the GAN method is based on a large training data set that includes pairs of data with and without ground roll. After training the neural network with the training data, the network can identify and filter out any noise in the data. To fulfill this purpose, the proposed method uses a generator and a discriminator. Through network training, the generator learns to create the data that can fool the discriminator, and the discriminator can then distinguish between the data produced by the generator and the training data. As a result of the competition between generators and discriminators, generators produce better images whereas discriminators accurately recognize targets. Tests on synthetic and real land seismic data show that the proposed method effectively reveals reflections masked by the ground roll and obtains better results in the attenuation of ground roll and in the preservation of signals compared to the three other methods.

Super-resolution in Structural Geological Models.

2021 ◽  
Author(s):  
Mustaeen Ur Rehman Qazi ◽  
Florian Wellmann
Keyword(s):  
High Resolution ◽  
Structural Model ◽  
Mean Squared Error ◽  
Super Resolution ◽  
Generative Adversarial Networks ◽  
Low Resolution ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Adversarial Networks ◽  
Geological Models

<p>Structural geological models are often calculated on a specific spatial resolution – for example in the form of grid representations, or when surfaces are extracted from implicit fields. However, the structural inventory in these models is limited by the underlying mathematical formulations. It is therefore logical that, above a certain resolution, no additional information is added to the representation.</p><p>We evaluate here if Deep Neural Networks can be trained to obtain a high-resolution representation based on a low-resolution structural model, at different levels of resolution. More specifically, we test the use of state-of-the-art Generative Adversarial Networks (GAN’s) for image superresolution in the context of 2-D geological model sections. These techniques aim to learn the hidden structure or information in high resolution image data set and then reproduce highly detailed and super resolved image from its low resolution counterpart. We propose the use of Generative Adversarial Networks GANS for super resolution of geological images and 2D geological models represented as images. In this work a generative adversarial network called SRGAN has been used which uses a perceptual loss function consisting of an adversarial loss, mean squared error loss and content loss for photo realistic image super resolution. First results are promising, but challenges remain due to the different interpretation of color in images for which these GAN’s are typically used, whereas we are mostly interested in structures.</p>

scholarly journals Generating Scenery Images with Larger Variety According to User Descriptions

2021 ◽  
Vol 11 (21) ◽  
pp. 10224
Author(s):  
Hsu-Yung Cheng ◽  
Chih-Chang Yu
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Memory Cell ◽  
Generative Adversarial Networks ◽  
Image Generation ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Hidden Layer

In this paper, a framework based on generative adversarial networks is proposed to perform nature-scenery generation according to descriptions from the users. The desired place, time and seasons of the generated scenes can be specified with the help of text-to-image generation techniques. The framework improves and modifies the architecture of a generative adversarial network with attention models by adding the imagination models. The proposed attentional and imaginative generative network uses the hidden layer information to initialize the memory cell of the recurrent neural network to produce the desired photos. A data set containing different categories of scenery images is established to train the proposed system. The experiments validate that the proposed method is able to increase the quality and diversity of the generated images compared to the existing method. A possible application of road image generation for data augmentation is also demonstrated in the experimental results.

scholarly journals PandoraGAN: Generating antiviral peptides using Generative Adversarial Network

2021 ◽  
Author(s):  
Shraddha Surana ◽  
Pooja Arora ◽  
Divye Singh ◽  
Deepti Sahasrabuddhe ◽  
Jayaraman Valadi
Keyword(s):  
Training Data ◽  
Laboratory Research ◽  
Generative Adversarial Networks ◽  
Limited Information ◽  
Data Set ◽  
Continuous Increase ◽  
Link Type ◽  
Highly Active ◽  
Antiviral Peptides ◽  
Antiviral Peptide

AbstractMotivationThe continuous increase in pathogenic viruses and the intensive laboratory research for development of novel antiviral therapies often poses challenge in terms of cost and time efficient drug design. This accelerates research for alternate drug candidates and contributes to recent rise in research of antiviral peptides against many of the viruses. With limited information regarding these peptides and their activity, modifying the existing peptide backbone or developing a novel peptide is very time consuming and a tedious process. Advanced deep learning approaches such as generative adversarial networks (GAN) can be helpful for wet lab scientist to screen potential antiviral candidates of interest and expedite the initial stage of peptide drug development. To our knowledge this is the first ever use of GAN models for antiviral peptides across the viral spectrum.ResultsIn this study, we develop PandoraGAN that utilizes GAN to design bio active antiviral peptides. Available antiviral peptide data was manually curated for preparing highly active peptides data set to include peptides with lower IC50 values. We further validated the generated sequences comparing the physico-chemical properties of generated antiviral peptides with manually curated highly active training data.AvailabilityAntiviral sequences generated by PandoraGAN are available on PandoraGAN server. https://pandora-gan.herokuapp.com/. The code is available at https://gitlab.com/shraddha.surana/antiviral-peptide-predictions-using-ganContactparora@thoughtworks.com

scholarly journals Anomaly detection in Hyper Suprime-Cam galaxy images with generative adversarial networks

2021 ◽  
Author(s):  
Kate Storey-Fisher ◽  
Marc Huertas-Company ◽  
Nesar Ramachandra ◽  
Francois Lanusse ◽  
Alexie Leauthaud ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Dwarf Galaxy ◽  
Generative Adversarial Networks ◽  
Data Sets ◽  
Galaxy Mergers ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Star Forming ◽  
Adversarial Network ◽  
Astronomical Surveys

Abstract The problem of anomaly detection in astronomical surveys is becoming increasingly important as data sets grow in size. We present the results of an unsupervised anomaly detection method using a Wasserstein generative adversarial network (WGAN) on nearly one million optical galaxy images in the Hyper Suprime-Cam (HSC) survey. The WGAN learns to generate realistic HSC-like galaxies that follow the distribution of the data set; anomalous images are defined based on a poor reconstruction by the generator and outlying features learned by the discriminator. We find that the discriminator is more attuned to potentially interesting anomalies compared to the generator, and compared to a simpler autoencoder-based anomaly detection approach, so we use the discriminator-selected images to construct a high-anomaly sample of ∼13 000 objects. We propose a new approach to further characterize these anomalous images: we use a convolutional autoencoder to reduce the dimensionality of the residual differences between the real and WGAN-reconstructed images and perform UMAP clustering on these. We report detected anomalies of interest including galaxy mergers, tidal features, and extreme star-forming galaxies. A follow-up spectroscopic analysis of one of these anomalies is detailed in the Appendix; we find that it is an unusual system most likely to be a metal-poor dwarf galaxy with an extremely blue, higher-metallicity H ii region. We have released a catalog with the WGAN anomaly scores; the code and catalog are available at https://github.com/kstoreyf/anomalies-GAN-HSC, and our interactive visualization tool for exploring the clustered data is at https://weirdgalaxi.es.

Sign in / Sign up

Export Citation Format

Share Document