scholarly journals PaDGAN: A Generative Adversarial Network for Performance Augmented Diverse Designs

2020 ◽  
Author(s):  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Loss Function ◽  
Design Space Exploration ◽  
Point Processes ◽  
Design Space ◽  
Space Exploration ◽  
Generative Models ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network

Abstract Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face three challenges: 1) generated designs lack diversity and do not cover all areas of the design space, 2) it is difficult to explicitly improve the overall performance or quality of generated designs, and 3) existing models generate do not generate novel designs, outside the domain of the training data. In this paper, we simultaneously address these challenges by proposing a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the Generative Adversarial Network, named “Performance Augmented Diverse Generative Adversarial Network” or PaDGAN, which can generate novel high-quality designs with good coverage of the design space. Using three synthetic examples and one real-world airfoil design example, we demonstrate that PaDGAN can generate diverse and high-quality designs. In comparison to a vanilla Generative Adversarial Network, on average, it generates samples with 28% higher mean quality score with larger diversity and without the mode collapse issue. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary outside the training data towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

PaDGAN: Learning to Generate High-Quality Novel Designs

2020 ◽  
Vol 143 (3) ◽  
Author(s):  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Loss Function ◽  
Design Space Exploration ◽  
Design Space ◽  
Space Exploration ◽  
Generative Models ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Design Synthesis ◽  
Adversarial Network

Abstract Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face three challenges: (1) generated designs lack diversity and do not cover all areas of the design space, (2) it is difficult to explicitly improve the overall performance or quality of generated designs, and (3) existing models generally do not generate novel designs, outside the domain of the training data. In this article, we simultaneously address these challenges by proposing a new determinantal point process-based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the generative adversarial network, named “performance augmented diverse generative adversarial network” (PaDGAN), which can generate novel high-quality designs with good coverage of the design space. By using three synthetic examples and one real-world airfoil design example, we demonstrate that PaDGAN can generate diverse and high-quality designs. In comparison to a vanilla generative adversarial network, on average, it generates samples with a 28% higher mean quality score with larger diversity and without the mode collapse issue. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary outside the training data towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

scholarly journals WHITE-BOX CARTOONIZATION USING AN EXTENDED GAN FRAMEWORK

2021 ◽  
Vol 5 (12) ◽  
Author(s):  
Amey Thakur ◽  
Hasan Rizvi ◽  
Mega Satish
Keyword(s):  
Smooth Surface ◽  
Generative Models ◽  
Surface Representation ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Structure Representation ◽  
Texture Representation ◽  
Adversarial Network ◽  
Representation Structure ◽  
New Framework

In the present study, we propose to implement a new framework for estimating generative models via an adversarial process to extend an existing GAN framework and develop a white-box controllable image cartoonization, which can generate high-quality cartooned images/videos from real-world photos and videos. The learning purposes of our system are based on three distinct representations: surface representation, structure representation, and texture representation. The surface representation refers to the smooth surface of the images. The structure representation relates to the sparse colour blocks and compresses generic content. The texture representation shows the texture, curves, and features in cartoon images. Generative Adversarial Network (GAN) framework decomposes the images into different representations and learns from them to generate cartoon images. This decomposition makes the framework more controllable and flexible which allows users to make changes based on the required output. This approach overcomes any previous system in terms of maintaining clarity, colours, textures, shapes of images yet showing the characteristics of cartoon images.

scholarly journals Creativity in Generative Musical Networks: Evidence From Two Case Studies

2021 ◽  
Vol 8 ◽  
Author(s):  
Rodrigo F. Cádiz ◽  
Agustín Macaya ◽  
Manuel Cartagena ◽  
Denis Parra
Keyword(s):  
Case Studies ◽  
Generative Models ◽  
Training Data ◽  
Current Understanding ◽  
Generative Adversarial Network ◽  
Training Time ◽  
Adversarial Network ◽  
The Arts ◽  
Musical Excerpts ◽  
Musical Chords

Deep learning, one of the fastest-growing branches of artificial intelligence, has become one of the most relevant research and development areas of the last years, especially since 2012, when a neural network surpassed the most advanced image classification techniques of the time. This spectacular development has not been alien to the world of the arts, as recent advances in generative networks have made possible the artificial creation of high-quality content such as images, movies or music. We believe that these novel generative models propose a great challenge to our current understanding of computational creativity. If a robot can now create music that an expert cannot distinguish from music composed by a human, or create novel musical entities that were not known at training time, or exhibit conceptual leaps, does it mean that the machine is then creative? We believe that the emergence of these generative models clearly signals that much more research needs to be done in this area. We would like to contribute to this debate with two case studies of our own: TimbreNet, a variational auto-encoder network trained to generate audio-based musical chords, and StyleGAN Pianorolls, a generative adversarial network capable of creating short musical excerpts, despite the fact that it was trained with images and not musical data. We discuss and assess these generative models in terms of their creativity and we show that they are in practice capable of learning musical concepts that are not obvious based on the training data, and we hypothesize that these deep models, based on our current understanding of creativity in robots and machines, can be considered, in fact, creative.

scholarly journals Seg2pix: Few Shot Training Line Art Colorization with Segmented Image Data

2021 ◽  
Vol 11 (4) ◽  
pp. 1464
Author(s):  
Chang Wook Seo ◽  
Yongduek Seo
Keyword(s):  
Image Data ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Segmented Image ◽  
Image Translation ◽  
Segmentation Image

There are various challenging issues in automating line art colorization. In this paper, we propose a GAN approach incorporating semantic segmentation image data. Our GAN-based method, named Seg2pix, can automatically generate high quality colorized images, aiming at computerizing one of the most tedious and repetitive jobs performed by coloring workers in the webtoon industry. The network structure of Seg2pix is mostly a modification of the architecture of Pix2pix, which is a convolution-based generative adversarial network for image-to-image translation. Through this method, we can generate high quality colorized images of a particular character with only a few training data. Seg2pix is designed to reproduce a segmented image, which becomes the suggestion data for line art colorization. The segmented image is automatically generated through a generative network with a line art image and a segmentation ground truth. In the next step, this generative network creates a colorized image from the line art and segmented image, which is generated from the former step of the generative network. To summarize, only one line art image is required for testing the generative model, and an original colorized image and segmented image are additionally required as the ground truth for training the model. These generations of the segmented image and colorized image proceed by an end-to-end method sharing the same loss functions. By using this method, we produce better qualitative results for automatic colorization of a particular character’s line art. This improvement can also be measured by quantitative results with Learned Perceptual Image Patch Similarity (LPIPS) comparison. We believe this may help artists exercise their creative expertise mainly in the area where computerization is not yet capable.

scholarly journals Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

2021 ◽  
Vol 14 (1) ◽  
pp. 144
Author(s):  
Luiz E. Christovam ◽  
Milton H. Shimabukuro ◽  
Maria de Lourdes B. T. Galo ◽  
Eija Honkavaara
Keyword(s):  
Time Series ◽  
Loss Function ◽  
Visual Analysis ◽  
Spectral Response ◽  
Semantic Segmentation ◽  
Generative Models ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Crop Type

Clouds are one of the major limitations to crop monitoring using optical satellite images. Despite all efforts to provide decision-makers with high-quality agricultural statistics, there is still a lack of techniques to optimally process satellite image time series in the presence of clouds. In this regard, in this article it was proposed to add a Multi-Layer Perceptron loss function to the pix2pix conditional Generative Adversarial Network (cGAN) objective function. The aim was to enforce the generative model to learn how to deliver synthetic pixels whose values were proxies for the spectral response improving further crop type mapping. Furthermore, it was evaluated the generalization capacity of the generative models in producing pixels with plausible values for images not used in the training. To assess the performance of the proposed approach it was compared real images with synthetic images generated with the proposed approach as well as with the original pix2pix cGAN. The comparative analysis was performed through visual analysis, pixel values analysis, semantic segmentation and similarity metrics. In general, the proposed approach provided slightly better synthetic pixels than the original pix2pix cGAN, removing more noise than the original pix2pix algorithm as well as providing better crop type semantic segmentation; the semantic segmentation of the synthetic image generated with the proposed approach achieved an F1-score of 44.2%, while the real image achieved 44.7%. Regarding the generalization, the models trained utilizing different regions of the same image provided better pixels than models trained using other images in the time series. Besides this, the experiments also showed that the models trained using a pair of images selected every three months along the time series also provided acceptable results on images that do not have cloud-free areas.

scholarly journals Disentangling Information in Artificial Images of Plant Seedlings Using Semi-Supervised GAN

2019 ◽  
Vol 11 (22) ◽  
pp. 2671
Author(s):  
Simon Leminen Madsen ◽  
Anders Krogh Mortensen ◽  
Rasmus Nyholm Jørgensen ◽  
Henrik Karstoft
Keyword(s):  
Latent Variables ◽  
Visual Recognition ◽  
Assessment Tool ◽  
Real Data ◽  
Generative Models ◽  
Training Data ◽  
Classification Model ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Plant Seedlings

Lack of annotated data for training of deep learning systems is a challenge for many visual recognition tasks. This is especially true for domain-specific applications, such as plant detection and recognition, where the annotation process can be both time-consuming and error-prone. Generative models can be used to alleviate this issue by producing artificial data that mimic properties of real data. This work presents a semi-supervised generative adversarial network (GAN) model to produce artificial samples of plant seedlings. By applying the semi-supervised approach, we are able to produce visually distinct samples for nine unique plant species using a single GAN model, while still maintaining a relatively high visual variance in the produced samples for each species. Additionally, we are able to control the appearance of the generated samples with respect to rotation and size through a set of latent variables, despite these not being annotated features in the training data. The generated samples resemble the intended species with an average recognition accuracy of ∼64.3%, evaluated using an external state-of-the-art plant seedling classification model. Additionally, we explore the potential of using the GAN model’s discriminator as a quality assessment tool to remove poor representations of plant seedlings from the artificial samples.

scholarly journals Conditional GAN with Discriminative Filter Generation for Text-to-Video Synthesis

2019 ◽  
Author(s):  
Yogesh Balaji ◽  
Martin Renqiang Min ◽  
Bing Bai ◽  
Rama Chellappa ◽  
Hans Peter Graf
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Generative Models ◽  
Synthetic Dataset ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Multi Scale ◽  
Adversarial Network

Developing conditional generative models for text-to-video synthesis is an extremely challenging yet an important topic of research in machine learning. In this work, we address this problem by introducing Text-Filter conditioning Generative Adversarial Network (TFGAN), a conditional GAN model with a novel multi-scale text-conditioning scheme that improves text-video associations. By combining the proposed conditioning scheme with a deep GAN architecture, TFGAN generates high quality videos from text on challenging real-world video datasets. In addition, we construct a synthetic dataset of text-conditioned moving shapes to systematically evaluate our conditioning scheme. Extensive experiments demonstrate that TFGAN significantly outperforms existing approaches, and can also generate videos of novel categories not seen during training.

scholarly journals A Generative Adversarial Network Technique for High-Quality Super-Resolution Reconstruction of Cardiac Magnetic Resonance Images

2021 ◽  
Author(s):  
Ming Zhao ◽  
Yang Wei ◽  
Kelvin Kian Loong Wong
Keyword(s):  
Neural Network ◽  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Loss Function ◽  
Super Resolution ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Feed Forward ◽  
Adversarial Network ◽  
Cardiac Images

Abstract Objective: In this paper, we proposed a Denoising Super-resolution Generative Adversarial Network (DnSRGAN) method for high-quality super-resolution reconstruction of noisy cardiac magnetic resonance (CMR) images.Methods: This method is based on feed-forward denoising convolutional neural network (DnCNN) and SRGAN architecture. Firstly, we used a feed-forward denoising neural network to pre-denoise the CMR image to ensure that the input is a clean image. Secondly, we use the gradient penalty (GP) method to solve the problem of the discriminator gradient disappearing, which improves the convergence speed of the model. Finally, a new loss function is added to the original SRGAN loss function to monitor GAN gradient descent to achieve more stable and efficient model training, thereby providing higher perceptual quality for the super-resolution of CMR images.Results: We divided the tested cardiac images into 3 groups, each group of 25 images, calculated the Peak Signal to Noise Ratio (PSNR) /Structural Similarity (SSIM) between Ground Truth (GT) and the images generated by super-resolution, used them to evaluate our model, and Compared with the current widely used method: Bicubic ESRGAN and SRGAN, our method has better reconstruction quality and higher PSNR/SSIM score.Conclusion: We used DnCNN to denoise the CMR image, and then using the improved SRGAN to perform super-resolution reconstruction of the denoised image, we can solve the problem of high noise and artifacts that cause the cardiac image to be reconstructed incorrectly during super-resolution. Furthermore, our method is capable of high-quality reconstruction of noisy cardiac images.

Sign in / Sign up

Export Citation Format

Share Document