PaDGAN: Learning to Generate High-Quality Novel Designs

2020 ◽  
Vol 143 (3) ◽  
Author(s):  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Loss Function ◽  
Design Space Exploration ◽  
Design Space ◽  
Space Exploration ◽  
Generative Models ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Design Synthesis ◽  
Adversarial Network

Abstract Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face three challenges: (1) generated designs lack diversity and do not cover all areas of the design space, (2) it is difficult to explicitly improve the overall performance or quality of generated designs, and (3) existing models generally do not generate novel designs, outside the domain of the training data. In this article, we simultaneously address these challenges by proposing a new determinantal point process-based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the generative adversarial network, named “performance augmented diverse generative adversarial network” (PaDGAN), which can generate novel high-quality designs with good coverage of the design space. By using three synthetic examples and one real-world airfoil design example, we demonstrate that PaDGAN can generate diverse and high-quality designs. In comparison to a vanilla generative adversarial network, on average, it generates samples with a 28% higher mean quality score with larger diversity and without the mode collapse issue. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary outside the training data towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

scholarly journals PaDGAN: A Generative Adversarial Network for Performance Augmented Diverse Designs

2020 ◽  
Author(s):  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Loss Function ◽  
Design Space Exploration ◽  
Point Processes ◽  
Design Space ◽  
Space Exploration ◽  
Generative Models ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network

Abstract Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face three challenges: 1) generated designs lack diversity and do not cover all areas of the design space, 2) it is difficult to explicitly improve the overall performance or quality of generated designs, and 3) existing models generate do not generate novel designs, outside the domain of the training data. In this paper, we simultaneously address these challenges by proposing a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the Generative Adversarial Network, named “Performance Augmented Diverse Generative Adversarial Network” or PaDGAN, which can generate novel high-quality designs with good coverage of the design space. Using three synthetic examples and one real-world airfoil design example, we demonstrate that PaDGAN can generate diverse and high-quality designs. In comparison to a vanilla Generative Adversarial Network, on average, it generates samples with 28% higher mean quality score with larger diversity and without the mode collapse issue. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary outside the training data towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

scholarly journals WHITE-BOX CARTOONIZATION USING AN EXTENDED GAN FRAMEWORK

2021 ◽  
Vol 5 (12) ◽  
Author(s):  
Amey Thakur ◽  
Hasan Rizvi ◽  
Mega Satish
Keyword(s):  
Smooth Surface ◽  
Generative Models ◽  
Surface Representation ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Structure Representation ◽  
Texture Representation ◽  
Adversarial Network ◽  
Representation Structure ◽  
New Framework

In the present study, we propose to implement a new framework for estimating generative models via an adversarial process to extend an existing GAN framework and develop a white-box controllable image cartoonization, which can generate high-quality cartooned images/videos from real-world photos and videos. The learning purposes of our system are based on three distinct representations: surface representation, structure representation, and texture representation. The surface representation refers to the smooth surface of the images. The structure representation relates to the sparse colour blocks and compresses generic content. The texture representation shows the texture, curves, and features in cartoon images. Generative Adversarial Network (GAN) framework decomposes the images into different representations and learns from them to generate cartoon images. This decomposition makes the framework more controllable and flexible which allows users to make changes based on the required output. This approach overcomes any previous system in terms of maintaining clarity, colours, textures, shapes of images yet showing the characteristics of cartoon images.

scholarly journals Creativity in Generative Musical Networks: Evidence From Two Case Studies

2021 ◽  
Vol 8 ◽  
Author(s):  
Rodrigo F. Cádiz ◽  
Agustín Macaya ◽  
Manuel Cartagena ◽  
Denis Parra
Keyword(s):  
Case Studies ◽  
Generative Models ◽  
Training Data ◽  
Current Understanding ◽  
Generative Adversarial Network ◽  
Training Time ◽  
Adversarial Network ◽  
The Arts ◽  
Musical Excerpts ◽  
Musical Chords

Deep learning, one of the fastest-growing branches of artificial intelligence, has become one of the most relevant research and development areas of the last years, especially since 2012, when a neural network surpassed the most advanced image classification techniques of the time. This spectacular development has not been alien to the world of the arts, as recent advances in generative networks have made possible the artificial creation of high-quality content such as images, movies or music. We believe that these novel generative models propose a great challenge to our current understanding of computational creativity. If a robot can now create music that an expert cannot distinguish from music composed by a human, or create novel musical entities that were not known at training time, or exhibit conceptual leaps, does it mean that the machine is then creative? We believe that the emergence of these generative models clearly signals that much more research needs to be done in this area. We would like to contribute to this debate with two case studies of our own: TimbreNet, a variational auto-encoder network trained to generate audio-based musical chords, and StyleGAN Pianorolls, a generative adversarial network capable of creating short musical excerpts, despite the fact that it was trained with images and not musical data. We discuss and assess these generative models in terms of their creativity and we show that they are in practice capable of learning musical concepts that are not obvious based on the training data, and we hypothesize that these deep models, based on our current understanding of creativity in robots and machines, can be considered, in fact, creative.

A Method for Visualizing the Relations Between Grammar Rules, Performance Objectives and Search Space Exploration in Grammar-Based Computational Design Synthesis

2015 ◽  
Author(s):  
Corinna Königseder ◽  
Kristina Shea
Keyword(s):  
Design Space Exploration ◽  
Design Space ◽  
Space Exploration ◽  
Computational Design ◽  
Search Space ◽  
Synthesis Process ◽  
Design Synthesis ◽  
Novel Approach ◽  
Grammar Rules ◽  
Computational Design Synthesis

Design grammars have been successfully applied in numerous engineering disciplines, e.g. in electrical engineering, architecture and mechanical engineering. A successful application of design grammars in Computational Design Synthesis (CDS) requires a) a meaningful representation of designs and the design task at hand, b) a careful formulation of grammar rules to synthesize new designs, c) problem specific design evaluations, and d) the selection of an appropriate algorithm to guide the synthesis process. Managing these different aspects of CDS requires not only a detailed understanding of each individual part, but also of the interdependencies between them. In this paper, a new method is presented to analyze the exploration of design spaces in CDS. The method analyzes the designs generated during the synthesis process and visualizes how the design space is explored with respect to a) design characteristics, and b) objectives. The selected algorithm as well as the grammar rules can be analyzed with this approach to support the human designer in successfully understanding and applying a CDS method. The case study demonstrates how the method is used to analyze the synthesis of bicycle frames. Two algorithms are compared for this task. Results demonstrate how the method increases the understanding of the different components in CDS. The presented research can be useful for both novices to CDS to help them gain a deeper understanding of the interplay between grammar rules and guidance of the synthesis process, as well as for experts aiming to further improve their CDS application by improving parameter settings of their search algorithms, or by further refining their design grammar. Additionally, the presented method constitutes a novel approach to interactively visualize design space exploration considering not only designs objectives, but also the characteristics and interdependencies of different designs.

scholarly journals Seg2pix: Few Shot Training Line Art Colorization with Segmented Image Data

2021 ◽  
Vol 11 (4) ◽  
pp. 1464
Author(s):  
Chang Wook Seo ◽  
Yongduek Seo
Keyword(s):  
Image Data ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Segmented Image ◽  
Image Translation ◽  
Segmentation Image

There are various challenging issues in automating line art colorization. In this paper, we propose a GAN approach incorporating semantic segmentation image data. Our GAN-based method, named Seg2pix, can automatically generate high quality colorized images, aiming at computerizing one of the most tedious and repetitive jobs performed by coloring workers in the webtoon industry. The network structure of Seg2pix is mostly a modification of the architecture of Pix2pix, which is a convolution-based generative adversarial network for image-to-image translation. Through this method, we can generate high quality colorized images of a particular character with only a few training data. Seg2pix is designed to reproduce a segmented image, which becomes the suggestion data for line art colorization. The segmented image is automatically generated through a generative network with a line art image and a segmentation ground truth. In the next step, this generative network creates a colorized image from the line art and segmented image, which is generated from the former step of the generative network. To summarize, only one line art image is required for testing the generative model, and an original colorized image and segmented image are additionally required as the ground truth for training the model. These generations of the segmented image and colorized image proceed by an end-to-end method sharing the same loss functions. By using this method, we produce better qualitative results for automatic colorization of a particular character’s line art. This improvement can also be measured by quantitative results with Learned Perceptual Image Patch Similarity (LPIPS) comparison. We believe this may help artists exercise their creative expertise mainly in the area where computerization is not yet capable.

scholarly journals Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

2021 ◽  
Vol 14 (1) ◽  
pp. 144
Author(s):  
Luiz E. Christovam ◽  
Milton H. Shimabukuro ◽  
Maria de Lourdes B. T. Galo ◽  
Eija Honkavaara
Keyword(s):  
Time Series ◽  
Loss Function ◽  
Visual Analysis ◽  
Spectral Response ◽  
Semantic Segmentation ◽  
Generative Models ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Crop Type

Clouds are one of the major limitations to crop monitoring using optical satellite images. Despite all efforts to provide decision-makers with high-quality agricultural statistics, there is still a lack of techniques to optimally process satellite image time series in the presence of clouds. In this regard, in this article it was proposed to add a Multi-Layer Perceptron loss function to the pix2pix conditional Generative Adversarial Network (cGAN) objective function. The aim was to enforce the generative model to learn how to deliver synthetic pixels whose values were proxies for the spectral response improving further crop type mapping. Furthermore, it was evaluated the generalization capacity of the generative models in producing pixels with plausible values for images not used in the training. To assess the performance of the proposed approach it was compared real images with synthetic images generated with the proposed approach as well as with the original pix2pix cGAN. The comparative analysis was performed through visual analysis, pixel values analysis, semantic segmentation and similarity metrics. In general, the proposed approach provided slightly better synthetic pixels than the original pix2pix cGAN, removing more noise than the original pix2pix algorithm as well as providing better crop type semantic segmentation; the semantic segmentation of the synthetic image generated with the proposed approach achieved an F1-score of 44.2%, while the real image achieved 44.7%. Regarding the generalization, the models trained utilizing different regions of the same image provided better pixels than models trained using other images in the time series. Besides this, the experiments also showed that the models trained using a pair of images selected every three months along the time series also provided acceptable results on images that do not have cloud-free areas.

scholarly journals Disentangling Information in Artificial Images of Plant Seedlings Using Semi-Supervised GAN

2019 ◽  
Vol 11 (22) ◽  
pp. 2671
Author(s):  
Simon Leminen Madsen ◽  
Anders Krogh Mortensen ◽  
Rasmus Nyholm Jørgensen ◽  
Henrik Karstoft
Keyword(s):  
Latent Variables ◽  
Visual Recognition ◽  
Assessment Tool ◽  
Real Data ◽  
Generative Models ◽  
Training Data ◽  
Classification Model ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Plant Seedlings

Lack of annotated data for training of deep learning systems is a challenge for many visual recognition tasks. This is especially true for domain-specific applications, such as plant detection and recognition, where the annotation process can be both time-consuming and error-prone. Generative models can be used to alleviate this issue by producing artificial data that mimic properties of real data. This work presents a semi-supervised generative adversarial network (GAN) model to produce artificial samples of plant seedlings. By applying the semi-supervised approach, we are able to produce visually distinct samples for nine unique plant species using a single GAN model, while still maintaining a relatively high visual variance in the produced samples for each species. Additionally, we are able to control the appearance of the generated samples with respect to rotation and size through a set of latent variables, despite these not being annotated features in the training data. The generated samples resemble the intended species with an average recognition accuracy of ∼64.3%, evaluated using an external state-of-the-art plant seedling classification model. Additionally, we explore the potential of using the GAN model’s discriminator as a quality assessment tool to remove poor representations of plant seedlings from the artificial samples.

scholarly journals Conditional GAN with Discriminative Filter Generation for Text-to-Video Synthesis

2019 ◽  
Author(s):  
Yogesh Balaji ◽  
Martin Renqiang Min ◽  
Bing Bai ◽  
Rama Chellappa ◽  
Hans Peter Graf
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Generative Models ◽  
Synthetic Dataset ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Multi Scale ◽  
Adversarial Network

Developing conditional generative models for text-to-video synthesis is an extremely challenging yet an important topic of research in machine learning. In this work, we address this problem by introducing Text-Filter conditioning Generative Adversarial Network (TFGAN), a conditional GAN model with a novel multi-scale text-conditioning scheme that improves text-video associations. By combining the proposed conditioning scheme with a deep GAN architecture, TFGAN generates high quality videos from text on challenging real-world video datasets. In addition, we construct a synthetic dataset of text-conditioned moving shapes to systematically evaluate our conditioning scheme. Extensive experiments demonstrate that TFGAN significantly outperforms existing approaches, and can also generate videos of novel categories not seen during training.

Sign in / Sign up

Export Citation Format

Share Document