scholarly journals WHITE-BOX CARTOONIZATION USING AN EXTENDED GAN FRAMEWORK

2021 ◽  
Vol 5 (12) ◽  
Author(s):  
Amey Thakur ◽  
Hasan Rizvi ◽  
Mega Satish
Keyword(s):  
Smooth Surface ◽  
Generative Models ◽  
Surface Representation ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Structure Representation ◽  
Texture Representation ◽  
Adversarial Network ◽  
Representation Structure ◽  
New Framework

In the present study, we propose to implement a new framework for estimating generative models via an adversarial process to extend an existing GAN framework and develop a white-box controllable image cartoonization, which can generate high-quality cartooned images/videos from real-world photos and videos. The learning purposes of our system are based on three distinct representations: surface representation, structure representation, and texture representation. The surface representation refers to the smooth surface of the images. The structure representation relates to the sparse colour blocks and compresses generic content. The texture representation shows the texture, curves, and features in cartoon images. Generative Adversarial Network (GAN) framework decomposes the images into different representations and learns from them to generate cartoon images. This decomposition makes the framework more controllable and flexible which allows users to make changes based on the required output. This approach overcomes any previous system in terms of maintaining clarity, colours, textures, shapes of images yet showing the characteristics of cartoon images.

PaDGAN: Learning to Generate High-Quality Novel Designs

2020 ◽  
Vol 143 (3) ◽  
Author(s):  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Loss Function ◽  
Design Space Exploration ◽  
Design Space ◽  
Space Exploration ◽  
Generative Models ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Design Synthesis ◽  
Adversarial Network

Abstract Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face three challenges: (1) generated designs lack diversity and do not cover all areas of the design space, (2) it is difficult to explicitly improve the overall performance or quality of generated designs, and (3) existing models generally do not generate novel designs, outside the domain of the training data. In this article, we simultaneously address these challenges by proposing a new determinantal point process-based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the generative adversarial network, named “performance augmented diverse generative adversarial network” (PaDGAN), which can generate novel high-quality designs with good coverage of the design space. By using three synthetic examples and one real-world airfoil design example, we demonstrate that PaDGAN can generate diverse and high-quality designs. In comparison to a vanilla generative adversarial network, on average, it generates samples with a 28% higher mean quality score with larger diversity and without the mode collapse issue. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary outside the training data towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

scholarly journals PaDGAN: A Generative Adversarial Network for Performance Augmented Diverse Designs

2020 ◽  
Author(s):  
Wei Chen ◽  
Faez Ahmed
Keyword(s):  
Loss Function ◽  
Design Space Exploration ◽  
Point Processes ◽  
Design Space ◽  
Space Exploration ◽  
Generative Models ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network

Abstract Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face three challenges: 1) generated designs lack diversity and do not cover all areas of the design space, 2) it is difficult to explicitly improve the overall performance or quality of generated designs, and 3) existing models generate do not generate novel designs, outside the domain of the training data. In this paper, we simultaneously address these challenges by proposing a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the Generative Adversarial Network, named “Performance Augmented Diverse Generative Adversarial Network” or PaDGAN, which can generate novel high-quality designs with good coverage of the design space. Using three synthetic examples and one real-world airfoil design example, we demonstrate that PaDGAN can generate diverse and high-quality designs. In comparison to a vanilla Generative Adversarial Network, on average, it generates samples with 28% higher mean quality score with larger diversity and without the mode collapse issue. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary outside the training data towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

scholarly journals Conditional GAN with Discriminative Filter Generation for Text-to-Video Synthesis

2019 ◽  
Author(s):  
Yogesh Balaji ◽  
Martin Renqiang Min ◽  
Bing Bai ◽  
Rama Chellappa ◽  
Hans Peter Graf
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Generative Models ◽  
Synthetic Dataset ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Multi Scale ◽  
Adversarial Network

Developing conditional generative models for text-to-video synthesis is an extremely challenging yet an important topic of research in machine learning. In this work, we address this problem by introducing Text-Filter conditioning Generative Adversarial Network (TFGAN), a conditional GAN model with a novel multi-scale text-conditioning scheme that improves text-video associations. By combining the proposed conditioning scheme with a deep GAN architecture, TFGAN generates high quality videos from text on challenging real-world video datasets. In addition, we construct a synthetic dataset of text-conditioned moving shapes to systematically evaluate our conditioning scheme. Extensive experiments demonstrate that TFGAN significantly outperforms existing approaches, and can also generate videos of novel categories not seen during training.

scholarly journals An Underwater Image Enhancement Algorithm Based on Generative Adversarial Network and Natural Image Quality Evaluation Index

2021 ◽  
Vol 9 (7) ◽  
pp. 691
Author(s):  
Kai Hu ◽  
Yanwen Zhang ◽  
Chenghang Weng ◽  
Pengsheng Wang ◽  
Zhiliang Deng ◽  
...  
Keyword(s):  
Image Quality ◽  
Image Enhancement ◽  
Quality Evaluation ◽  
High Efficiency ◽  
Natural Image ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Image Quality Evaluation ◽  
Adversarial Network ◽  
Underwater Image

When underwater vehicles work, underwater images are often absorbed by light and scattered and diffused by floating objects, which leads to the degradation of underwater images. The generative adversarial network (GAN) is widely used in underwater image enhancement tasks because it can complete image-style conversions with high efficiency and high quality. Although the GAN converts low-quality underwater images into high-quality underwater images (truth images), the dataset of truth images also affects high-quality underwater images. However, an underwater truth image lacks underwater image enhancement, which leads to a poor effect of the generated image. Thus, this paper proposes to add the natural image quality evaluation (NIQE) index to the GAN to provide generated images with higher contrast and make them more in line with the perception of the human eye, and at the same time, grant generated images a better effect than the truth images set by the existing dataset. In this paper, several groups of experiments are compared, and through the subjective evaluation and objective evaluation indicators, it is verified that the enhanced image of this algorithm is better than the truth image set by the existing dataset.

Sound Field Reconstruction in Rooms with Deep Generative Models

2021 ◽  
Vol 263 (5) ◽  
pp. 1527-1538
Author(s):  
Xenofon Karakonstantis ◽  
Efren Fernandez Grande
Keyword(s):  
Plane Waves ◽  
Sound Field ◽  
Real Data ◽  
Generative Models ◽  
Random Wave ◽  
Generative Adversarial Network ◽  
Underlying Distribution ◽  
Adversarial Network ◽  
Reconstruction Methods ◽  
Free Region

The characterization of Room Impulse Responses (RIR) over an extended region in a room by means of measurements requires dense spatial with many microphones. This can often become intractable and time consuming in practice. Well established reconstruction methods such as plane wave regression show that the sound field in a room can be reconstructed from sparsely distributed measurements. However, these reconstructions usually rely on assuming physical sparsity (i.e. few waves compose the sound field) or trait in the measured sound field, making the models less generalizable and problem specific. In this paper we introduce a method to reconstruct a sound field in an enclosure with the use of a Generative Adversarial Network (GAN), which s new variants of the data distributions that it is trained upon. The goal of the proposed GAN model is to estimate the underlying distribution of plane waves in any source free region, and map these distributions from a stochastic, latent representation. A GAN is trained on a large number of synthesized sound fields represented by a random wave field and then tested on both simulated and real data sets, of lightly damped and reverberant rooms.

scholarly journals Joint Entity and Event Extraction with Generative Adversarial Imitation Learning

2019 ◽  
Vol 1 (2) ◽  
pp. 99-120 ◽  
Author(s):  
Tongtao Zhang ◽  
Heng Ji ◽  
Avirup Sil
Keyword(s):  
State Of The Art ◽  
Ground Truth ◽  
Event Extraction ◽  
Imitation Learning ◽  
Learning Method ◽  
Inverse Reinforcement Learning ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
The Difference ◽  
New Framework

We propose a new framework for entity and event extraction based on generative adversarial imitation learning—an inverse reinforcement learning method using a generative adversarial network (GAN). We assume that instances and labels yield to various extents of difficulty and the gains and penalties (rewards) are expected to be diverse. We utilize discriminators to estimate proper rewards according to the difference between the labels committed by the ground-truth (expert) and the extractor (agent). Our experiments demonstrate that the proposed framework outperforms state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document