scholarly journals Deep image synthesis from intuitive user input: A review and perspectives

2021 ◽  
Vol 8 (1) ◽  
pp. 3-31
Author(s):  
Yuan Xue ◽  
Yuan-Chen Guo ◽  
Han Zhang ◽  
Tao Xu ◽  
Song-Hai Zhang ◽  
...  
Keyword(s):  
Image Synthesis ◽  
Generative Models ◽  
Generative Adversarial Networks ◽  
Image Generation ◽  
Art And Design ◽  
User Input ◽  
Adversarial Networks ◽  
Benchmark Datasets ◽  
Deep Image ◽  
Realistic Images

AbstractIn many applications of computer graphics, art, and design, it is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph, or layout, and have a computer system automatically generate photo-realistic images according to that input. While classically, works that allow such automatic image content generation have followed a framework of image retrieval and composition, recent advances in deep generative models such as generative adversarial networks (GANs), variational autoencoders (VAEs), and flow-based methods have enabled more powerful and versatile image generation approaches. This paper reviews recent works for image synthesis given intuitive user input, covering advances in input versatility, image generation methodology, benchmark datasets, and evaluation metrics. This motivates new perspectives on input representation and interactivity, cross fertilization between major image generation paradigms, and evaluation and comparison of generation methods.

scholarly journals Successive Image Generation from a Single Sentence

2021 ◽  
Vol 40 ◽  
pp. 03017
Author(s):  
Amogh Parab ◽  
Ananya Malik ◽  
Arish Damania ◽  
Arnav Parekhji ◽  
Pranit Bari
Keyword(s):  
State Of The Art ◽  
Image Synthesis ◽  
Image Sequence ◽  
Generative Adversarial Networks ◽  
Image Generation ◽  
Single Sentence ◽  
Adversarial Networks ◽  
Successive Image ◽  
Diagrammatic Representations ◽  
Transfer Of Information

Through various examples in history such as the early man’s carving on caves, dependence on diagrammatic representations, the immense popularity of comic books we have seen that vision has a higher reach in communication than written words. In this paper, we analyse and propose a new task of transfer of information from text to image synthesis. Through this paper we aim to generate a story from a single sentence and convert our generated story into a sequence of images. We plan to use state of the art technology to implement this task. With the advent of Generative Adversarial Networks text to image synthesis have found a new awakening. We plan to take this task a step further, in order to automate the entire process. Our system generates a multi-lined story given a single sentence using a deep neural network. This story is then fed into our networks of multiple stage GANs inorder to produce a photorealistic image sequence.

scholarly journals Distilling Portable Generative Adversarial Networks for Image Translation

2020 ◽  
Vol 34 (04) ◽  
pp. 3585-3592 ◽  
Author(s):  
Hanting Chen ◽  
Yunhe Wang ◽  
Han Shu ◽  
Changyuan Wen ◽  
Chunjing Xu ◽  
...  
Keyword(s):  
Generative Models ◽  
Generative Adversarial Networks ◽  
Adversarial Networks ◽  
Image Translation ◽  
Level Information ◽  
Benchmark Datasets ◽  
Knowledge Distillation ◽  
Network Compression ◽  
High Level ◽  
And Storage

Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy computation and storage cost. Traditional network compression methods focus on visually recognition tasks, but never deal with generation tasks. Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. To promote the capability of student generator, we include a student discriminator to measure the distances between real images, and images generated by student and teacher generators. An adversarial learning process is therefore established to optimize student generator and student discriminator. Qualitative and quantitative analysis by conducting experiments on benchmark datasets demonstrate that the proposed method can learn portable generative models with strong performance.

scholarly journals Hierarchical Modes Exploring in Generative Adversarial Networks

2020 ◽  
Vol 34 (07) ◽  
pp. 10981-10988
Author(s):  
Mengxiao Hu ◽  
Jinlong Li ◽  
Maolin Hu ◽  
Tao Hu
Keyword(s):  
Image Synthesis ◽  
Generative Adversarial Networks ◽  
Image Generation ◽  
Specific Level ◽  
Regularization Term ◽  
Adversarial Networks ◽  
Image Translation ◽  
Real Change ◽  
Quantitative Results ◽  
Conditional Information

In conditional Generative Adversarial Networks (cGANs), when two different initial noises are concatenated with the same conditional information, the distance between their outputs is relatively smaller, which makes minor modes likely to collapse into large modes. To prevent this happen, we proposed a hierarchical mode exploring method to alleviate mode collapse in cGANs by introducing a diversity measurement into the objective function as the regularization term. We also introduced the Expected Ratios of Expansion (ERE) into the regularization term, by minimizing the sum of differences between the real change of distance and ERE, we can control the diversity of generated images w.r.t specific-level features. We validated the proposed algorithm on four conditional image synthesis tasks including categorical generation, paired and un-paired image translation and text-to-image generation. Both qualitative and quantitative results show that the proposed method is effective in alleviating the mode collapse problem in cGANs, and can control the diversity of output images w.r.t specific-level features.

scholarly journals Perceptual Pyramid Adversarial Networks for Text-to-Image Synthesis

2019 ◽  
Vol 33 ◽  
pp. 8312-8319 ◽  
Author(s):  
Lianli Gao ◽  
Daiyuan Chen ◽  
Jingkuan Song ◽  
Xing Xu ◽  
Dongxiang Zhang ◽  
...  
Keyword(s):  
Semantic Similarity ◽  
Ground Truth ◽  
Image Synthesis ◽  
Multi Scale ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Benchmark Datasets ◽  
Realistic Images ◽  
Hierarchical Representations ◽  
Forward Process

Generating photo-realistic images conditioned on semantic text descriptions is a challenging task in computer vision field. Due to the nature of hierarchical representations learned in CNN, it is intuitive to utilize richer convolutional features to improve text-to-image synthesis. In this paper, we propose Perceptual Pyramid Adversarial Network (PPAN) to directly synthesize multi-scale images conditioned on texts in an adversarial way. Specifically, we design one pyramid generator and three independent discriminators to synthesize and regularize multi-scale photo-realistic images in one feed-forward process. At each pyramid level, our method takes coarse-resolution features as input, synthesizes highresolution images, and uses convolutions for up-sampling to finer level. Furthermore, the generator adopts the perceptual loss to enforce semantic similarity between the synthesized image and the ground truth, while a multi-purpose discriminator encourages semantic consistency, image fidelity and class invariance. Experimental results show that our PPAN sets new records for text-to-image synthesis on two benchmark datasets: CUB (i.e., 4.38 Inception Score and .290 Visual-semantic Similarity) and Oxford-102 (i.e., 3.52 Inception Score and .297 Visual-semantic Similarity).

scholarly journals Image Synthesis Based On Feature Description

2021 ◽  
Vol 9 (8) ◽  
pp. 2492-2494
Author(s):  
Rohan Bolusani
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Language Processing ◽  
Image Synthesis ◽  
Generative Models ◽  
Generative Adversarial Networks ◽  
Feature Representations ◽  
Adversarial Networks ◽  
Text Feature ◽  
Image Modelling

Abstract: Generating realistic images from text is innovative and interesting, but modern-day machine learning models are still far from this goal. With research and development in the field of natural language processing, neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, in the field of machine learning, generative adversarial networks (GANs) have begun to generate extremely accurate images of especially in categories, such as faces, album covers, and room interiors. In this work, the main goal is to develop a neural network to bridge these advances in text and image modelling, by essentially translating characters to pixels the project will demonstrate the capability of generative models by taking detailed text descriptions and generate plausible images. Keywords: Deep Learning, Computer Vision, NLP, Generative Adversarial Networks

scholarly journals Generating Video from Images using GANs

2020 ◽  
Vol 9 (10) ◽  
pp. 377-380
Keyword(s):  
Random Noise ◽  
Generative Models ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
External Information ◽  
Data Set ◽  
Adversarial Networks ◽  
Wide Range ◽  
Realistic Images ◽  
New Framework

Generative adversarial networks are a category of neural networks used extensively for the generation of a wide range of content. The generative models are trained through an adversarial process that offers a lot of potential in the world of deep learning. GANs are a popular approach to generate new data from random noise vector that are similar or have the same distribution as that in the training data set. The Generative Adversarial Networks (GANs) approach has been proposed to generate more realistic images. An extension of GANs is the conditional GANs which allows the model to condition external information. Conditional GANs have seen increasing uses and more implications than ever. We also propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models, a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. Our work aims at highlighting the uses of conditional GANs specifically with Generating images. We present some of the use cases of conditional GANs with images specifically in video generation.

Sign in / Sign up

Export Citation Format

Share Document