A Novel Image Captioning Method Based on Generative Adversarial Networks

2019 ◽  
pp. 281-292
Author(s):  
Yang Fan ◽  
Jungang Xu ◽  
Yingfei Sun ◽  
Yiyu Wang
Keyword(s):  
Generative Adversarial Networks ◽  
Image Captioning ◽  
Adversarial Networks

TextCycleGAN: cyclical-generative adversarial networks for image captioning

2021 ◽  
Author(s):  
Mohammad Alam ◽  
Nicole Isoda ◽  
Mitch Manzanares ◽  
Anthony Delgado ◽  
Antonius F. Panggabean
Keyword(s):  
Generative Adversarial Networks ◽  
Image Captioning ◽  
Adversarial Networks

scholarly journals Recurrent Relational Memory Network for Unsupervised Image Captioning

2020 ◽  
Author(s):  
Dan Guo ◽  
Yang Wang ◽  
Peipei Song ◽  
Meng Wang
Keyword(s):  
Generative Adversarial Networks ◽  
Relational Reasoning ◽  
Relational Memory ◽  
Image Captioning ◽  
Adversarial Networks ◽  
The Arts ◽  
Benchmark Datasets ◽  
Unsupervised Training ◽  
Visual Concepts ◽  
Memory Network

Unsupervised image captioning with no annotations is an emerging challenge in computer vision, where the existing arts usually adopt GAN (Generative Adversarial Networks) models. In this paper, we propose a novel memory-based network rather than GAN, named Recurrent Relational Memory Network (R2M). Unlike complicated and sensitive adversarial learning that non-ideally performs for long sentence generation, R2M implements a concepts-to-sentence memory translator through two-stage memory mechanisms: fusion and recurrent memories, correlating the relational reasoning between common visual concepts and the generated words for long periods. R2M encodes visual context through unsupervised training on images, while enabling the memory to learn from irrelevant textual corpus via supervised fashion. Our solution enjoys less learnable parameters and higher computational efficiency than GAN-based methods, which heavily bear parameter sensitivity. We experimentally validate the superiority of R2M than state-of-the-arts on all benchmark datasets.

scholarly journals Interactive Dual Generative Adversarial Networks for Image Captioning

2020 ◽  
Vol 34 (07) ◽  
pp. 11588-11595 ◽  
Author(s):  
Junhao Liu ◽  
Kai Wang ◽  
Chunpu Xu ◽  
Zhou Zhao ◽  
Ruifeng Xu ◽  
...  
Keyword(s):  
Generative Adversarial Networks ◽  
Image Captioning ◽  
Generative Adversarial Network ◽  
Informative Feedback ◽  
Adversarial Network ◽  
Adversarial Networks

Image captioning is usually built on either generation-based or retrieval-based approaches. Both ways have certain strengths but suffer from their own limitations. In this paper, we propose an Interactive Dual Generative Adversarial Network (IDGAN) for image captioning, which mutually combines the retrieval-based and generation-based methods to learn a better image captioning ensemble. IDGAN consists of two generators and two discriminators, where the generation- and retrieval-based generators mutually benefit from each other's complementary targets that are learned from two dual adversarial discriminators. Specifically, the generation- and retrieval-based generators provide improved synthetic and retrieved candidate captions with informative feedback signals from the two respective discriminators that are trained to distinguish the generated captions from the true captions and assign top rankings to true captions respectively, thus featuring the merits of both retrieval-based and generation-based approaches. Extensive experiments on MSCOCO dataset demonstrate that the proposed IDGAN model significantly outperforms the compared methods for image captioning.

ORGANIC (1).pdf

2017 ◽  
Author(s):  
Benjamin Sanchez-Lengeling ◽  
Carlos Outeiral ◽  
Gabriel L. Guimaraes ◽  
Alan Aspuru-Guzik
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
Chemical Species ◽  
Material Design ◽  
Organic Photovoltaic ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

Sign in / Sign up

Export Citation Format

Share Document