Increasingly Specialized Generative Adversarial Network for fine-grained visual categorization

2021 ◽  
pp. 107480
Author(s):  
Zhongqi Lin ◽  
Wanlin Gao ◽  
Feng Huang ◽  
Jingdun Jia
Keyword(s):  
Generative Adversarial Network ◽  
Visual Categorization ◽  
Fine Grained ◽  
Adversarial Network

scholarly journals Generating Natural Counterfactual Visual Explanations

2020 ◽  
Author(s):  
Wenqi Zhao ◽  
Satoshi Oyama ◽  
Masahito Kurihara
Keyword(s):  
Image Classification ◽  
Classification Accuracy ◽  
Classification Task ◽  
Learning Models ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Class A ◽  
Adversarial Network ◽  
Class B ◽  
Machine Learning Models

Counterfactual explanations help users to understand the behaviors of machine learning models by changing the inputs for the existing outputs. For an image classification task, an example counterfactual visual explanation explains: "for an example that belongs to class A, what changes do we need to make to the input so that the output is more inclined to class B." Our research considers changing the attribute description text of class A on the basis of the attributes of class B and generating counterfactual images on the basis of the modified text. We can use the prediction results of the model on counterfactual images to find the attributes that have the greatest effect when the model is predicting classes A and B. We applied our method to a fine-grained image classification dataset and used the generative adversarial network to generate natural counterfactual visual explanations. To evaluate these explanations, we used them to assist crowdsourcing workers in an image classification task. We found that, within a specific range, they improved classification accuracy.

scholarly journals Text to Realistic Image Generation with Attentional Concatenation Generative Adversarial Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Linyan Li ◽  
Yu Sun ◽  
Fuyuan Hu ◽  
Tao Zhou ◽  
Xuefeng Xi ◽  
...  
Keyword(s):  
High Resolution ◽  
Image Synthesis ◽  
Semantic Space ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Cascade Structure ◽  
High Resolution Images

In this paper, we propose an Attentional Concatenation Generative Adversarial Network (ACGAN) aiming at generating 1024 × 1024 high-resolution images. First, we propose a multilevel cascade structure, for text-to-image synthesis. During training progress, we gradually add new layers and, at the same time, use the results and word vectors from the previous layer as inputs to the next layer to generate high-resolution images with photo-realistic details. Second, the deep attentional multimodal similarity model is introduced into the network, and we match word vectors with images in a common semantic space to compute a fine-grained matching loss for training the generator. In this way, we can pay attention to the fine-grained information of the word level in the semantics. Finally, the measure of diversity is added to the discriminator, which enables the generator to obtain more diverse gradient directions and improve the diversity of generated samples. The experimental results show that the inception scores of the proposed model on the CUB and Oxford-102 datasets have reached 4.48 and 4.16, improved by 2.75% and 6.42% compared to Attentional Generative Adversarial Networks (AttenGAN). The ACGAN model has a better effect on text-generated images, and the resulting image is closer to the real image.

Fine-grained Image Inpainting with Scale-Enhanced Generative Adversarial Network

2021 ◽  
Author(s):  
Weirong Liu ◽  
ChengruiJie CaoLiu ◽  
Chenwen Ren ◽  
Yulin Wei ◽  
Honglin Guo
Keyword(s):  
Image Inpainting ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Adversarial Network

scholarly journals SSAH: Semi-Supervised Adversarial Deep Hashing with Self-Paced Hard Sample Generation

2020 ◽  
Vol 34 (07) ◽  
pp. 11157-11164
Author(s):  
Sheng Jin ◽  
Shangchen Zhou ◽  
Yao Liu ◽  
Chao Chen ◽  
Xiaoshuai Sun ◽  
...  
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
Unified Framework ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Multi Scale ◽  
Deep Hashing ◽  
Adversarial Network ◽  
Improve State

Deep hashing methods have been proved to be effective and efficient for large-scale Web media search. The success of these data-driven methods largely depends on collecting sufficient labeled data, which is usually a crucial limitation in practical cases. The current solutions to this issue utilize Generative Adversarial Network (GAN) to augment data in semi-supervised learning. However, existing GAN-based methods treat image generations and hashing learning as two isolated processes, leading to generation ineffectiveness. Besides, most works fail to exploit the semantic information in unlabeled data. In this paper, we propose a novel Semi-supervised Self-pace Adversarial Hashing method, named SSAH to solve the above problems in a unified framework. The SSAH method consists of an adversarial network (A-Net) and a hashing network (H-Net). To improve the quality of generative images, first, the A-Net learns hard samples with multi-scale occlusions and multi-angle rotated deformations which compete against the learning of accurate hashing codes. Second, we design a novel self-paced hard generation policy to gradually increase the hashing difficulty of generated samples. To make use of the semantic information in unlabeled ones, we propose a semi-supervised consistent loss. The experimental results show that our method can significantly improve state-of-the-art models on both the widely-used hashing datasets and fine-grained datasets.

scholarly journals Dynamic Detection and Recognition of Objects Based on Sequential RGB Images

2021 ◽  
Vol 13 (7) ◽  
pp. 176
Author(s):  
Shuai Dong ◽  
Zhihua Yang ◽  
Wensheng Li ◽  
Kun Zou
Keyword(s):  
Level Set ◽  
Mapping Function ◽  
Feature Space ◽  
Retrieval Task ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Adversarial Network ◽  
Dynamic Detection ◽  
Instance Segmentation ◽  
Detection And Recognition

Conveyors are used commonly in industrial production lines and automated sorting systems. Many applications require fast, reliable, and dynamic detection and recognition for the objects on conveyors. Aiming at this goal, we design a framework that involves three subtasks: one-class instance segmentation (OCIS), multiobject tracking (MOT), and zero-shot fine-grained recognition of 3D objects (ZSFGR3D). A new level set map network (LSMNet) and a multiview redundancy-free feature network (MVRFFNet) are proposed for the first and third subtasks, respectively. The level set map (LSM) is used to annotate instances instead of the traditional multichannel binary mask, and each peak of the LSM represents one instance. Based on the LSM, LSMNet can adopt a pix2pix architecture to segment instances. MVRFFNet is a generalized zero-shot learning (GZSL) framework based on the Wasserstein generative adversarial network for 3D object recognition. Multi-view features of an object are combined into a compact registered feature. By treating the registered features as the category attribution in the GZSL setting, MVRFFNet learns a mapping function that maps original retrieve features into a new redundancy-free feature space. To validate the performance of the proposed methods, a segmentation dataset and a fine-grained classification dataset about objects on a conveyor are established. Experimental results on these datasets show that LSMNet can achieve a recalling accuracy close to the light instance segmentation framework You Only Look At CoefficienTs (YOLACT), while its computing speed on an NVIDIA GTX1660TI GPU is 80 fps, which is much faster than YOLACT‘s 25 fps. Redundancy-free features generated by MVRFFNet perform much better than original features in the retrieval task.

Fine-Grained Semantic Image Synthesis with Object-Attention Generative Adversarial Network

2021 ◽  
Vol 12 (5) ◽  
pp. 1-18
Author(s):  
Min Wang ◽  
Congyan Lang ◽  
Liqian Liang ◽  
Songhe Feng ◽  
Tao Wang ◽  
...  
Keyword(s):  
Feature Fusion ◽  
Image Synthesis ◽  
Image Features ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Multi Scale ◽  
Adversarial Network ◽  
Object Attention ◽  
Object Features

Semantic image synthesis is a new rising and challenging vision problem accompanied by the recent promising advances in generative adversarial networks. The existing semantic image synthesis methods only consider the global information provided by the semantic segmentation mask, such as class label, global layout, and location, so the generative models cannot capture the rich local fine-grained information of the images (e.g., object structure, contour, and texture). To address this issue, we adopt a multi-scale feature fusion algorithm to refine the generated images by learning the fine-grained information of the local objects. We propose OA-GAN, a novel object-attention generative adversarial network that allows attention-driven, multi-fusion refinement for fine-grained semantic image synthesis. Specifically, the proposed model first generates multi-scale global image features and local object features, respectively, then the local object features are fused into the global image features to improve the correlation between the local and the global. In the process of feature fusion, the global image features and the local object features are fused through the channel-spatial-wise fusion block to learn ‘what’ and ‘where’ to attend in the channel and spatial axes, respectively. The fused features are used to construct correlation filters to obtain feature response maps to determine the locations, contours, and textures of the objects. Extensive quantitative and qualitative experiments on COCO-Stuff, ADE20K and Cityscapes datasets demonstrate that our OA-GAN significantly outperforms the state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document