A survey on generative adversarial networks for imbalance problems in computer vision tasks

AbstractAny computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

Download Full-text

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

10.21203/rs.3.rs-45616/v4 ◽

2021 ◽

Author(s):

Vignesh Sampath ◽

Iñaki Maurtua ◽

Juan José Aguilar Martín ◽

Aitor Gutierrez

Keyword(s):

Computer Vision ◽

Real World ◽

Medical Image Analysis ◽

Image Data ◽

Training Dataset ◽

Generative Adversarial Networks ◽

Metallic Surface ◽

Synthetic Image ◽

Application Development ◽

Model Complex

Abstract Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

Download Full-text

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

10.21203/rs.3.rs-45616/v3 ◽

2020 ◽

Author(s):

Vignesh Sampath ◽

Iñaki Maurtua ◽

Juan José Aguilar Martín ◽

Aitor Gutierrez

Keyword(s):

Computer Vision ◽

Real World ◽

Medical Image Analysis ◽

Image Data ◽

Training Dataset ◽

Generative Adversarial Networks ◽

Metallic Surface ◽

Synthetic Image ◽

Adversarial Networks ◽

Model Complex

Abstract Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

Download Full-text

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

10.21203/rs.3.rs-45616/v2 ◽

2020 ◽

Author(s):

Vignesh Sampath ◽

Iñaki Maurtua ◽

Juan José Aguilar Martín ◽

Aitor Gutierrez

Keyword(s):

Computer Vision ◽

Real World ◽

Medical Image Analysis ◽

Image Data ◽

Training Dataset ◽

Generative Adversarial Networks ◽

Metallic Surface ◽

Synthetic Image ◽

Adversarial Networks ◽

Model Complex

Abstract Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets. In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examine key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and further provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

Download Full-text

A Survey on Generative Adversarial Networks for Imbalance Problems in Computer Vision Tasks

10.21203/rs.3.rs-45616/v1 ◽

2020 ◽

Author(s):

Vignesh Sampath ◽

Iñaki Maurtua ◽

Juan José Aguilar Martín ◽

Aitor Gutierrez

Keyword(s):

Computer Vision ◽

Real World ◽

Medical Image Analysis ◽

Image Data ◽

Generative Adversarial Networks ◽

Metallic Surface ◽

Application Development ◽

Adversarial Networks ◽

Surface Defect Detection ◽

Image Models

Abstract Any computer vision application development starts off by acquiring images and data, then preprocessingand pattern recognition steps to perform a task. When the acquired image is highly imbalanced and notadequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems inacquired image datasets in certain complex real-world problems such as anomaly detection, emotionrecognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction,etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when thetraining dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gainedimmense attention by researchers across a variety of application domains due to their capability to modelcomplex real-world image data. It is particularly important that GANs can not only be used to generatesynthetic images, but also its fascinating adversarial learning idea showed good potential in restoringbalance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressingimbalance problems in image data. The real-world challenges and implementations of synthetic imagegeneration based on GANs are extensively covered in this survey. Our survey first introduces variousimbalance problems in computer vision tasks and its existing solutions, and then examine key conceptssuch as deep generative image models and GANs. After that, we propose taxonomy to summarize GANsbased techniques for addressing imbalance problems in computer vision tasks into three major categories:Image level imbalances in classification, object level imbalances in object detection and pixel levelimbalances in segmentation tasks. We elaborate the imbalance problems of each group, and furtherprovide GANs based solutions in each group. Readers will understand how GANs based techniques canhandle the problem of imbalances and boost performance of the computer vision algorithms.

Download Full-text

Improvement of Multiparametric MR Image Segmentation by Augmenting the Data With Generative Adversarial Networks for Glioma Patients

Frontiers in Computational Neuroscience ◽

10.3389/fncom.2020.495075 ◽

2021 ◽

Vol 14 ◽

Author(s):

Eric Nathan Carver ◽

Zhenzhen Dai ◽

Evan Liang ◽

James Snyder ◽

Ning Wen

Keyword(s):

Medical Image ◽

Medical Image Analysis ◽

Model Performance ◽

Image Data ◽

Generative Adversarial Networks ◽

Dice Similarity Coefficient ◽

Generative Adversarial Network ◽

Post Contrast ◽

Adversarial Network ◽

Fluid Attenuated Inversion Recovery

Every year thousands of patients are diagnosed with a glioma, a type of malignant brain tumor. MRI plays an essential role in the diagnosis and treatment assessment of these patients. Neural networks show great potential to aid physicians in the medical image analysis. This study investigated the creation of synthetic brain T1-weighted (T1), post-contrast T1-weighted (T1CE), T2-weighted (T2), and T2 Fluid Attenuated Inversion Recovery (Flair) MR images. These synthetic MR (synMR) images were assessed quantitatively with four metrics. The synMR images were also assessed qualitatively by an authoring physician with notions that synMR possessed realism in its portrayal of structural boundaries but struggled to accurately depict tumor heterogeneity. Additionally, this study investigated the synMR images created by generative adversarial network (GAN) to overcome the lack of annotated medical image data in training U-Nets to segment enhancing tumor, whole tumor, and tumor core regions on gliomas. Multiple two-dimensional (2D) U-Nets were trained with original BraTS data and differing subsets of the synMR images. Dice similarity coefficient (DSC) was used as the loss function during training as well a quantitative metric. Additionally, Hausdorff Distance 95% CI (HD) was used to judge the quality of the contours created by these U-Nets. The model performance was improved in both DSC and HD when incorporating synMR in the training set. In summary, this study showed the ability to generate high quality Flair, T2, T1, and T1CE synMR images using GAN. Using synMR images showed encouraging results to improve the U-Net segmentation performance and shows potential to address the scarcity of annotated medical images.

Download Full-text

Autoencoder-Combined Generative Adversarial Networks for Synthetic Image Data Generation and Detection of Jellyfish Swarm

IEEE Access ◽

10.1109/access.2018.2872025 ◽

2018 ◽

Vol 6 ◽

pp. 54207-54214 ◽

Cited By ~ 7

Author(s):

Kyukwang Kim ◽

Hyun Myung

Keyword(s):

Image Data ◽

Generative Adversarial Networks ◽

Synthetic Image ◽

Data Generation ◽

Adversarial Networks

Download Full-text

Methods and Applications for Segmenting 3D Medical Image Data

User Centered Design for Medical Visualization ◽

10.4018/978-1-59904-777-5.ch015 ◽

2011 ◽

pp. 303-320

Author(s):

Hong Shen

Keyword(s):

Computer Vision ◽

Image Segmentation ◽

General Problem ◽

Real World ◽

Medical Image ◽

Image Data ◽

Medical Image Segmentation ◽

Knowledge Based ◽

Typical Sample ◽

Medical Image Data

In this chapter, we will give an intuitive introduction to the general problem of 3D medical image segmentation. We will give an overview of the popular and relevant methods that may be applicable, with a discussion about their advantages and limits. Specifically, we will discuss the issue of incorporating prior knowledge into the segmentation of anatomic structures and describe in detail the concept and issues of knowledge-based segmentation. Typical sample applications will accompany the discussions throughout this chapter. We hope this will help an application developer to improve insights in the understanding and application of various computer vision approaches to solve real-world problems of medical image segmentation.

Download Full-text

Adaptive Weighted Multi-Discriminator CycleGAN for Underwater Image Enhancement

Journal of Marine Science and Engineering ◽

10.3390/jmse7070200 ◽

2019 ◽

Vol 7 (7) ◽

pp. 200 ◽

Cited By ~ 2

Author(s):

Jaihyun Park ◽

David K. Han ◽

Hanseok Ko

Keyword(s):

Image Enhancement ◽

Real World ◽

Input Image ◽

Generative Adversarial Networks ◽

Synthetic Image ◽

Training Procedure ◽

Weighting Method ◽

Adversarial Networks ◽

Underwater Image ◽

Enhancement Method

In this paper, we propose a novel underwater image enhancement method. Typical deep learning models for underwater image enhancement are trained by paired synthetic dataset. Therefore, these models are mostly effective for synthetic image enhancement but less so for real-world images. In contrast, cycle-consistent generative adversarial networks (CycleGAN) can be trained with unpaired dataset. However, performance of the CycleGAN is highly dependent upon the dataset, thus it may generate unrealistic images with less content information than original images. A novel solution we propose here is by starting with a CycleGAN, we add a pair of discriminators to preserve contents of input image while enhancing the image. As a part of the solution, we introduce an adaptive weighting method for limiting losses of the two types of discriminators to balance their influence and stabilize the training procedure. Extensive experiments demonstrate that the proposed method significantly outperforms the state-of-the-art methods on real-world underwater images.

Download Full-text