scholarly journals Automatic Colorization of Anime Style Illustrations Using a Two-Stage Generator

2020 ◽  
Vol 10 (23) ◽  
pp. 8699
Author(s):  
Yeongseop Lee ◽  
Seongjin Lee
Keyword(s):  
Color Image ◽  
Histogram Equalization ◽  
Media Industry ◽  
Generative Adversarial Network ◽  
Detection Model ◽  
Adversarial Network ◽  
Difference Of Gaussians ◽  
Proposed Model ◽  
The Media ◽  
Line Loss

Line-arts are used in many ways in the media industry. However, line-art colorization is tedious, labor-intensive, and time consuming. For such reasons, a Generative Adversarial Network (GAN)-based image-to-image colorization method has received much attention because of its promising results. In this paper, we propose to use color a point hinting method with two GAN-based generators used for enhancing the image quality. To improve the coloring performance of drawing with various line styles, generator takes account of the loss of the line-art. We propose a Line Detection Model (LDM) which is used in measuring line loss. LDM is a method of extracting line from a color image. We also propose histogram equalizer in the input line-art to generalize the distribution of line styles. This approach allows the generalization of the distribution of line style without increasing the complexity of inference stage. In addition, we propose seven segment hint pointing constraints to evaluate the colorization performance of the model with Fréchet Inception Distance (FID) score. We present visual and qualitative evaluations of the proposed methods. The result shows that using histogram equalization and LDM enabled line loss exhibits the best result. The Base model with XDoG (eXtended Difference-Of-Gaussians)generated line-art with and without color hints exhibits FID for colorized images score of 35.83 and 44.70, respectively, whereas the proposed model in the same scenario exhibits 32.16 and 39.77, respectively.

scholarly journals Accurate Hand Detection from Single-Color Images by Reconstructing Hand Appearances

Sensors ◽  
2019 ◽  
Vol 20 (1) ◽  
pp. 192 ◽  
Author(s):  
Chi Xu ◽  
Wendi Cai ◽  
Yongbo Li ◽  
Jun Zhou ◽  
Longsheng Wei
Keyword(s):  
Detection Method ◽  
Color Image ◽  
Color Images ◽  
Human Hand ◽  
Hand Detection ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Human Activity Analysis ◽  
Proposed Model ◽  
Hybrid Detection

Hand detection is a crucial pre-processing procedure for many human hand related computer vision tasks, such as hand pose estimation, hand gesture recognition, human activity analysis, and so on. However, reliably detecting multiple hands from cluttering scenes remains to be a challenging task because of complex appearance diversities of dexterous human hands (e.g., different hand shapes, skin colors, illuminations, orientations, and scales, etc.) in color images. To tackle this problem, an accurate hand detection method is proposed to reliably detect multiple hands from a single color image using a hybrid detection/reconstruction convolutional neural networks (CNN) framework, in which regions of hands are detected and appearances of hands are reconstructed in parallel by sharing features extracted from a region proposal layer, and the proposed model is trained in an end-to-end manner. Furthermore, it is observed that the generative adversarial network (GAN) could further boost the detection performance by generating more realistic hand appearances. The experimental results show that the proposed approach outperforms the state-of-the-art on public challenging hand detection benchmarks.

Technique for Removing Unnecessary Superimposed Patterns from Image using Generative Network

2021 ◽  
Author(s):  
Kazutake Uehira ◽  
Hiroshi Unno
Keyword(s):  
Color Image ◽  
Depth Map ◽  
Experimental Results ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Blue Component ◽  
Component Image

A technique for removing unnecessary patterns from captured images by using a generative network is studied. The patterns, composed of lines and spaces, are superimposed onto a blue component image of RGB color image when the image is captured for the purpose of acquiring a depth map. The superimposed patterns become unnecessary after the depth map is acquired. We tried to remove these unnecessary patterns by using a generative adversarial network (GAN) and an auto encoder (AE). The experimental results show that the patterns can be removed by using a GAN and AE to the point of being invisible. They also show that the performance of GAN is much higher than that of AE and that its PSNR and SSIM were over 45 and about 0.99, respectively. From the results, we demonstrate the effectiveness of the technique with a GAN.

scholarly journals IntroVNMT: An Introspective Model for Variational Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8830-8837
Author(s):  
Xin Sheng ◽  
Linli Xu ◽  
Junliang Guo ◽  
Jingchang Liu ◽  
Ruoyu Zhao ◽  
...  
Keyword(s):  
Machine Translation ◽  
Latent Variables ◽  
Image Synthesis ◽  
Target Language ◽  
Generative Adversarial Network ◽  
Neural Machine Translation ◽  
Adversarial Network ◽  
Proposed Model ◽  
Model Training ◽  
High Level

We propose a novel introspective model for variational neural machine translation (IntroVNMT) in this paper, inspired by the recent successful application of introspective variational autoencoder (IntroVAE) in high quality image synthesis. Different from the vanilla variational NMT model, IntroVNMT is capable of improving itself introspectively by evaluating the quality of the generated target sentences according to the high-level latent variables of the real and generated target sentences. As a consequence of introspective training, the proposed model is able to discriminate between the generated and real sentences of the target language via the latent variables generated by the encoder of the model. In this way, IntroVNMT is able to generate more realistic target sentences in practice. In the meantime, IntroVNMT inherits the advantages of the variational autoencoders (VAEs), and the model training process is more stable than the generative adversarial network (GAN) based models. Experimental results on different translation tasks demonstrate that the proposed model can achieve significant improvements over the vanilla variational NMT model.

scholarly journals Measuring Traffic Volumes Using an Autoencoder with No Need to Tag Images with Labels

Electronics ◽  
2020 ◽  
Vol 9 (5) ◽  
pp. 702
Author(s):  
Seungbin Roh ◽  
Johyun Shin ◽  
Keemin Sohn
Keyword(s):  
Input Image ◽  
Video Frame ◽  
Generative Adversarial Network ◽  
Detection Algorithms ◽  
Adversarial Network ◽  
Simpler Algorithm ◽  
Proposed Model ◽  
Traffic Volumes ◽  
Step Algorithm ◽  
Almost All

Almost all vision technologies that are used to measure traffic volume use a two-step procedure that involves tracking and detecting. Object detection algorithms such as YOLO and Fast-RCNN have been successfully applied to detecting vehicles. The tracking of vehicles requires an additional algorithm that can trace the vehicles that appear in a previous video frame to their appearance in a subsequent frame. This two-step algorithm prevails in the field but requires substantial computation resources for training, testing, and evaluation. The present study devised a simpler algorithm based on an autoencoder that requires no labeled data for training. An autoencoder was trained on the pixel intensities of a virtual line placed on images in an unsupervised manner. The last hidden node of the former encoding portion of the autoencoder generates a scalar signal that can be used to judge whether a vehicle is passing. A cycle-consistent generative adversarial network (CycleGAN) was used to transform an original input photo of complex vehicle images and backgrounds into a simple illustration input image that enhances the performance of the autoencoder in judging the presence of a vehicle. The proposed model is much lighter and faster than a YOLO-based model, and accuracy of the proposed model is equivalent to, or better than, a YOLO-based model. In measuring traffic volumes, the proposed approach turned out to be robust in terms of both accuracy and efficiency.

scholarly journals Impact of GAN-based lesion-focused medical image super-resolution on the robustness of radiomic features

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Erick Costa de Farias ◽  
Christian di Noia ◽  
Changhee Han ◽  
Evis Sala ◽  
Mauro Castelli ◽  
...  
Keyword(s):  
Biomarker Discovery ◽  
Super Resolution ◽  
Principal Component ◽  
Medical Decision ◽  
Perceptual Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Proposed Model ◽  
Spatial Pyramid Pooling ◽  
Image Super Resolution

AbstractRobust machine learning models based on radiomic features might allow for accurate diagnosis, prognosis, and medical decision-making. Unfortunately, the lack of standardized radiomic feature extraction has hampered their clinical use. Since the radiomic features tend to be affected by low voxel statistics in regions of interest, increasing the sample size would improve their robustness in clinical studies. Therefore, we propose a Generative Adversarial Network (GAN)-based lesion-focused framework for Computed Tomography (CT) image Super-Resolution (SR); for the lesion (i.e., cancer) patch-focused training, we incorporate Spatial Pyramid Pooling (SPP) into GAN-Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE). At $$2\times $$ 2 × SR, the proposed model achieved better perceptual quality with less blurring than the other considered state-of-the-art SR methods, while producing comparable results at $$4\times $$ 4 × SR. We also evaluated the robustness of our model’s radiomic feature in terms of quantization on a different lung cancer CT dataset using Principal Component Analysis (PCA). Intriguingly, the most important radiomic features in our PCA-based analysis were the most robust features extracted on the GAN-super-resolved images. These achievements pave the way for the application of GAN-based image Super-Resolution techniques for studies of radiomics for robust biomarker discovery.

scholarly journals InsulatorGAN: A Transmission Line Insulator Detection Model Using Multi-Granularity Conditional Generative Adversarial Nets for UAV Inspection

2021 ◽  
Vol 13 (19) ◽  
pp. 3971
Author(s):  
Wenxiang Chen ◽  
Yingna Li ◽  
Zhengang Zhao
Keyword(s):  
Transmission Line ◽  
Transmission Lines ◽  
State Of The Art ◽  
Generative Adversarial Network ◽  
Detection Model ◽  
Adversarial Network ◽  
Monte Carlo Search ◽  
Model Training ◽  
Inspection Tasks

Insulator detection is one of the most significant issues in high-voltage transmission line inspection using unmanned aerial vehicles (UAVs) and has attracted attention from researchers all over the world. The state-of-the-art models in object detection perform well in insulator detection, but the precision is limited by the scale of the dataset and parameters. Recently, the Generative Adversarial Network (GAN) was found to offer excellent image generation. Therefore, we propose a novel model called InsulatorGAN based on using conditional GANs to detect insulators in transmission lines. However, due to the fixed categories in datasets such as ImageNet and Pascal VOC, the generated insulator images are of a low resolution and are not sufficiently realistic. To solve these problems, we established an insulator dataset called InsuGenSet for model training. InsulatorGAN can generate high-resolution, realistic-looking insulator-detection images that can be used for data expansion. Moreover, InsulatorGAN can be easily adapted to other power equipment inspection tasks and scenarios using one generator and multiple discriminators. To give the generated images richer details, we also introduced a penalty mechanism based on a Monte Carlo search in InsulatorGAN. In addition, we proposed a multi-scale discriminator structure based on a multi-task learning mechanism to improve the quality of the generated images. Finally, experiments on the InsuGenSet and CPLID datasets demonstrated that our model outperforms existing state-of-the-art models by advancing both the resolution and quality of the generated images as well as the position of the detection box in the images.

scholarly journals Data augmentation-based conditional Wasserstein generative adversarial network-gradient penalty for XSS attack detection system

2020 ◽  
Vol 6 ◽  
pp. e328
Author(s):  
Fawaz Mahiuob Mohammed Mokbal ◽  
Dan Wang ◽  
Xiaoxi Wang ◽  
Lihua Fu
Keyword(s):  
Web Applications ◽  
Data Augmentation ◽  
Detection System ◽  
Attack Detection ◽  
Generative Adversarial Network ◽  
Minority Class ◽  
Adversarial Network ◽  
Proposed Model ◽  
Worldwide Web ◽  
Resource Data

The rapid growth of the worldwide web and accompanied opportunities of web applications in various aspects of life have attracted the attention of organizations, governments, and individuals. Consequently, web applications have increasingly become the target of cyberattacks. Notably, cross-site scripting (XSS) attacks on web applications are increasing and have become the critical focus of information security experts’ reports. Machine learning (ML) technique has significantly advanced and shown impressive results in the area of cybersecurity. However, XSS training datasets are often limited and significantly unbalanced, which does not meet well-developed ML algorithms’ requirements and potentially limits the detection system efficiency. Furthermore, XSS attacks have multiple payload vectors that execute in different ways, resulting in many real threats passing through the detection system undetected. In this study, we propose a conditional Wasserstein generative adversarial network with a gradient penalty to enhance the XSS detection system in a low-resource data environment. The proposed method integrates a conditional generative adversarial network and Wasserstein generative adversarial network with a gradient penalty to obtain necessary data from directivity, which improves the strength of the security system over unbalance data. The proposed method generates synthetic samples of minority class that have identical distribution as real XSS attack scenarios. The augmented data were used to train a new boosting model and subsequently evaluated the model using a real test dataset. Experiments on two unbalanced XSS attack datasets demonstrate that the proposed model generates valid and reliable samples. Furthermore, the samples were indistinguishable from real XSS data and significantly enhanced the detection of XSS attacks compared with state-of-the-art methods.

scholarly journals Generative Adversarial Network for Imitation Learning from Single Demonstration

2021 ◽  
Vol 18 (4(Suppl.)) ◽  
pp. 1350
Author(s):  
Tho Nguyen Duc ◽  
Chanh Minh Tran ◽  
Phan Xuan Tan ◽  
Eiji Kamioka
Keyword(s):  
Imitation Learning ◽  
Autonomous Agent ◽  
Complex Behavior ◽  
Learning Methods ◽  
Generative Adversarial Network ◽  
Complex Tasks ◽  
Adversarial Network ◽  
Proposed Model ◽  
Optimal Policies ◽  
Single Demonstration

Imitation learning is an effective method for training an autonomous agent to accomplish a task by imitating expert behaviors in their demonstrations. However, traditional imitation learning methods require a large number of expert demonstrations in order to learn a complex behavior. Such a disadvantage has limited the potential of imitation learning in complex tasks where the expert demonstrations are not sufficient. In order to address the problem, we propose a Generative Adversarial Network-based model which is designed to learn optimal policies using only a single demonstration. The proposed model is evaluated on two simulated tasks in comparison with other methods. The results show that our proposed model is capable of completing considered tasks despite the limitation in the number of expert demonstrations, which clearly indicate the potential of our model.

scholarly journals Asymmetric Encoder-Decoder Structured FCN Based LiDAR to Color Image Generation

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4818 ◽  
Author(s):  
Hyun-Koo Kim ◽  
Kook-Yeol Yoo ◽  
Ju H. Park ◽  
Ho-Youl Jung
Keyword(s):  
Color Image ◽  
Ground Truth ◽  
Sensor Data ◽  
Image Generation ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Adversarial Network ◽  
Ground Truth Image ◽  
And Performance ◽  
Reflection Intensity

In this paper, we propose a method of generating a color image from light detection and ranging (LiDAR) 3D reflection intensity. The proposed method is composed of two steps: projection of LiDAR 3D reflection intensity into 2D intensity, and color image generation from the projected intensity by using a fully convolutional network (FCN). The color image should be generated from a very sparse projected intensity image. For this reason, the FCN is designed to have an asymmetric network structure, i.e., the layer depth of the decoder in the FCN is deeper than that of the encoder. The well-known KITTI dataset for various scenarios is used for the proposed FCN training and performance evaluation. Performance of the asymmetric network structures are empirically analyzed for various depth combinations for the encoder and decoder. Through simulations, it is shown that the proposed method generates fairly good visual quality of images while maintaining almost the same color as the ground truth image. Moreover, the proposed FCN has much higher performance than conventional interpolation methods and generative adversarial network based Pix2Pix. One interesting result is that the proposed FCN produces shadow-free and daylight color images. This result is caused by the fact that the LiDAR sensor data is produced by the light reflection and is, therefore, not affected by sunlight and shadow.

Sign in / Sign up

Export Citation Format

Share Document