scholarly journals A New Image Completion Method Inserting an Image Generated by Sketch Image

2021 ◽  
Vol 10 (4) ◽  
pp. 14-18
Author(s):  
Hyung-Hwa Ko ◽  
GilHee Choi ◽  
KyoungHak Lee
Keyword(s):  
Training Data ◽  
Object Image ◽  
Image Completion ◽  
Translation Model ◽  
Image Translation ◽  
Convolution Process ◽  
Realistic Image

Recently, many studies on the image completion methods make us erase obstacles and fill the hole realistically but putting a new object in its place cannot be solved with the existing Image Completion. To solve this problem, this paper proposes Image Completion which filled a new object that is created through sketch image. The proposed network use pix2pix image translation model for generating object image from sketch image. The image completion network used gated convolution to reduce the weight of meaningless pixels in the convolution process. And WGAN-GP loss is used to reduce the mode dropping. In addition, by adding a contextual attention layer in the middle of the network, image completion is performed by referring to the feature value at a distant pixel. To train the models, Places2 dataset was used as background training data for image completion and Standard Dog dataset was used as training data for pix2pix. As a result of the experiment, an image of dog is generated well by sketch image and use this image as an input of the image completion network, it can generate the realistic image as a result.

RECONSTRUCTED TEETH IMAGE FROM BRACES WITH GAN

2021 ◽  
pp. 2150043
Author(s):  
Vu Tuan Hai ◽  
Dang Thanh Vu ◽  
Huynh Ho Thi Mong Trinh ◽  
Pham The Bao
Keyword(s):  
Health Care ◽  
Deep Learning ◽  
Training Data ◽  
Learning Models ◽  
Paired Data ◽  
Object Removal ◽  
Interactive Tool ◽  
Learning Techniques ◽  
Large Context ◽  
Image Translation

Recent advances in deep learning models have shown promising potential in object removal, which is the task of replacing undesired objects with appropriate pixel values using known context. Object removal-based deep learning can commonly be solved by modeling it as the Img2Img (image to image) translation or Inpainting. Instead of dealing with a large context, this paper aims at a specific application of object removal, that is, erasing braces trace out of an image having teeth with braces (called braces2teeth problem). We solved the problem by three methods corresponding to different datasets. Firstly, we use the CycleGAN model to deal with the problem that paired training data is not available. In the second case, we try to create pseudo-paired data to train the Pix2Pix model. In the last case, we utilize GraphCut combining generative inpainting model to build a user-interactive tool that can improve the result in case the user is not satisfied with previous results. To our best knowledge, this study is one of the first attempts to take the braces2teeth problem into account by using deep learning techniques and it can be applied in various fields, from health care to entertainment.

scholarly journals Neural Sign Language Translation Based on Human Keypoint Estimation

2019 ◽  
Vol 9 (13) ◽  
pp. 2683 ◽  
Author(s):  
Sang-Ki Ko ◽  
Chang Jo Kim ◽  
Hyedong Jung ◽  
Choongsang Cho
Keyword(s):  
Neural Network ◽  
Sign Language ◽  
Language Translation ◽  
Network Models ◽  
Training Data ◽  
Translation System ◽  
Body Parts ◽  
Neural Network Models ◽  
Translation Model ◽  
Starting Point

We propose a sign language translation system based on human keypoint estimation. It is well-known that many problems in the field of computer vision require a massive dataset to train deep neural network models. The situation is even worse when it comes to the sign language translation problem as it is far more difficult to collect high-quality training data. In this paper, we introduce the KETI (Korea Electronics Technology Institute) sign language dataset, which consists of 14,672 videos of high resolution and quality. Considering the fact that each country has a different and unique sign language, the KETI sign language dataset can be the starting point for further research on the Korean sign language translation. Using the KETI sign language dataset, we develop a neural network model for translating sign videos into natural language sentences by utilizing the human keypoints extracted from the face, hands, and body parts. The obtained human keypoint vector is normalized by the mean and standard deviation of the keypoints and used as input to our translation model based on the sequence-to-sequence architecture. As a result, we show that our approach is robust even when the size of the training data is not sufficient. Our translation model achieved 93.28% (55.28%, respectively) translation accuracy on the validation set (test set, respectively) for 105 sentences that can be used in emergency situations. We compared several types of our neural sign translation models based on different attention mechanisms in terms of classical metrics for measuring the translation performance.

scholarly journals Correction to: Generative image completion with image-to-image translation

2020 ◽  
Vol 32 (23) ◽  
pp. 17809-17809
Author(s):  
Shuzhen Xu ◽  
Qing Zhu ◽  
Jin Wang
Keyword(s):  
Image Completion ◽  
Image Translation

Statistical Translation After Source Reordering: Oracles, Context-Aware Models, and Empirical Analysis

2012 ◽  
Vol 18 (4) ◽  
pp. 491-519 ◽  
Author(s):  
MAXIM KHALILOV ◽  
KHALIL SIMA'AN
Keyword(s):  
Word Order ◽  
Target Sentence ◽  
Training Data ◽  
Translation System ◽  
Long Distance ◽  
Translation Model ◽  
Unseen Data ◽  
Carry Over ◽  
Set Up ◽  
The Impact

AbstractIn source reordering the order of the source words is permuted to minimize word order differences with the target sentence and then fed to a translation model. Earlier work highlights the benefits of resolving long-distance reorderings as a pre-processing step to standard phrase-based models. However, the potential performance improvement of source reordering and its impact on the components of the subsequent translation model remain unexplored. In this paper we study both aspects of source reordering. We set up idealized source reordering (oracle) models with/without syntax and present our own syntax-driven model of source reordering. The latter is a statistical model of inversion transduction grammar (ITG)-like tree transductions manipulating a syntactic parse and working with novel conditional reordering parameters. Having set up the models, we report translation experiments showing significant improvement on three language pairs, and contribute an extensive analysis of the impact of source reordering (both oracle and model) on the translation model regarding the quality of its input, phrase-table, and output. Our experiments show that oracle source reordering has untapped potential in improving translation system output. Besides solving difficult reorderings, we find that source reordering creates more monotone parallel training data at the back-end, leading to significantly larger phrase tables with higher coverage of phrase types in unseen data. Unfortunately, this nice property does not carry over to tree-constrained source reordering. Our analysis shows that, from the string-level perspective, tree-constrained reordering might selectively permute word order, leading to larger phrase tables but without increase in phrase coverage in unseen data.

Generative image completion with image-to-image translation

2019 ◽  
Vol 32 (11) ◽  
pp. 7333-7345 ◽  
Author(s):  
Shuzhen Xu ◽  
Qing Zhu ◽  
Jin Wang
Keyword(s):  
Image Completion ◽  
Image Translation

scholarly journals Accurate and Consistent Image-to-Image Conditional Adversarial Network

Electronics ◽  
2020 ◽  
Vol 9 (3) ◽  
pp. 395 ◽  
Author(s):  
Naeem Ul Islam ◽  
Sungmin Lee ◽  
Jaebyung Park
Keyword(s):  
Image Representation ◽  
Terrain Analysis ◽  
Generative Adversarial Network ◽  
Training Strategy ◽  
Adversarial Network ◽  
Image Translation ◽  
Successful Approach ◽  
Realistic Image ◽  
Potential Impact ◽  
Realistic Images

Image-to-image translation based on deep learning has attracted interest in the robotics and vision community because of its potential impact on terrain analysis and image representation, interpretation, modification, and enhancement. Currently, the most successful approach for generating a translated image is a conditional generative adversarial network (cGAN) for training an autoencoder with skip connections. Despite its impressive performance, it has low accuracy and a lack of consistency; further, its training is imbalanced. This paper proposes a balanced training strategy for image-to-image translation, resulting in an accurate and consistent network. The proposed approach uses two generators and a single discriminator. The generators translate images from one domain to another. The discriminator takes the input of three different configurations and guides both the generators to generate realistic images in their corresponding domains while ensuring high accuracy and consistency. Experiments are conducted on different datasets. In particular, the proposed approach outperforms the cGAN in realistic image translation in terms of accuracy and consistency in training.

Photo Realistic Image Completion via Dense Correspondence

2018 ◽  
Vol 27 (11) ◽  
pp. 5234-5247 ◽  
Author(s):  
Jun-Jie Huang ◽  
Pier Luigi Dragotti
Keyword(s):  
Image Completion ◽  
Realistic Image ◽  
Dense Correspondence

scholarly journals Seg2pix: Few Shot Training Line Art Colorization with Segmented Image Data

2021 ◽  
Vol 11 (4) ◽  
pp. 1464
Author(s):  
Chang Wook Seo ◽  
Yongduek Seo
Keyword(s):  
Image Data ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Training Data ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Segmented Image ◽  
Image Translation ◽  
Segmentation Image

There are various challenging issues in automating line art colorization. In this paper, we propose a GAN approach incorporating semantic segmentation image data. Our GAN-based method, named Seg2pix, can automatically generate high quality colorized images, aiming at computerizing one of the most tedious and repetitive jobs performed by coloring workers in the webtoon industry. The network structure of Seg2pix is mostly a modification of the architecture of Pix2pix, which is a convolution-based generative adversarial network for image-to-image translation. Through this method, we can generate high quality colorized images of a particular character with only a few training data. Seg2pix is designed to reproduce a segmented image, which becomes the suggestion data for line art colorization. The segmented image is automatically generated through a generative network with a line art image and a segmentation ground truth. In the next step, this generative network creates a colorized image from the line art and segmented image, which is generated from the former step of the generative network. To summarize, only one line art image is required for testing the generative model, and an original colorized image and segmented image are additionally required as the ground truth for training the model. These generations of the segmented image and colorized image proceed by an end-to-end method sharing the same loss functions. By using this method, we produce better qualitative results for automatic colorization of a particular character’s line art. This improvement can also be measured by quantitative results with Learned Perceptual Image Patch Similarity (LPIPS) comparison. We believe this may help artists exercise their creative expertise mainly in the area where computerization is not yet capable.

Sign in / Sign up

Export Citation Format

Share Document