scholarly journals GTAE: Graph Transformer–Based Auto-Encoders for Linguistic-Constrained Text Style Transfer

2021 ◽  
Vol 12 (3) ◽  
pp. 1-16
Author(s):  
Yukai Shi ◽  
Sen Zhang ◽  
Chenxing Zhou ◽  
Xiaodan Liang ◽  
Xiaojun Yang ◽  
...  
Keyword(s):  
Deep Neural Networks ◽  
Model Space ◽  
Style Transfer ◽  
Unconstrained Model ◽  
Linguistic Rules ◽  
Comparable Performance ◽  
Parallel Text ◽  
Content Preservation ◽  
Transfer Tasks ◽  
Transfer Accuracy

Non-parallel text style transfer has attracted increasing research interests in recent years. Despite successes in transferring the style based on the encoder-decoder framework, current approaches still lack the ability to preserve the content and even logic of original sentences, mainly due to the large unconstrained model space or too simplified assumptions on latent embedding space. Since language itself is an intelligent product of humans with certain grammars and has a limited rule-based model space by its nature, relieving this problem requires reconciling the model capacity of deep neural networks with the intrinsic model constraints from human linguistic rules. To this end, we propose a method called Graph Transformer–based Auto-Encoder, which models a sentence as a linguistic graph and performs feature extraction and style transfer at the graph level, to maximally retain the content and the linguistic structure of original sentences. Quantitative experiment results on three non-parallel text style transfer tasks show that our model outperforms state-of-the-art methods in content preservation, while achieving comparable performance on transfer accuracy and sentence naturalness.

scholarly journals Text Style Transfer via Learning Style Instance Supported Latent Space

2020 ◽  
Author(s):  
Xiaoyuan Yi ◽  
Zhenghao Liu ◽  
Wenhao Li ◽  
Maosong Sun
Keyword(s):  
Recent Work ◽  
Learning Style ◽  
Latent Variables ◽  
Paired Data ◽  
Style Transfer ◽  
Parallel Corpora ◽  
Latent Space ◽  
Content Preservation ◽  
Transfer Tasks ◽  
Transfer Accuracy

Text style transfer pursues altering the style of a sentence while remaining its main content unchanged. Due to the lack of parallel corpora, most recent work focuses on unsupervised methods and has achieved noticeable progress. Nonetheless, the intractability of completely disentangling content from style for text leads to a contradiction of content preservation and style transfer accuracy. To address this problem, we propose a style instance supported method, StyIns. Instead of representing styles with embeddings or latent variables learned from single sentences, our model leverages the generative flow technique to extract underlying stylistic properties from multiple instances of each style, which form a more discriminative and expressive latent style space. By combining such a space with the attention-based structure, our model can better maintain the content and simultaneously achieve high transfer accuracy. Furthermore, the proposed method can be flexibly extended to semi-supervised learning so as to utilize available limited paired data. Experiments on three transfer tasks, sentiment modification, formality rephrasing, and poeticness generation, show that StyIns obtains a better balance between content and style, outperforming several recent baselines.

scholarly journals Mask Transformer: Unpaired Text Style Transfer Based on Masked Language

2020 ◽  
Vol 10 (18) ◽  
pp. 6196
Author(s):  
Chunhua Wu ◽  
Xiaolong Chen ◽  
Xingbiao Li
Keyword(s):  
Target Sentence ◽  
Explicit Representation ◽  
Experimental Results ◽  
Current Work ◽  
Semantic Meaning ◽  
Style Transfer ◽  
Original Sentence ◽  
Generation Structure ◽  
Content Preservation ◽  
Transfer Accuracy

Currently, most text style transfer methods encode the text into a style-independent latent representation and decode it into new sentences with the target style. Due to the limitation of the latent representation, previous works can hardly get satisfactory target style sentence especially in terms of semantic remaining of the original sentence. We propose a “Mask and Generation” structure, which can obtain an explicit representation of the content of original sentence and generate the target sentence with a transformer. This explicit representation is a masked text that masks the words with the strong style attribute in the sentence. Therefore, it can preserve most of the semantic meaning of the original sentence. In addition, as it is the input of the generator, it also simplified this process compared to the current work who generate the target sentence from scratch. As the explicit representation is readable and the model has better interpretability, we can clearly know which words changed and why the words changed. We evaluate our model on two review datasets with quantitative, qualitative, and human evaluations. The experimental results show that our model generally outperform other methods in terms of transfer accuracy and content preservation.

scholarly journals Recognizing Instagram Filtered Images with Feature De-Stylization

2020 ◽  
Vol 34 (07) ◽  
pp. 12418-12425
Author(s):  
Zhe Wu ◽  
Zuxuan Wu ◽  
Bharat Singh ◽  
Larry Davis
Keyword(s):  
Deep Neural Networks ◽  
Simple Structure ◽  
Feature Space ◽  
Natural Image ◽  
Feature Maps ◽  
Small Perturbations ◽  
Style Transfer ◽  
Image Transformations ◽  
Transfer Tasks ◽  
Entire Network

Deep neural networks have been shown to suffer from poor generalization when small perturbations are added (like Gaussian noise), yet little work has been done to evaluate their robustness to more natural image transformations like photo filters. This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. To this end, we introduce ImageNet-Instagram, a filtered version of ImageNet, where 20 popular Instagram filters are applied to each image in ImageNet. Our analysis suggests that simple structure preserving filters which only alter the global appearance of an image can lead to large differences in the convolutional feature space. To improve generalization, we introduce a lightweight de-stylization module that predicts parameters used for scaling and shifting feature maps to “undo” the changes incurred by filters, inverting the process of style transfer tasks. We further demonstrate the module can be readily plugged into modern CNN architectures together with skip connections. We conduct extensive studies on ImageNet-Instagram, and show quantitatively and qualitatively, that the proposed module, among other things, can effectively improve generalization by simply learning normalization parameters without retraining the entire network, thus recovering the alterations in the feature space caused by the filters.

scholarly journals Utilizing Non-Parallel Text for Style Transfer by Making Partial Comparisons

2019 ◽  
Author(s):  
Di Yin ◽  
Shujian Huang ◽  
Xin-Yu Dai ◽  
Jiajun Chen
Keyword(s):  
Data Augmentation ◽  
State Of The Art ◽  
Language Models ◽  
Style Transfer ◽  
Parallel Corpora ◽  
Parallel Data ◽  
Distributional Information ◽  
Parallel Text ◽  
Using Data ◽  
Content Preservation

Text style transfer aims to rephrase a given sentence into a different style without changing its original content. Since parallel corpora (i.e. sentence pairs with the same content but different styles) are usually unavailable, most previous works solely guide the transfer process with distributional information, i.e. using style-related classifiers or language models, which neglect the correspondence of instances, leading to poor transfer performance, especially for the content preservation. In this paper, we propose making partial comparisons to explicitly model the content and style correspondence of instances, respectively. To train the partial comparators, we propose methods to extract partial-parallel training instances automatically from the non-parallel data, and to further enhance the training process by using data augmentation. We perform experiments that compare our method to other existing approaches on two review datasets. Both automatic and manual evaluations show that our approach can significantly improve the performance of existing adversarial methods, and outperforms most state-of-the-art models. Our code and data will be available on Github.

scholarly journals Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 8376-8383
Author(s):  
Dayiheng Liu ◽  
Jie Fu ◽  
Yidan Zhang ◽  
Chris Pal ◽  
Jiancheng Lv
Keyword(s):  
Experimental Studies ◽  
Target Sentence ◽  
Sentence Length ◽  
Adversarial Learning ◽  
Style Transfer ◽  
Continuous Space ◽  
Fine Grained ◽  
Gradient Based ◽  
Transfer Tasks ◽  
New Framework

Typical methods for unsupervised text style transfer often rely on two key ingredients: 1) seeking the explicit disentanglement of the content and the attributes, and 2) troublesome adversarial learning. In this paper, we show that neither of these components is indispensable. We propose a new framework that utilizes the gradients to revise the sentence in a continuous space during inference to achieve text style transfer. Our method consists of three key components: a variational auto-encoder (VAE), some attribute predictors (one for each attribute), and a content predictor. The VAE and the two types of predictors enable us to perform gradient-based optimization in the continuous space, which is mapped from sentences in a discrete space, to find the representation of a target sentence with the desired attributes and preserved content. Moreover, the proposed method naturally has the ability to simultaneously manipulate multiple fine-grained attributes, such as sentence length and the presence of specific words, when performing text style transfer tasks. Compared with previous adversarial learning based methods, the proposed method is more interpretable, controllable and easier to train. Extensive experimental studies on three popular text style transfer tasks show that the proposed method significantly outperforms five state-of-the-art methods.

scholarly journals Towards Fully 8-bit Integer Inference for the Transformer Model

2020 ◽  
Author(s):  
Ye Lin ◽  
Yanyang Li ◽  
Tengbo Liu ◽  
Tong Xiao ◽  
Tongran Liu ◽  
...  
Keyword(s):  
Deep Neural Networks ◽  
Floating Point ◽  
Memory Footprint ◽  
Great Progress ◽  
Comparable Performance ◽  
Complex Models ◽  
Language Modelling ◽  
Transformer Model ◽  
And Storage ◽  
Modelling Task

8-bit integer inference, as a promising direction in reducing both the latency and storage of deep neural networks, has made great progress recently. On the other hand, previous systems still rely on 32-bit floating point for certain functions in complex models (e.g., Softmax in Transformer), and make heavy use of quantization and de-quantization. In this work, we show that after a principled modification on the Transformer architecture, dubbed Integer Transformer, an (almost) fully 8-bit integer inference algorithm Scale Propagation could be derived. De-quantization is adopted when necessary, which makes the network more efficient. Our experiments on WMT16 En<->Ro, WMT14 En<->De and En->Fr translation tasks as well as the WikiText-103 language modelling task show that the fully 8-bit Transformer system achieves comparable performance with the floating point baseline but requires nearly 4x less memory footprint.

scholarly journals Specific Area Style Transfer on Real-Time Video

2021 ◽  
Vol 10 (5) ◽  
pp. 50-56
Author(s):  
Hyung-Hwa Ko ◽  
GeunTae Kim ◽  
Hyunmin Kim
Keyword(s):  
Real Time ◽  
Deep Neural Networks ◽  
Specific Area ◽  
Image Generation ◽  
Style Transfer ◽  
Real Time Processing ◽  
Time Processing ◽  
Related Research ◽  
City Street ◽  
Speed And Accuracy

Since deep learning applications in object recognition, object detection, segmentation, and image generation are needed increasingly, related research has been actively conducted. In this paper, using segmentation and style transfer together, a method of producing desired images in the desired area in real-time video is proposed. Two deep neural networks were used to enable as possible as in real-time with the trade-off relationship between speed and accuracy. Modified BiSeNet for segmentation and CycleGAN for style transfer were processed on a desktop PC equipped with two RTX-2080-Ti GPU boards. This enables real-time processing over SD video in decent level. We obtained good results in subjective quality to segment Road area in city street video and change into the Grass style at no less than 6(fps).

scholarly journals Interactive Artistic Multi-style Transfer

2021 ◽  
Vol 14 (1) ◽  
Author(s):  
Xiaohui Wang ◽  
Yiran Lyu ◽  
Junfeng Huang ◽  
Ziying Wang ◽  
Jingyan Qin
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Deep Neural Networks ◽  
Subjective Evaluation ◽  
User Interaction ◽  
Evaluation Framework ◽  
Transfer System ◽  
Style Transfer ◽  
Artistic Style ◽  
Artistic Styles

AbstractArtistic style transfer is to render an image in the style of another image, which is a challenge problem in both image processing and arts. Deep neural networks are adopted to artistic style transfer and achieve remarkable success, such as AdaIN (adaptive instance normalization), WCT (whitening and coloring transforms), MST (multimodal style transfer), and SEMST (structure-emphasized multimodal style transfer). These algorithms modify the content image as a whole using only one style and one algorithm, which is easy to cause the foreground and background to be blurred together. In this paper, an iterative artistic multi-style transfer system is built to edit the image with multiple styles by flexible user interaction. First, a subjective evaluation experiment with art professionals is conducted to build an open evaluation framework for style transfer, including the universal evaluation questions and personalized answers for ten typical artistic styles. Then, we propose the interactive artistic multi-style transfer system, in which an interactive image crop tool is designed to cut a content image into several parts. For each part, users select a style image and an algorithm from AdaIN, WCT, MST, and SEMST by referring to the characteristics of styles and algorithms summarized by the evaluation experiments. To obtain richer results, the system provides a semantic-based parameter adjustment mode and the function of preserving colors of content image. Finally, case studies show the effectiveness and flexibility of the system.

Sign in / Sign up

Export Citation Format

Share Document