scholarly journals Distribution Aligned Multimodal and Multi-domain Image Stylization

2021 ◽  
Vol 17 (3) ◽  
pp. 1-17
Author(s):  
Minxuan Lin ◽  
Fan Tang ◽  
Weiming Dong ◽  
Xiao Li ◽  
Changsheng Xu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Unified Framework ◽  
High Quality ◽  
Style Transfer ◽  
Multiple Images ◽  
Qualitative And Quantitative ◽  
Image Stylization ◽  
Art Methods ◽  
Artistic Styles ◽  
Painting Style Transfer

Multimodal and multi-domain stylization are two important problems in the field of image style transfer. Currently, there are few methods that can perform multimodal and multi-domain stylization simultaneously. In this study, we propose a unified framework for multimodal and multi-domain style transfer with the support of both exemplar-based reference and randomly sampled guidance. The key component of our method is a novel style distribution alignment module that eliminates the explicit distribution gaps between various style domains and reduces the risk of mode collapse. The multimodal diversity is ensured by either guidance from multiple images or random style codes, while the multi-domain controllability is directly achieved by using a domain label. We validate our proposed framework on painting style transfer with various artistic styles and genres. Qualitative and quantitative comparisons with state-of-the-art methods demonstrate that our method can generate high-quality results of multi-domain styles and multimodal instances from reference style guidance or a random sampled style.

scholarly journals TET-GAN: Text Effects Transfer via Stylization and Destylization

2019 ◽  
Vol 33 ◽  
pp. 1238-1245 ◽  
Author(s):  
Shuai Yang ◽  
Jiaying Liu ◽  
Wenjing Wang ◽  
Zongming Guo
Keyword(s):  
Network Design ◽  
State Of The Art ◽  
The State ◽  
High Quality ◽  
Style Transfer ◽  
Feature Representations ◽  
Art Methods ◽  
Matching Criteria

Text effects transfer technology automatically makes the text dramatically more impressive. However, previous style transfer methods either study the model for general style, which cannot handle the highly-structured text effects along the glyph, or require manual design of subtle matching criteria for text effects. In this paper, we focus on the use of the powerful representation abilities of deep neural features for text effects transfer. For this purpose, we propose a novel Texture Effects Transfer GAN (TET-GAN), which consists of a stylization subnetwork and a destylization subnetwork. The key idea is to train our network to accomplish both the objective of style transfer and style removal, so that it can learn to disentangle and recombine the content and style features of text effects images. To support the training of our network, we propose a new text effects dataset with as much as 64 professionally designed styles on 837 characters. We show that the disentangled feature representations enable us to transfer or remove all these styles on arbitrary glyphs using one network. Furthermore, the flexible network design empowers TET-GAN to efficiently extend to a new text style via oneshot learning where only one example is required. We demonstrate the superiority of the proposed method in generating high-quality stylized text over the state-of-the-art methods.

scholarly journals HIGH QUALITY FACADE SEGMENTATION BASED ON STRUCTURED RANDOM FOREST, REGION PROPOSAL NETWORK AND RECTANGULAR FITTING

2018 ◽  
Vol IV-2 ◽  
pp. 223-230 ◽  
Author(s):  
K. Rahmani ◽  
H. Mayer
Keyword(s):  
Neural Network ◽  
Random Forest ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
High Quality ◽  
Current State ◽  
Forest Region ◽  
Building Facades ◽  
Art Methods

In this paper we present a pipeline for high quality semantic segmentation of building facades using Structured Random Forest (SRF), Region Proposal Network (RPN) based on a Convolutional Neural Network (CNN) as well as rectangular fitting optimization. Our main contribution is that we employ features created by the RPN as channels in the SRF.We empirically show that this is very effective especially for doors and windows. Our pipeline is evaluated on two datasets where we outperform current state-of-the-art methods. Additionally, we quantify the contribution of the RPN and the rectangular fitting optimization on the accuracy of the result.

scholarly journals Deep Video Frame Interpolation Using Cyclic Frame Generation

2019 ◽  
Vol 33 ◽  
pp. 8794-8802 ◽  
Author(s):  
Yu-Lun Liu ◽  
Yi-Tung Liao ◽  
Yen-Yu Lin ◽  
Yung-Yu Chuang
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
Training Data ◽  
Video Frame ◽  
High Quality ◽  
Edge Information ◽  
Frame Interpolation ◽  
Source Codes ◽  
Qualitative And Quantitative ◽  
Loss Term

Video frame interpolation algorithms predict intermediate frames to produce videos with higher frame rates and smooth view transitions given two consecutive frames as inputs. We propose that: synthesized frames are more reliable if they can be used to reconstruct the input frames with high quality. Based on this idea, we introduce a new loss term, the cycle consistency loss. The cycle consistency loss can better utilize the training data to not only enhance the interpolation results, but also maintain the performance better with less training data. It can be integrated into any frame interpolation network and trained in an end-to-end manner. In addition to the cycle consistency loss, we propose two extensions: motion linearity loss and edge-guided training. The motion linearity loss approximates the motion between two input frames to be linear and regularizes the training. By applying edge-guided training, we further improve results by integrating edge information into training. Both qualitative and quantitative experiments demonstrate that our model outperforms the state-of-the-art methods. The source codes of the proposed method and more experimental results will be available at https://github.com/alex04072000/CyclicGen.

scholarly journals Monolingual Transfer Learning via Bilingual Translators for Style-Sensitive Paraphrase Generation

2020 ◽  
Vol 34 (05) ◽  
pp. 8042-8049
Author(s):  
Tomoyuki Kajiwara ◽  
Biwa Miura ◽  
Yuki Arase
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
State Of The Art ◽  
Experimental Results ◽  
Fine Tuning ◽  
Training Methods ◽  
High Quality ◽  
Style Transfer ◽  
Parallel Corpus ◽  
Paraphrase Generation

We tackle the low-resource problem in style transfer by employing transfer learning that utilizes abundantly available raw corpora. Our method consists of two steps: pre-training learns to generate a semantically equivalent sentence with an input assured grammaticality, and fine-tuning learns to add a desired style. Pre-training has two options, auto-encoding and machine translation based methods. Pre-training based on AutoEncoder is a simple way to learn these from a raw corpus. If machine translators are available, the model can learn more diverse paraphrasing via roundtrip translation. After these, fine-tuning achieves high-quality paraphrase generation even in situations where only 1k sentence pairs of the parallel corpus for style transfer is available. Experimental results of formality style transfer indicated the effectiveness of both pre-training methods and the method based on roundtrip translation achieves state-of-the-art performance.

scholarly journals Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching

2021 ◽  
Author(s):  
Bofeng Wu ◽  
Guocheng Niu ◽  
Jun Yu ◽  
Xinyan Xiao ◽  
Jian Zhang ◽  
...  
Keyword(s):  
State Of The Art ◽  
Semantic Matching ◽  
High Quality ◽  
Video Captioning ◽  
Art Methods ◽  
Knowledge Distillation ◽  
Matching Performance ◽  
Weakly Supervised

This paper proposes an approach to Dense Video Captioning (DVC) without pairwise event-sentence annotation. First, we adopt the knowledge distilled from relevant and well solved tasks to generate high-quality event proposals. Then we incorporate contrastive loss and cycle-consistency loss typically applied to cross-modal retrieval tasks to build semantic matching between the proposals and sentences, which are eventually used to train the caption generation module. In addition, the parameters of matching module are initialized via pre-training based on annotated images to improve the matching performance. Extensive experiments on ActivityNet-Caption dataset reveal the significance of distillation-based event proposal generation and cross-modal retrieval-based semantic matching to weakly supervised DVC, and demonstrate the superiority of our method to existing state-of-the-art methods.

Industrial-Scale Production of High-Quality Graphene Sheets by Millstone Grinders

2021 ◽  
Author(s):  
Peng Lv ◽  
Xiaoshi Li ◽  
Zihan Zhang ◽  
Biao Nie ◽  
Yiliang Wu ◽  
...  
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
The State ◽  
Industrial Scale ◽  
Graphene Sheets ◽  
Scale Production ◽  
Large Area ◽  
High Quality ◽  
Art Methods ◽  
Mechanical Cleavage

Abstract Graphene exhibits a variety of unprecedented innate properties and has sparked great interest in both fundamental science and regarding prospective commercial applications. To meet the ever-increasing demand for high-quality graphene sheets, an industrial-scale, reliable, environmental-friendly, low-cost production process is required. However, large-scale production high quality graphene remains elusive. Here we demonstrate a scalable mechanical cleavage method for large-quantity production of high quality large-area and few-layer graphene sheets by introducing a millstone grinding process. The average thickness of the graphene sheets is around 5 nm. This procedure is simpler than the state-of-the-art methods that allows for scalable preparation of graphene dispersion in hundreds of litres by mechanical cleavage of graphite, and the yield is 30-40%. The size of the prepared graphene sheets can be tuneable from few micrometres to tens of micrometres by varying the dimension of raw graphite, which is larger than that produced by the state-of-the-art methods. Moreover, comparing to conductive agents, the conductivity of wafers containing graphene can be increased by one order of magnitude, suggesting a high potential of the prepared graphene sheets for the application as conductive agent in lithium battery cathodes. This allows the requirements of different sizes graphene sheets for industry applications in different fields.

scholarly journals A Foreground-Aware Framework for Local Face Attribute Transfer

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 615
Author(s):  
Yuanbin Fu ◽  
Jiayi Ma ◽  
Xiaojie Guo
Keyword(s):  
Social Media ◽  
State Of The Art ◽  
Input Image ◽  
Experimental Results ◽  
Reference Image ◽  
Background Region ◽  
High Quality ◽  
Style Transfer ◽  
Internet Users ◽  
Foreground Region

In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there are many algorithms automatically and globally transferring the style from one image to another, they fail to respect the semantics of the scene and are unable to allow users to merely transfer the attributes of one or two face organs in the foreground region leaving the background region unchanged. To overcome this problem, we developed a novel framework for semantically meaningful local face attribute transfer, which can flexibly transfer the local attribute of a face organ from the reference image to a semantically equivalent organ in the input image, while preserving the background. Our method involves warping the reference photo to match the shape, pose, location, and expression of the input image. The fusion of the warped reference image and input image is then taken as the initialized image for a neural style transfer algorithm. Our method achieves better performance in terms of inception score (3.81) and Fréchet inception distance (80.31), which is about 10% higher than those of competitors, indicating that our framework is capable of producing high-quality and photorealistic attribute transfer results. Both theoretical findings and experimental results are provided to demonstrate the efficacy of the proposed framework, reveal its superiority over other state-of-the-art alternatives.

scholarly journals Multi-hop assortativities for network classification

2018 ◽  
Vol 7 (4) ◽  
pp. 603-622 ◽  
Author(s):  
Leonardo Gutiérrez-Gómez ◽  
Jean-Charles Delvenne
Keyword(s):  
Machine Learning ◽  
Scientific Collaboration ◽  
State Of The Art ◽  
Medical Engineering ◽  
Research Field ◽  
Classification Task ◽  
Collaboration Network ◽  
Structural Patterns ◽  
Art Methods

Abstract Several social, medical, engineering and biological challenges rely on discovering the functionality of networks from their structure and node metadata, when it is available. For example, in chemoinformatics one might want to detect whether a molecule is toxic based on structure and atomic types, or discover the research field of a scientific collaboration network. Existing techniques rely on counting or measuring structural patterns that are known to show large variations from network to network, such as the number of triangles, or the assortativity of node metadata. We introduce the concept of multi-hop assortativity, that captures the similarity of the nodes situated at the extremities of a randomly selected path of a given length. We show that multi-hop assortativity unifies various existing concepts and offers a versatile family of ‘fingerprints’ to characterize networks. These fingerprints allow in turn to recover the functionalities of a network, with the help of the machine learning toolbox. Our method is evaluated empirically on established social and chemoinformatic network benchmarks. Results reveal that our assortativity based features are competitive providing highly accurate results often outperforming state of the art methods for the network classification task.

UBAR

2021 ◽  
Vol 20 (3) ◽  
pp. 1-25
Author(s):  
Elham Shamsa ◽  
Alma Pröbstl ◽  
Nima TaheriNejad ◽  
Anil Kanduri ◽  
Samarjit Chakraborty ◽  
...  
Keyword(s):  
Resource Management ◽  
Quality Of Experience ◽  
State Of The Art ◽  
State Of Charge ◽  
User Preference ◽  
Management Approach ◽  
High Quality ◽  
Trade Off ◽  
Run Time

Smartphone users require high Battery Cycle Life (BCL) and high Quality of Experience (QoE) during their usage. These two objectives can be conflicting based on the user preference at run-time. Finding the best trade-off between QoE and BCL requires an intelligent resource management approach that considers and learns user preference at run-time. Current approaches focus on one of these two objectives and neglect the other, limiting their efficiency in meeting users’ needs. In this article, we present UBAR, User- and Battery-aware Resource management, which considers dynamic workload, user preference, and user plug-in/out pattern at run-time to provide a suitable trade-off between BCL and QoE. UBAR personalizes this trade-off by learning the user’s habits and using that to satisfy QoE, while considering battery temperature and State of Charge (SOC) pattern to maximize BCL. The evaluation results show that UBAR achieves 10% to 40% improvement compared to the existing state-of-the-art approaches.

Sign in / Sign up

Export Citation Format

Share Document