scholarly journals Modality-aware Style Adaptation for RGB-Infrared Person Re-Identification

2021 ◽  
Author(s):  
Ziling Miao ◽  
Hong Liu ◽  
Wei Shi ◽  
Wanlu Xu ◽  
Hanrong Ye
Keyword(s):  
Collaborative Learning ◽  
Learning Process ◽  
State Of The Art ◽  
Image Space ◽  
Experimental Results ◽  
Image Generation ◽  
High Similarity ◽  
High Quality ◽  
Computation Cost ◽  
Generation Structure

RGB-infrared (IR) person re-identification is a challenging task due to the large modality gap between RGB and IR images. Many existing methods bridge the modality gap by style conversion, requiring high-similarity images exchanged by complex CNN structures, like GAN. In this paper, we propose a highly compact modality-aware style adaptation (MSA) framework, which aims to explore more potential relations between RGB and IR modalities by introducing new related modalities. Therefore, the attention is shifted from bridging to filling the modality gap with no requirement on high-quality generated images. To this end, we firstly propose a concise feature-free image generation structure to adapt the original modalities to two new styles that are compatible with both inputs by patch-based pixel redistribution. Secondly, we devise two image style quantification metrics to discriminate styles in image space using luminance and contrast. Thirdly, we design two image-level losses based on the quantified results to guide the style adaptation during an end-to-end four-modality collaborative learning process. Experimental results on two datasets SYSU-MM01 and RegDB show that MSA achieves significant improvements with little extra computation cost and outperforms the state-of-the-art methods.

Technical Details and Educational Applications for Virtual Reality Technologies

2021 ◽  
pp. 74-95
Author(s):  
Yongzhi Wang
Keyword(s):  
Higher Education ◽  
Virtual Reality ◽  
Learning Process ◽  
State Of The Art ◽  
The State ◽  
High Quality ◽  
Comprehensive Survey ◽  
Technical Details ◽  
Educational Applications ◽  
Teaching Learning

The application of virtual reality (VR) in higher education has drawn attention. Understanding the state of the art for VR technologies helps educators identify appropriate applications and develop a high-quality engaging teaching-learning process. This chapter provides a comprehensive survey of current hardware and software supports on VR. Secondly, important technical metrics in VR technology are considered with comparisons of different VR devices using identified metrics. Third, there is a focus on software tools and an explore of various development frameworks, which facilitate the implementation of VR applications. With this information as a foundation, there is a VR use in higher education. Finally, there is a discussion of VR applications that can be potentially used in education.

Language-Independent Type Inference of the Instances from Multilingual Wikipedia

2019 ◽  
Vol 15 (2) ◽  
pp. 22-46 ◽  
Author(s):  
Tianxing Wu ◽  
Guilin Qi ◽  
Bin Luo ◽  
Lei Zhang ◽  
Haofen Wang
Keyword(s):  
Knowledge Base ◽  
State Of The Art ◽  
The State ◽  
Type Inference ◽  
Experimental Results ◽  
High Quality ◽  
New Approach ◽  
Semantic Associations ◽  
Quality Type ◽  
Type Information

Extracting knowledge from Wikipedia has attracted much attention in recent ten years. One of the most valuable kinds of knowledge is type information, which refers to the axioms stating that an instance is of a certain type. Current approaches for inferring the types of instances from Wikipedia mainly rely on some language-specific rules. Since these rules cannot catch the semantic associations between instances and classes (i.e. candidate types), it may lead to mistakes and omissions in the process of type inference. The authors propose a new approach leveraging attributes to perform language-independent type inference of the instances from Wikipedia. The proposed approach is applied to the whole English and Chinese Wikipedia, which results in the first version of MulType (Multilingual Type Information), a knowledge base describing the types of instances from multilingual Wikipedia. Experimental results show that not only the proposed approach outperforms the state-of-the-art comparison methods, but also MulType contains lots of new and high-quality type information.

scholarly journals Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 6722-6729 ◽  
Author(s):  
Ziming Li ◽  
Julia Kiseleva ◽  
Maarten De Rijke
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
The State ◽  
Experimental Results ◽  
Imitation Learning ◽  
Local Optimum ◽  
Inverse Reinforcement Learning ◽  
High Quality ◽  
Overall Performance

The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.

scholarly journals MetaCoAG: Binning Metagenomic Contigs via Composition, Coverage and Assembly Graphs

2021 ◽  
Author(s):  
Vijini Mallawaarachchi ◽  
Yu Lin
Keyword(s):  
Microbial Communities ◽  
De Novo ◽  
State Of The Art ◽  
Genetic Material ◽  
Single Copy ◽  
Experimental Results ◽  
Marker Genes ◽  
High Quality ◽  
Second Best ◽  
Direct Use

ABSTRACTMetagenomics binning has allowed us to study and characterize various genetic material of different species and gain insights into microbial communities. While existing binning tools bin metagenomics de novo assemblies, they do not make use of the assembly graphs that produce such assemblies. Here we propose MetaCoAG, a tool that utilizes assembly graphs with the composition and coverage information to bin metagenomic contigs. MetaCoAG uses single-copy marker genes to estimate the number of initial bins, assigns contigs into bins iteratively and adjusts the number of bins dynamically throughout the binning process. Experimental results on simulated and real datasets demonstrate that MetaCoAG significantly outperforms state-of-the-art binning tools, producing more high-quality bins than the second-best tool, with an average median F1-score of 88.40%. To the best of our knowledge, MetaCoAG is the first stand-alone binning tool to make direct use of the assembly graph information. MetaCoAG is available at https://github.com/Vini2/MetaCoAG.

Language-Independent Type Inference of the Instances from Multilingual Wikipedia

2022 ◽  
pp. 580-606
Author(s):  
Tianxing Wu ◽  
Guilin Qi ◽  
Bin Luo ◽  
Lei Zhang ◽  
Haofen Wang
Keyword(s):  
Knowledge Base ◽  
State Of The Art ◽  
The State ◽  
Type Inference ◽  
Experimental Results ◽  
High Quality ◽  
New Approach ◽  
Semantic Associations ◽  
Quality Type ◽  
Type Information

Extracting knowledge from Wikipedia has attracted much attention in recent ten years. One of the most valuable kinds of knowledge is type information, which refers to the axioms stating that an instance is of a certain type. Current approaches for inferring the types of instances from Wikipedia mainly rely on some language-specific rules. Since these rules cannot catch the semantic associations between instances and classes (i.e. candidate types), it may lead to mistakes and omissions in the process of type inference. The authors propose a new approach leveraging attributes to perform language-independent type inference of the instances from Wikipedia. The proposed approach is applied to the whole English and Chinese Wikipedia, which results in the first version of MulType (Multilingual Type Information), a knowledge base describing the types of instances from multilingual Wikipedia. Experimental results show that not only the proposed approach outperforms the state-of-the-art comparison methods, but also MulType contains lots of new and high-quality type information.

scholarly journals Mask-aware photorealistic facial attribute manipulation

2021 ◽  
Author(s):  
Ruoqi Sun ◽  
Chen Huang ◽  
Hengliang Zhu ◽  
Lizhuang Ma
Keyword(s):  
Face Recognition ◽  
Experimental Results ◽  
Global Information ◽  
Image Generation ◽  
Feature Maps ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Variational Autoencoder

AbstractThe technique of facial attribute manipulation has found increasing application, but it remains challenging to restrict editing of attributes so that a face’s unique details are preserved. In this paper, we introduce our method, which we call a mask-adversarial autoencoder (M-AAE). It combines a variational autoencoder (VAE) and a generative adversarial network (GAN) for photorealistic image generation. We use partial dilated layers to modify a few pixels in the feature maps of an encoder, changing the attribute strength continuously without hindering global information. Our training objectives for the VAE and GAN are reinforced by supervision of face recognition loss and cycle consistency loss, to faithfully preserve facial details. Moreover, we generate facial masks to enforce background consistency, which allows our training to focus on the foreground face rather than the background. Experimental results demonstrate that our method can generate high-quality images with varying attributes, and outperforms existing methods in detail preservation.

scholarly journals Deep Video Frame Interpolation Using Cyclic Frame Generation

2019 ◽  
Vol 33 ◽  
pp. 8794-8802 ◽  
Author(s):  
Yu-Lun Liu ◽  
Yi-Tung Liao ◽  
Yen-Yu Lin ◽  
Yung-Yu Chuang
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
Training Data ◽  
Video Frame ◽  
High Quality ◽  
Edge Information ◽  
Frame Interpolation ◽  
Source Codes ◽  
Qualitative And Quantitative ◽  
Loss Term

Video frame interpolation algorithms predict intermediate frames to produce videos with higher frame rates and smooth view transitions given two consecutive frames as inputs. We propose that: synthesized frames are more reliable if they can be used to reconstruct the input frames with high quality. Based on this idea, we introduce a new loss term, the cycle consistency loss. The cycle consistency loss can better utilize the training data to not only enhance the interpolation results, but also maintain the performance better with less training data. It can be integrated into any frame interpolation network and trained in an end-to-end manner. In addition to the cycle consistency loss, we propose two extensions: motion linearity loss and edge-guided training. The motion linearity loss approximates the motion between two input frames to be linear and regularizes the training. By applying edge-guided training, we further improve results by integrating edge information into training. Both qualitative and quantitative experiments demonstrate that our model outperforms the state-of-the-art methods. The source codes of the proposed method and more experimental results will be available at https://github.com/alex04072000/CyclicGen.

scholarly journals Monolingual Transfer Learning via Bilingual Translators for Style-Sensitive Paraphrase Generation

2020 ◽  
Vol 34 (05) ◽  
pp. 8042-8049
Author(s):  
Tomoyuki Kajiwara ◽  
Biwa Miura ◽  
Yuki Arase
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
State Of The Art ◽  
Experimental Results ◽  
Fine Tuning ◽  
Training Methods ◽  
High Quality ◽  
Style Transfer ◽  
Parallel Corpus ◽  
Paraphrase Generation

We tackle the low-resource problem in style transfer by employing transfer learning that utilizes abundantly available raw corpora. Our method consists of two steps: pre-training learns to generate a semantically equivalent sentence with an input assured grammaticality, and fine-tuning learns to add a desired style. Pre-training has two options, auto-encoding and machine translation based methods. Pre-training based on AutoEncoder is a simple way to learn these from a raw corpus. If machine translators are available, the model can learn more diverse paraphrasing via roundtrip translation. After these, fine-tuning achieves high-quality paraphrase generation even in situations where only 1k sentence pairs of the parallel corpus for style transfer is available. Experimental results of formality style transfer indicated the effectiveness of both pre-training methods and the method based on roundtrip translation achieves state-of-the-art performance.

Content-Based Attention Network for Person Image Generation

2020 ◽  
Vol 29 (15) ◽  
pp. 2050250
Author(s):  
Xiongfei Liu ◽  
Bengao Li ◽  
Xin Chen ◽  
Haiyan Zhang ◽  
Shu Zhan
Keyword(s):  
Major Part ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Experimental Results ◽  
Generative Adversarial Networks ◽  
Image Generation ◽  
Attention Network ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Novel Method

This paper proposes a novel method for person image generation with arbitrary target pose. Given a person image and an arbitrary target pose, our proposed model can synthesize images with the same person but different poses. The Generative Adversarial Networks (GANs) are the major part of the proposed model. Different from the traditional GANs, we add attention mechanism to the generator in order to generate realistic-looking images, we also use content reconstruction with a pretrained VGG16 Net to keep the content consistency between generated images and target images. Furthermore, we test our model on DeepFashion and Market-1501 datasets. The experimental results show that the proposed network performs favorably against state-of-the-art methods.

scholarly journals DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue

2020 ◽  
Author(s):  
Xiaoze Jiang ◽  
Jing Yu ◽  
Yajing Sun ◽  
Zengchang Qin ◽  
Zihao Zhu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
The Novel ◽  
High Quality ◽  
Response Level ◽  
Information Selection ◽  
Word Generation ◽  
Adaptive Combination ◽  
Word Level ◽  
Selection Mode

Visual Dialogue task requires an agent to be engaged in a conversation with human about an image. The ability of generating detailed and non-repetitive responses is crucial for the agent to achieve human-like conversation. In this paper, we propose a novel generative decoding architecture to generate high-quality responses, which moves away from decoding the whole encoded semantics towards the design that advocates both transparency and flexibility. In this architecture, word generation is decomposed into a series of attention-based information selection steps, performed by the novel recurrent Deliberation, Abandon and Memory (DAM) module. Each DAM module performs an adaptive combination of the response-level semantics captured from the encoder and the word-level semantics specifically selected for generating each word. Therefore, the responses contain more detailed and non-repetitive descriptions while maintaining the semantic accuracy. Furthermore, DAM is flexible to cooperate with existing visual dialogue encoders and adaptive to the encoder structures by constraining the information selection mode in DAM. We apply DAM to three typical encoders and verify the performance on the VisDial v1.0 dataset. Experimental results show that the proposed models achieve new state-of-the-art performance with high-quality responses. The code is available at https://github.com/JXZe/DAM.

Sign in / Sign up

Export Citation Format

Share Document