scholarly journals DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue

2020 ◽  
Author(s):  
Xiaoze Jiang ◽  
Jing Yu ◽  
Yajing Sun ◽  
Zengchang Qin ◽  
Zihao Zhu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
The Novel ◽  
High Quality ◽  
Response Level ◽  
Information Selection ◽  
Word Generation ◽  
Adaptive Combination ◽  
Word Level ◽  
Selection Mode

Visual Dialogue task requires an agent to be engaged in a conversation with human about an image. The ability of generating detailed and non-repetitive responses is crucial for the agent to achieve human-like conversation. In this paper, we propose a novel generative decoding architecture to generate high-quality responses, which moves away from decoding the whole encoded semantics towards the design that advocates both transparency and flexibility. In this architecture, word generation is decomposed into a series of attention-based information selection steps, performed by the novel recurrent Deliberation, Abandon and Memory (DAM) module. Each DAM module performs an adaptive combination of the response-level semantics captured from the encoder and the word-level semantics specifically selected for generating each word. Therefore, the responses contain more detailed and non-repetitive descriptions while maintaining the semantic accuracy. Furthermore, DAM is flexible to cooperate with existing visual dialogue encoders and adaptive to the encoder structures by constraining the information selection mode in DAM. We apply DAM to three typical encoders and verify the performance on the VisDial v1.0 dataset. Experimental results show that the proposed models achieve new state-of-the-art performance with high-quality responses. The code is available at https://github.com/JXZe/DAM.

Language-Independent Type Inference of the Instances from Multilingual Wikipedia

2019 ◽  
Vol 15 (2) ◽  
pp. 22-46 ◽  
Author(s):  
Tianxing Wu ◽  
Guilin Qi ◽  
Bin Luo ◽  
Lei Zhang ◽  
Haofen Wang
Keyword(s):  
Knowledge Base ◽  
State Of The Art ◽  
The State ◽  
Type Inference ◽  
Experimental Results ◽  
High Quality ◽  
New Approach ◽  
Semantic Associations ◽  
Quality Type ◽  
Type Information

Extracting knowledge from Wikipedia has attracted much attention in recent ten years. One of the most valuable kinds of knowledge is type information, which refers to the axioms stating that an instance is of a certain type. Current approaches for inferring the types of instances from Wikipedia mainly rely on some language-specific rules. Since these rules cannot catch the semantic associations between instances and classes (i.e. candidate types), it may lead to mistakes and omissions in the process of type inference. The authors propose a new approach leveraging attributes to perform language-independent type inference of the instances from Wikipedia. The proposed approach is applied to the whole English and Chinese Wikipedia, which results in the first version of MulType (Multilingual Type Information), a knowledge base describing the types of instances from multilingual Wikipedia. Experimental results show that not only the proposed approach outperforms the state-of-the-art comparison methods, but also MulType contains lots of new and high-quality type information.

scholarly journals Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 6722-6729 ◽  
Author(s):  
Ziming Li ◽  
Julia Kiseleva ◽  
Maarten De Rijke
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
The State ◽  
Experimental Results ◽  
Imitation Learning ◽  
Local Optimum ◽  
Inverse Reinforcement Learning ◽  
High Quality ◽  
Overall Performance

The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.

scholarly journals MetaCoAG: Binning Metagenomic Contigs via Composition, Coverage and Assembly Graphs

2021 ◽  
Author(s):  
Vijini Mallawaarachchi ◽  
Yu Lin
Keyword(s):  
Microbial Communities ◽  
De Novo ◽  
State Of The Art ◽  
Genetic Material ◽  
Single Copy ◽  
Experimental Results ◽  
Marker Genes ◽  
High Quality ◽  
Second Best ◽  
Direct Use

ABSTRACTMetagenomics binning has allowed us to study and characterize various genetic material of different species and gain insights into microbial communities. While existing binning tools bin metagenomics de novo assemblies, they do not make use of the assembly graphs that produce such assemblies. Here we propose MetaCoAG, a tool that utilizes assembly graphs with the composition and coverage information to bin metagenomic contigs. MetaCoAG uses single-copy marker genes to estimate the number of initial bins, assigns contigs into bins iteratively and adjusts the number of bins dynamically throughout the binning process. Experimental results on simulated and real datasets demonstrate that MetaCoAG significantly outperforms state-of-the-art binning tools, producing more high-quality bins than the second-best tool, with an average median F1-score of 88.40%. To the best of our knowledge, MetaCoAG is the first stand-alone binning tool to make direct use of the assembly graph information. MetaCoAG is available at https://github.com/Vini2/MetaCoAG.

Language-Independent Type Inference of the Instances from Multilingual Wikipedia

2022 ◽  
pp. 580-606
Author(s):  
Tianxing Wu ◽  
Guilin Qi ◽  
Bin Luo ◽  
Lei Zhang ◽  
Haofen Wang
Keyword(s):  
Knowledge Base ◽  
State Of The Art ◽  
The State ◽  
Type Inference ◽  
Experimental Results ◽  
High Quality ◽  
New Approach ◽  
Semantic Associations ◽  
Quality Type ◽  
Type Information

Extracting knowledge from Wikipedia has attracted much attention in recent ten years. One of the most valuable kinds of knowledge is type information, which refers to the axioms stating that an instance is of a certain type. Current approaches for inferring the types of instances from Wikipedia mainly rely on some language-specific rules. Since these rules cannot catch the semantic associations between instances and classes (i.e. candidate types), it may lead to mistakes and omissions in the process of type inference. The authors propose a new approach leveraging attributes to perform language-independent type inference of the instances from Wikipedia. The proposed approach is applied to the whole English and Chinese Wikipedia, which results in the first version of MulType (Multilingual Type Information), a knowledge base describing the types of instances from multilingual Wikipedia. Experimental results show that not only the proposed approach outperforms the state-of-the-art comparison methods, but also MulType contains lots of new and high-quality type information.

scholarly journals Deep Video Frame Interpolation Using Cyclic Frame Generation

2019 ◽  
Vol 33 ◽  
pp. 8794-8802 ◽  
Author(s):  
Yu-Lun Liu ◽  
Yi-Tung Liao ◽  
Yen-Yu Lin ◽  
Yung-Yu Chuang
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
Training Data ◽  
Video Frame ◽  
High Quality ◽  
Edge Information ◽  
Frame Interpolation ◽  
Source Codes ◽  
Qualitative And Quantitative ◽  
Loss Term

Video frame interpolation algorithms predict intermediate frames to produce videos with higher frame rates and smooth view transitions given two consecutive frames as inputs. We propose that: synthesized frames are more reliable if they can be used to reconstruct the input frames with high quality. Based on this idea, we introduce a new loss term, the cycle consistency loss. The cycle consistency loss can better utilize the training data to not only enhance the interpolation results, but also maintain the performance better with less training data. It can be integrated into any frame interpolation network and trained in an end-to-end manner. In addition to the cycle consistency loss, we propose two extensions: motion linearity loss and edge-guided training. The motion linearity loss approximates the motion between two input frames to be linear and regularizes the training. By applying edge-guided training, we further improve results by integrating edge information into training. Both qualitative and quantitative experiments demonstrate that our model outperforms the state-of-the-art methods. The source codes of the proposed method and more experimental results will be available at https://github.com/alex04072000/CyclicGen.

scholarly journals Monolingual Transfer Learning via Bilingual Translators for Style-Sensitive Paraphrase Generation

2020 ◽  
Vol 34 (05) ◽  
pp. 8042-8049
Author(s):  
Tomoyuki Kajiwara ◽  
Biwa Miura ◽  
Yuki Arase
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
State Of The Art ◽  
Experimental Results ◽  
Fine Tuning ◽  
Training Methods ◽  
High Quality ◽  
Style Transfer ◽  
Parallel Corpus ◽  
Paraphrase Generation

We tackle the low-resource problem in style transfer by employing transfer learning that utilizes abundantly available raw corpora. Our method consists of two steps: pre-training learns to generate a semantically equivalent sentence with an input assured grammaticality, and fine-tuning learns to add a desired style. Pre-training has two options, auto-encoding and machine translation based methods. Pre-training based on AutoEncoder is a simple way to learn these from a raw corpus. If machine translators are available, the model can learn more diverse paraphrasing via roundtrip translation. After these, fine-tuning achieves high-quality paraphrase generation even in situations where only 1k sentence pairs of the parallel corpus for style transfer is available. Experimental results of formality style transfer indicated the effectiveness of both pre-training methods and the method based on roundtrip translation achieves state-of-the-art performance.

scholarly journals A Plant Propagation Algorithm for Constrained Engineering Optimisation Problems

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Muhammad Sulaiman ◽  
Abdellah Salhi ◽  
Birsen Irem Selamoglu ◽  
Omar Bahaaldin Kirikchi
Keyword(s):  
State Of The Art ◽  
The Novel ◽  
High Quality ◽  
Plant Propagation ◽  
Propagation Algorithm ◽  
Highly Nonlinear ◽  
Continuous Problems ◽  
Comparative Results ◽  
Optimisation Algorithms ◽  
Selection Of

Optimisation problems arising in industry are some of the hardest, often because of the tight specifications of the products involved. They are almost invariably constrained and they involve highly nonlinear, and non-convex functions both in the objective and in the constraints. It is also often the case that the solutions required must be of high quality and obtained in realistic times. Although there are already a number of well performing optimisation algorithms for such problems, here we consider the novel Plant Propagation Algorithm (PPA) which on continuous problems seems to be very competitive. It is presented in a modified form to handle a selection of problems of interest. Comparative results obtained with PPA and state-of-the-art optimisation algorithms of the Nature-inspired type are presented and discussed. On this selection of problems, PPA is found to be as good as and in some cases superior to these algorithms.

scholarly journals A Foreground-Aware Framework for Local Face Attribute Transfer

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 615
Author(s):  
Yuanbin Fu ◽  
Jiayi Ma ◽  
Xiaojie Guo
Keyword(s):  
Social Media ◽  
State Of The Art ◽  
Input Image ◽  
Experimental Results ◽  
Reference Image ◽  
Background Region ◽  
High Quality ◽  
Style Transfer ◽  
Internet Users ◽  
Foreground Region

In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there are many algorithms automatically and globally transferring the style from one image to another, they fail to respect the semantics of the scene and are unable to allow users to merely transfer the attributes of one or two face organs in the foreground region leaving the background region unchanged. To overcome this problem, we developed a novel framework for semantically meaningful local face attribute transfer, which can flexibly transfer the local attribute of a face organ from the reference image to a semantically equivalent organ in the input image, while preserving the background. Our method involves warping the reference photo to match the shape, pose, location, and expression of the input image. The fusion of the warped reference image and input image is then taken as the initialized image for a neural style transfer algorithm. Our method achieves better performance in terms of inception score (3.81) and Fréchet inception distance (80.31), which is about 10% higher than those of competitors, indicating that our framework is capable of producing high-quality and photorealistic attribute transfer results. Both theoretical findings and experimental results are provided to demonstrate the efficacy of the proposed framework, reveal its superiority over other state-of-the-art alternatives.

scholarly journals Modality-aware Style Adaptation for RGB-Infrared Person Re-Identification

2021 ◽  
Author(s):  
Ziling Miao ◽  
Hong Liu ◽  
Wei Shi ◽  
Wanlu Xu ◽  
Hanrong Ye
Keyword(s):  
Collaborative Learning ◽  
Learning Process ◽  
State Of The Art ◽  
Image Space ◽  
Experimental Results ◽  
Image Generation ◽  
High Similarity ◽  
High Quality ◽  
Computation Cost ◽  
Generation Structure

RGB-infrared (IR) person re-identification is a challenging task due to the large modality gap between RGB and IR images. Many existing methods bridge the modality gap by style conversion, requiring high-similarity images exchanged by complex CNN structures, like GAN. In this paper, we propose a highly compact modality-aware style adaptation (MSA) framework, which aims to explore more potential relations between RGB and IR modalities by introducing new related modalities. Therefore, the attention is shifted from bridging to filling the modality gap with no requirement on high-quality generated images. To this end, we firstly propose a concise feature-free image generation structure to adapt the original modalities to two new styles that are compatible with both inputs by patch-based pixel redistribution. Secondly, we devise two image style quantification metrics to discriminate styles in image space using luminance and contrast. Thirdly, we design two image-level losses based on the quantified results to guide the style adaptation during an end-to-end four-modality collaborative learning process. Experimental results on two datasets SYSU-MM01 and RegDB show that MSA achieves significant improvements with little extra computation cost and outperforms the state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document