scholarly journals Progressive Open-Domain Response Generation with Multiple Controllable Attributes

2021 ◽  
Author(s):  
Haiqin Yang ◽  
Xiaoyuan Yao ◽  
Yiqun Duan ◽  
Jianping Shen ◽  
Jie Zhong ◽  
...  
Keyword(s):  
Latent Variables ◽  
State Of The Art ◽  
Semantic Features ◽  
Dialogue Systems ◽  
Open Domain ◽  
Specific Variable ◽  
Variational Autoencoder ◽  
Attribute Information ◽  
Global Variable ◽  
The Common

It is desirable to include more controllable attributes to enhance the diversity of generated responses in open-domain dialogue systems. However, existing methods can generate responses with only one controllable attribute or lack a flexible way to generate them with multiple controllable attributes. In this paper, we propose a Progressively trained Hierarchical Encoder-Decoder (PHED) to tackle this task. More specifically, PHED deploys Conditional Variational AutoEncoder (CVAE) on Transformer to include one aspect of attributes at one stage. A vital characteristic of the CVAE is to separate the latent variables at each stage into two types: a global variable capturing the common semantic features and a specific variable absorbing the attribute information at that stage. PHED then couples the CVAE latent variables with the Transformer encoder and is trained by minimizing a newly derived ELBO and controlled losses to produce the next stage's input and produce responses as required. Finally, we conduct extensive evaluations to show that PHED significantly outperforms the state-of-the-art neural generation models and produces more diverse responses as expected.

scholarly journals Chinese Medical Question Answer Matching based on Interactive Sentence Representation Learning

2020 ◽  
Author(s):  
Xiongtao Cui ◽  
Jungang Han
Keyword(s):  
Semantic Information ◽  
State Of The Art ◽  
Representation Learning ◽  
Semantic Features ◽  
Learning Method ◽  
Open Domain ◽  
Semantic Association ◽  
Performance Deficits ◽  
Multi Scale ◽  
Questions And Answers

Chinese medical question-answer matching is more challenging than the open-domain questionanswer matching in English. Even though the deep learning method has performed well in improving the performance of question-answer matching, these methods only focus on the semantic information inside sentences, while ignoring the semantic association between questions and answers, thus resulting in performance deficits. In this paper, we design a series of interactive sentence representation learning models to tackle this problem. To better adapt to Chinese medical question-answer matching and take the advantages of different neural network structures, we propose the Crossed BERT network to extract the deep semantic information inside the sentence and the semantic association between question and answer, and then combine with the multi-scale CNNs network or BiGRU network to take the advantage of different structure of neural networks to learn more semantic features into the sentence representation. The experiments on the cMedQA V2.0 and cMedQA V1.0 dataset show that our model significantly outperforms all the existing state-of-the-art models of Chinese medical question answer matching.

scholarly journals Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness

2021 ◽  
Author(s):  
Dazhong Shen ◽  
Chuan Qin ◽  
Chao Wang ◽  
Hengshu Zhu ◽  
Enhong Chen ◽  
...  
Keyword(s):  
Latent Variables ◽  
State Of The Art ◽  
Likelihood Estimation ◽  
Generative Models ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Benchmark Datasets ◽  
Classification Tasks ◽  
Latent Representations ◽  
Low Uncertainty

As one of the most popular generative models, Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. However, when the decoder network is sufficiently expressive, VAE may lead to posterior collapse; that is, uninformative latent representations may be learned. To this end, in this paper, we propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space, and thus the representation can be learned in a meaningful and compact manner. Specifically, we first theoretically demonstrate that it will result in better latent space with high diversity and low uncertainty awareness by controlling the distribution of posterior’s parameters across the whole data accordingly. Then, without the introduction of new loss terms or modifying training strategies, we propose to exploit Dropout on the variances and Batch-Normalization on the means simultaneously to regularize their distributions implicitly. Furthermore, to evaluate the generalization effect, we also exploit DU-VAE for inverse autoregressive flow based-VAE (VAE-IAF) empirically. Finally, extensive experiments on three benchmark datasets clearly show that our approach can outperform state-of-the-art baselines on both likelihood estimation and underlying classification tasks.

scholarly journals TransET: Knowledge Graph Embedding with Entity Types

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1407
Author(s):  
Peng Wang ◽  
Jing Zhou ◽  
Yuzhang Liu ◽  
Xingchen Zhou
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Score Function ◽  
Graph Embedding ◽  
Vector Spaces ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Knowledge Graphs ◽  
Real World Datasets ◽  
Low Dimensional

Knowledge graph embedding aims to embed entities and relations into low-dimensional vector spaces. Most existing methods only focus on triple facts in knowledge graphs. In addition, models based on translation or distance measurement cannot fully represent complex relations. As well-constructed prior knowledge, entity types can be employed to learn the representations of entities and relations. In this paper, we propose a novel knowledge graph embedding model named TransET, which takes advantage of entity types to learn more semantic features. More specifically, circle convolution based on the embeddings of entity and entity types is utilized to map head entity and tail entity to type-specific representations, then translation-based score function is used to learn the presentation triples. We evaluated our model on real-world datasets with two benchmark tasks of link prediction and triple classification. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

Caged-electron States and Split-electron States in Endohedral Alkali C60

2021 ◽  
Author(s):  
yifan yang ◽  
Lorenz S Cederbaum
Keyword(s):  
State Of The Art ◽  
Electronic States ◽  
Electron States ◽  
The Common ◽  
High Level

The low-lying electronic states of neutral X@C60(X=Li, Na, K, Rb) have been computed and analyzed by employing state-of-the-art high level many-electron methods. Apart from the common charge-separated states, well known...

scholarly journals DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6780
Author(s):  
Zhitong Lai ◽  
Rui Tian ◽  
Zhiguo Wu ◽  
Nannan Ding ◽  
Linjian Sun ◽  
...  
Keyword(s):  
Multiple Scales ◽  
Feature Fusion ◽  
State Of The Art ◽  
Depth Estimation ◽  
Multi Scale ◽  
Pyramid Structure ◽  
Benchmark Datasets ◽  
The Common ◽  
Monocular Depth ◽  
Multiple Stages

Pyramid architecture is a useful strategy to fuse multi-scale features in deep monocular depth estimation approaches. However, most pyramid networks fuse features only within the adjacent stages in a pyramid structure. To take full advantage of the pyramid structure, inspired by the success of DenseNet, this paper presents DCPNet, a densely connected pyramid network that fuses multi-scale features from multiple stages of the pyramid structure. DCPNet not only performs feature fusion between the adjacent stages, but also non-adjacent stages. To fuse these features, we design a simple and effective dense connection module (DCM). In addition, we offer a new consideration of the common upscale operation in our approach. We believe DCPNet offers a more efficient way to fuse features from multiple scales in a pyramid-like network. We perform extensive experiments using both outdoor and indoor benchmark datasets (i.e., the KITTI and the NYU Depth V2 datasets) and DCPNet achieves the state-of-the-art results.

scholarly journals Analyze COVID-19 CT images based on evolutionary algorithm with dynamic searching space

2021 ◽  
Author(s):  
Yunhong Gong ◽  
Yanan Sun ◽  
Dezhong Peng ◽  
Peng Chen ◽  
Zhongtai Yan ◽  
...  
Keyword(s):  
Common Sense ◽  
Expert Knowledge ◽  
State Of The Art ◽  
Data Set ◽  
The Past ◽  
Computed Tomography Images ◽  
Intensive Use ◽  
Hospital Resources ◽  
The Common ◽  
Ct Data

AbstractThe COVID-19 pandemic has caused a global alarm. With the advances in artificial intelligence, the COVID-19 testing capabilities have been greatly expanded, and hospital resources are significantly alleviated. Over the past years, computer vision researches have focused on convolutional neural networks (CNNs), which can significantly improve image analysis ability. However, CNN architectures are usually manually designed with rich expertise that is scarce in practice. Evolutionary algorithms (EAs) can automatically search for the proper CNN architectures and voluntarily optimize the related hyperparameters. The networks searched by EAs can be used to effectively process COVID-19 computed tomography images without expert knowledge and manual setup. In this paper, we propose a novel EA-based algorithm with a dynamic searching space to design the optimal CNN architectures for diagnosing COVID-19 before the pathogenic test. The experiments are performed on the COVID-CT data set against a series of state-of-the-art CNN models. The experiments demonstrate that the architecture searched by the proposed EA-based algorithm achieves the best performance yet without any preprocessing operations. Furthermore, we found through experimentation that the intensive use of batch normalization may deteriorate the performance. This contrasts with the common sense approach of manually designing CNN architectures and will help the related experts in handcrafting CNN models to achieve the best performance without any preprocessing operations

scholarly journals Trajectory-User Linking via Variational AutoEncoder

2018 ◽  
Author(s):  
Fan Zhou ◽  
Qiang Gao ◽  
Goce Trajcevski ◽  
Kunpeng Zhang ◽  
Ting Zhong ◽  
...  
Keyword(s):  
Latent Variables ◽  
Large Scale ◽  
Human Mobility ◽  
Hierarchical Structures ◽  
High Dimensional ◽  
Generation Process ◽  
Mobility Patterns ◽  
Variational Autoencoder ◽  
Hidden States ◽  
Structural Semantics

Trajectory-User Linking (TUL) is an essential task in Geo-tagged social media (GTSM) applications, enabling personalized Point of Interest (POI) recommendation and activity identification. Existing works on mining mobility patterns often model trajectories using Markov Chains (MC) or recurrent neural networks (RNN) -- either assuming independence between non-adjacent locations or following a shallow generation process. However, most of them ignore the fact that human trajectories are often sparse, high-dimensional and may contain embedded hierarchical structures. We tackle the TUL problem with a semi-supervised learning framework, called TULVAE (TUL via Variational AutoEncoder), which learns the human mobility in a neural generative architecture with stochastic latent variables that span hidden states in RNN. TULVAE alleviates the data sparsity problem by leveraging large-scale unlabeled data and represents the hierarchical and structural semantics of trajectories with high-dimensional latent variables. Our experiments demonstrate that TULVAE improves efficiency and linking performance in real GTSM datasets, in comparison to existing methods.

scholarly journals Relevance-guided Supervision for OpenQA with ColBERT

2021 ◽  
Vol 9 ◽  
pp. 929-944
Author(s):  
Omar Khattab ◽  
Christopher Potts ◽  
Matei Zaharia
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Training Data ◽  
Coarse Grained ◽  
Retrieval Model ◽  
Open Domain ◽  
Weak Supervision ◽  
Fine Grained ◽  
Vector Representations ◽  
Large Corpus

Abstract Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions between questions and passages. We propose an efficient weak supervision strategy that iteratively uses ColBERT to create its own training data. This greatly improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA, and the resulting system attains state-of-the-art extractive OpenQA performance on all three datasets.

Sign in / Sign up

Export Citation Format

Share Document