A short text conversation generation model combining BERT and context attention mechanism

Author(s):  
Huan Zhao ◽  
Jian Lu ◽  
Jie Cao
Author(s):  
Jun Gao ◽  
Wei Bi ◽  
Xiaojiang Liu ◽  
Junhui Li ◽  
Shuming Shi

Neural generative models have become popular and achieved promising performance on short-text conversation tasks. They are generally trained to build a 1-to-1 mapping from the input post to its output response. However, a given post is often associated with multiple replies simultaneously in real applications. Previous research on this task mainly focuses on improving the relevance and informativeness of the top one generated response for each post. Very few works study generating multiple accurate and diverse responses for the same post. In this paper, we propose a novel response generation model, which considers a set of responses jointly and generates multiple diverse responses simultaneously. A reinforcement learning algorithm is designed to solve our model. Experiments on two short-text conversation tasks validate that the multiple responses generated by our model obtain higher quality and larger diversity compared with various state-ofthe-art generative models.


Author(s):  
Hao Zhou ◽  
Tom Young ◽  
Minlie Huang ◽  
Haizhou Zhao ◽  
Jingfang Xu ◽  
...  

Commonsense knowledge is vital to many natural language processing tasks. In this paper, we present a novel open-domain conversation generation model to demonstrate how large-scale commonsense knowledge can facilitate language understanding and generation. Given a user post, the model retrieves relevant knowledge graphs from a knowledge base and then encodes the graphs with a static graph attention mechanism, which augments the semantic information of the post and thus supports better understanding of the post. Then, during word generation, the model attentively reads the retrieved knowledge graphs and the knowledge triples within each graph to facilitate better generation through a dynamic graph attention mechanism. This is the first attempt that uses large-scale commonsense knowledge in conversation generation. Furthermore, unlike existing models that use knowledge triples (entities) separately and independently, our model treats each knowledge graph as a whole, which encodes more structured, connected semantic information in the graphs. Experiments show that the proposed model can generate more appropriate and informative responses than state-of-the-art baselines. 


2020 ◽  
Vol 1693 ◽  
pp. 012067
Author(s):  
Ying Zuo ◽  
Lifen Jiang ◽  
Huazhi Sun ◽  
Chunmei Ma ◽  
Yan Liang ◽  
...  

2020 ◽  
Author(s):  
Kunli Zhang ◽  
Linkun Cai ◽  
Yu Song ◽  
Tao Liu ◽  
Yueshu Zhao

Abstract Background: Data-driven medical health information processing has become a new development trend in obstetrics. Electronic medical records (EMRs) are the basis of evidence-based medicine and important information source for intelligent diagnosis. To obtain diagnostic results, doctors combine clinical experience and medical knowledge in their diagnosis process. External medical knowledge provides strong support for diagnosis. Therefore, how to make full use of EMRs and medical knowledge in intelligent diagnosis is worth studying.Methods: As an EMR usually contains multiple types of diagnostic results, the intelligent diagnosis can be treated as a multi-label classification task. We propose a novel neural network model called Knowledge-aware Hierarchical Diagnosis Model (KHDM) in which Chinese obstetric EMRs and external medical knowledge can be synchronously and effectively used for intelligent diagnostics. In the KHDM, EMRs and external knowledge documents are integrated by the attention mechanism contained in the hierarchical deep learning framework. In this way, we enrich the language model with curated knowledge documents, combining the advantages of both to make a knowledge-aware diagnosis.Results: We evaluate our model on a real-world Chinese obstetric EMR dataset show that KHDM achieves an accuracy of 0.8929, which exceeds that of the most advanced classification benchmark methods.Conclusion: In this paper, an improved model combining medical knowledge and an attention mechanism is proposed, which is based on the problem of the diversity of diagnostic results in Chinese EMRs. KHDM can effectively integrate domain knowledge to greatly improve the accuracy of diagnosis, and we also verify the model's interpretability advantage.


Sign in / Sign up

Export Citation Format

Share Document