scholarly journals Elastic CRFs for Open-Ontology Slot Filling

2021 ◽  
Vol 11 (22) ◽  
pp. 10675
Author(s):  
Yinpei Dai ◽  
Yichi Zhang ◽  
Hong Liu ◽  
Zhijian Ou ◽  
Yi Huang ◽  
...  
Keyword(s):  
Random Field ◽  
Conditional Random Field ◽  
Dialog Systems ◽  
Cross Domain ◽  
Crucial Component ◽  
Semantic Concepts ◽  
The One ◽  
Task Oriented ◽  
Slot Filling ◽  
Language Description

Slot filling is a crucial component in task-oriented dialog systems that is used to parse (user) utterances into semantic concepts called slots. An ontology is defined by the collection of slots and the values that each slot can take. The most widely used practice of treating slot filling as a sequence labeling task suffers from two main drawbacks. First, the ontology is usually pre-defined and fixed and therefore is not able to detect new labels for unseen slots. Second, the one-hot encoding of slot labels ignores the correlations between slots with similar semantics, which makes it difficult to share knowledge learned across different domains. To address these problems, we propose a new model called elastic conditional random field (eCRF), where each slot is represented by the embedding of its natural language description and modeled by a CRF layer. New slot values can be detected by eCRF whenever a language description is available for the slot. In our experiment, we show that eCRFs outperform existing models in both in-domain and cross-domain tasks, especially in predicting unseen slots and values.

scholarly journals Task-Oriented Dialog Systems That Consider Multiple Appropriate Responses under the Same Context

2020 ◽  
Vol 34 (05) ◽  
pp. 9604-9611
Author(s):  
Yichi Zhang ◽  
Zhijian Ou ◽  
Zhou Yu
Keyword(s):  
Data Augmentation ◽  
State Of The Art ◽  
Task Completion ◽  
Dialog Systems ◽  
State Action ◽  
Response Diversity ◽  
Dialog System ◽  
Additional State ◽  
The One ◽  
Task Oriented

Conversations have an intrinsic one-to-many property, which means that multiple responses can be appropriate for the same dialog context. In task-oriented dialogs, this property leads to different valid dialog policies towards task completion. However, none of the existing task-oriented dialog generation approaches takes this property into account. We propose a Multi-Action Data Augmentation (MADA) framework to utilize the one-to-many property to generate diverse appropriate dialog responses. Specifically, we first use dialog states to summarize the dialog history, and then discover all possible mappings from every dialog state to its different valid system actions. During dialog system training, we enable the current dialog state to map to all valid system actions discovered in the previous process to create additional state-action pairs. By incorporating these additional pairs, the dialog policy learns a balanced action distribution, which further guides the dialog model to generate diverse responses. Experimental results show that the proposed framework consistently improves dialog policy diversity, and results in improved response diversity and appropriateness. Our model obtains state-of-the-art results on MultiWOZ.

scholarly journals Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling

2021 ◽  
Vol 11 (11) ◽  
pp. 4887
Author(s):  
Ting He ◽  
Xiaohong Xu ◽  
Yating Wu ◽  
Huazhen Wang ◽  
Jian Chen
Keyword(s):  
Knowledge Base ◽  
Resource Sharing ◽  
Short Term Memory ◽  
State Of The Art ◽  
Detection System ◽  
Joint Model ◽  
Detection Accuracy ◽  
Dialog Systems ◽  
Task Oriented ◽  
Slot Filling

Intent detection and slot filling are important modules in task-oriented dialog systems. In order to make full use of the relationship between different modules and resource sharing, solving the problem of a lack of semantics, this paper proposes a multitasking learning intent-detection system, based on the knowledge-base and slot-filling joint model. The approach has been used to share information and rich external utility between intent and slot modules in a three-part process. First, this model obtains shared parameters and features between the two modules based on long short-term memory and convolutional neural networks. Second, a knowledge base is introduced into the model to improve its performance. Finally, a weighted-loss function is built to optimize the joint model. Experimental results demonstrate that our model achieves better performance compared with state-of-the-art algorithms on a benchmark Airline Travel Information System (ATIS) dataset and the Snips dataset. Our joint model achieves state-of-the-art results on the benchmark ATIS dataset with a 1.33% intent-detection accuracy improvement, a 0.94% slot filling F value improvement, and with 0.19% and 0.31% improvements respectively on the Snips dataset.

scholarly journals Cross-Domain Slot Filling as Machine Reading Comprehension

2021 ◽  
Author(s):  
Mengshi Yu ◽  
Jian Liu ◽  
Yufeng Chen ◽  
Jinan Xu ◽  
Yujie Zhang
Keyword(s):  
Reading Comprehension ◽  
Large Scale ◽  
Training Data ◽  
Dialogue Systems ◽  
Domain Specific ◽  
Cross Domain ◽  
New Perspective ◽  
Task Oriented ◽  
Slot Filling ◽  
Machine Reading

With task-oriented dialogue systems being widely applied in everyday life, slot filling, the essential component of task-oriented dialogue systems, is required to be quickly adapted to new domains that contain domain-specific slots with few or no training data. Previous methods for slot filling usually adopt sequence labeling framework, which, however, often has limited ability when dealing with the domain-specific slots. In this paper, we take a new perspective on cross-domain slot filling by framing it as a machine reading comprehension (MRC) problem. Our approach firstly transforms slot names into well-designed queries, which contain rich informative prior knowledge and are very helpful for the detection of domain-specific slots. In addition, we utilize the large-scale MRC dataset for pre-training, which further alleviates the data scarcity problem. Experimental results on SNIPS and ATIS datasets show that our approach consistently outperforms the existing state-of-the-art methods by a large margin.

“Opinion Target: Opinion Word” Pairs Extraction Based on CRF

Symmetry ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 251
Author(s):  
Yan Yan ◽  
Faguo Zhou ◽  
Yifan Ge ◽  
Cheng Liu ◽  
Jingwu Feng
Keyword(s):  
Random Field ◽  
Mobile Applications ◽  
Conditional Random Field ◽  
User Reviews ◽  
Visualization System ◽  
Target Extraction

With the spread of mobile applications and online interactive platforms, the number of user reviews are increasing explosively and becoming one of the most important ways for users to voice opinions. Opinion target extraction and opinion word extraction are two key procedures used to determine the helpfulness of reviews. In this paper, we implement a system to extract “opinion target:opinion word” pairs based on the Conditional Random Field (CRF). Firstly, we used the CRF model to extract opinion targets and opinion words, then combined these into pairs in order. In addition, Node.js was used to build a visualization system to display “opinion target:opinion word” pairs. In order to verify the effectiveness of the system, experiments were conducted on the Laptop and Restaurant datasets of SemEval-2014-task4, and the accuracy of the F value extracted by the model reached 86% and 90%, respectively. All the code and datasets for this experiment are available on GitHub.

Sign in / Sign up

Export Citation Format

Share Document