Joint intent detection and slot filling with wheel-graph attention networks

Intent detection and slot filling are recognized as two very important tasks in a spoken language understanding (SLU) system. In order to model these two tasks at the same time, many joint models based on deep neural networks have been proposed recently and archived excellent results. In addition, graph neural network has made good achievements in the field of vision. Therefore, we combine these two advantages and propose a new joint model with a wheel-graph attention network (Wheel-GAT), which is able to model interrelated connections directly for single intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges. Intent nodes can provide utterance-level semantic information for slot filling, while slot nodes can also provide local keyword information for intent detection. The two tasks promote each other and carry out end-to-end training at the same time. Experiments show that our proposed approach is superior to multiple baselines on ATIS and SNIPS datasets. Besides, we also demonstrate that using bi-directional encoder representation from transformer (BERT) model further boosts the performance of the SLU task.

Download Full-text

Focus on Interaction: A Novel Dynamic Graph Model for Joint Multiple Intent Detection and Slot Filling

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/523 ◽

2021 ◽

Author(s):

Zeyuan Ding ◽

Zhihao Yang ◽

Hongfei Lin ◽

Jian Wang

Keyword(s):

Error Propagation ◽

State Of The Art ◽

Graph Model ◽

Joint Models ◽

Dynamic Graph ◽

Language Understanding ◽

Spoken Language Understanding ◽

Sentence Level ◽

Novel Method ◽

Slot Filling

Intent detection and slot filling are two main tasks for building a spoken language understanding (SLU) system. Since the two tasks are closely related, the joint models for the two tasks always outperform the pipeline models in SLU. However, most joint models directly incorporate multiple intent information for each token, which introduces intent noise into the sentence semantics, causing a decrease in the performance of the joint model. In this paper, we propose a Dynamic Graph Model (DGM) for joint multiple intent detection and slot filling, in which we adopt a sentence-level intent-slot interactive graph to model the correlation between the intents and slot. Besides, we design a novel method of constructing the graph, which can dynamically update the interactive graph and further alleviate the error propagation. Experimental results on several multi-intent and single-intent datasets show that our model not only achieves the state-of-the-art (SOTA) performance but also boosts the speed by three to six times over the SOTA model.

Download Full-text

Joint Slot Filling and Intent Detection in Spoken Language Understanding by Hybrid CNN-LSTM Model

2020 International Conference on Control, Robotics and Intelligent System ◽

10.1145/3437802.3437822 ◽

2020 ◽

Author(s):

Moath Al Ali ◽

Bassel Zaity ◽

Pavel Drobintsev ◽

Hazem Wannous ◽

Igor Chernoruckiy ◽

...

Keyword(s):

Spoken Language ◽

Language Understanding ◽

Spoken Language Understanding ◽

Slot Filling

Download Full-text

Graph LSTM with Context-Gated Mechanism for Spoken Language Understanding

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6499 ◽

2020 ◽

Vol 34 (05) ◽

pp. 9539-9546

Author(s):

Linhao Zhang ◽

Dehong Ma ◽

Xiaodong Zhang ◽

Xiaohui Yan ◽

Houfeng Wang

Keyword(s):

Message Passing ◽

Spoken Language ◽

Language Understanding ◽

Semantic Correlation ◽

Spoken Language Understanding ◽

Extensive Evaluation ◽

Proposed Model ◽

Sequential Models ◽

Almost All ◽

Slot Filling

Much research in recent years has focused on spoken language understanding (SLU), which usually involves two tasks: intent detection and slot filling. Since Yao et al.(2013), almost all SLU systems are RNN-based, which have been shown to suffer various limitations due to their sequential nature. In this paper, we propose to tackle this task with Graph LSTM, which first converts text into a graph and then utilizes the message passing mechanism to learn the node representation. Not only the Graph LSTM addresses the limitations of sequential models, but it can also help to utilize the semantic correlation between slot and intent. We further propose a context-gated mechanism to make better use of context information for slot filling. Our extensive evaluation shows that the proposed model outperforms the state-of-the-art results by a large margin.

Download Full-text

Sequential Convolutional Neural Networks for Slot Filling in Spoken Language Understanding

10.21437/interspeech.2016-395 ◽

2016 ◽

Cited By ~ 10

Author(s):

Ngoc Thang Vu

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Spoken Language ◽

Language Understanding ◽

Spoken Language Understanding ◽

Slot Filling

Download Full-text

Using Deep Time Delay Neural Network for Slot Filling in Spoken Language Understanding

Symmetry ◽

10.3390/sym12060993 ◽

2020 ◽

Vol 12 (6) ◽

pp. 993

Author(s):

Zhen Zhang ◽

Hao Huang ◽

Kai Wang

Keyword(s):

Neural Network ◽

Time Delay ◽

Target Word ◽

State Of The Art ◽

Contextual Information ◽

Spoken Language ◽

Continuous Speech Recognition ◽

Language Understanding ◽

Spoken Language Understanding ◽

Slot Filling

Modeling the context of a target word is of fundamental importance in predicting the semantic label for slot filling task in Spoken Language Understanding (SLU). Although Recurrent Neural Network (RNN) has shown to successfully achieve the state-of-the-art results for SLU, and Bidirectional RNN is capable of obtaining further improvement by modeling information not only from the past, but also from the future, they only consider limited contextual information of the target word. In order to make the network deeper and hence obtain longer contextual information, we propose to use a multi-layer Time Delay Neural Network (TDNN), which is prevalent in current large vocabulary continuous speech recognition tasks. In particular, we use a TDNN with symmetric time delay offset. To make the stacked TDNN easily trained, residual structures and skip concatenation are adopted. In addition, we further improve the model by introducing ResTDNN-BiLSTM, which combines the advantages of both the residual TDNN and BiLSTM. Experiments on slot filling tasks on the Air Travel Information System (ATIS) and Snips benchmark datasets show the proposed SC-TDNN-C achieves state-of-the-art results without any additional knowledge and data resources. Finally, we review and compare slot filling results by using a variety of existing models and methods.

Download Full-text