A Deep Learning Based Multi-task Ensemble Model for Intent Detection and Slot Filling in Spoken Language Understanding

2018 ◽  
pp. 647-658 ◽  
Author(s):  
Mauajama Firdaus ◽  
Shobhit Bhatnagar ◽  
Asif Ekbal ◽  
Pushpak Bhattacharyya
Keyword(s):  
Deep Learning ◽  
Spoken Language ◽  
Ensemble Model ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Slot Filling

scholarly journals Research on Spoken Language Understanding Based on Deep Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Hui Yanli
Keyword(s):  
Deep Learning ◽  
Spoken Language ◽  
Recognition Algorithm ◽  
Practical Application ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Simulation Results ◽  
Gate Unit ◽  
Slot Filling ◽  
Cyclic Network

Aiming at solving the problem that the recognition effect of rare slot values in spoken language is poor, which affects the accuracy of oral understanding task, a spoken language understanding method is designed based on deep learning. The local features of semantic text are extracted and classified to make the classification results match the dialogue task. An intention recognition algorithm is designed for the classification results. Each datum has a corresponding intention label to complete the task of semantic slot filling. The attention mechanism is applied to the recognition of rare slot value information, the weight of hidden state and corresponding slot characteristics are obtained, and the updated slot value is used to represent the tracking state. An auxiliary gate unit is constructed between the upper and lower slots of historical dialogue, and the word vector is trained based on deep learning to complete the task of spoken language understanding. The simulation results show that the proposed method can realize multiple rounds of man-machine spoken language. Compared with the spoken language understanding methods based on cyclic network, context information, and label decomposition, it has higher accuracy and F1 value and has higher practical application value.

scholarly journals Graph LSTM with Context-Gated Mechanism for Spoken Language Understanding

2020 ◽  
Vol 34 (05) ◽  
pp. 9539-9546
Author(s):  
Linhao Zhang ◽  
Dehong Ma ◽  
Xiaodong Zhang ◽  
Xiaohui Yan ◽  
Houfeng Wang
Keyword(s):  
Message Passing ◽  
Spoken Language ◽  
Language Understanding ◽  
Semantic Correlation ◽  
Spoken Language Understanding ◽  
Extensive Evaluation ◽  
Proposed Model ◽  
Sequential Models ◽  
Almost All ◽  
Slot Filling

Much research in recent years has focused on spoken language understanding (SLU), which usually involves two tasks: intent detection and slot filling. Since Yao et al.(2013), almost all SLU systems are RNN-based, which have been shown to suffer various limitations due to their sequential nature. In this paper, we propose to tackle this task with Graph LSTM, which first converts text into a graph and then utilizes the message passing mechanism to learn the node representation. Not only the Graph LSTM addresses the limitations of sequential models, but it can also help to utilize the semantic correlation between slot and intent. We further propose a context-gated mechanism to make better use of context information for slot filling. Our extensive evaluation shows that the proposed model outperforms the state-of-the-art results by a large margin.

scholarly journals Using Deep Time Delay Neural Network for Slot Filling in Spoken Language Understanding

Symmetry ◽  
2020 ◽  
Vol 12 (6) ◽  
pp. 993
Author(s):  
Zhen Zhang ◽  
Hao Huang ◽  
Kai Wang
Keyword(s):  
Neural Network ◽  
Time Delay ◽  
Target Word ◽  
State Of The Art ◽  
Contextual Information ◽  
Spoken Language ◽  
Continuous Speech Recognition ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Slot Filling

Modeling the context of a target word is of fundamental importance in predicting the semantic label for slot filling task in Spoken Language Understanding (SLU). Although Recurrent Neural Network (RNN) has shown to successfully achieve the state-of-the-art results for SLU, and Bidirectional RNN is capable of obtaining further improvement by modeling information not only from the past, but also from the future, they only consider limited contextual information of the target word. In order to make the network deeper and hence obtain longer contextual information, we propose to use a multi-layer Time Delay Neural Network (TDNN), which is prevalent in current large vocabulary continuous speech recognition tasks. In particular, we use a TDNN with symmetric time delay offset. To make the stacked TDNN easily trained, residual structures and skip concatenation are adopted. In addition, we further improve the model by introducing ResTDNN-BiLSTM, which combines the advantages of both the residual TDNN and BiLSTM. Experiments on slot filling tasks on the Air Travel Information System (ATIS) and Snips benchmark datasets show the proposed SC-TDNN-C achieves state-of-the-art results without any additional knowledge and data resources. Finally, we review and compare slot filling results by using a variety of existing models and methods.

Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding

2015 ◽  
Vol 23 (3) ◽  
pp. 530-539 ◽  
Author(s):  
Gregoire Mesnil ◽  
Yann Dauphin ◽  
Kaisheng Yao ◽  
Yoshua Bengio ◽  
Li Deng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Spoken Language ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Slot Filling
Sign in / Sign up

Export Citation Format

Share Document