scholarly journals Sequential Convolutional Neural Networks for Slot Filling in Spoken Language Understanding

Author(s):  
Ngoc Thang Vu
Author(s):  
Gregoire Mesnil ◽  
Yann Dauphin ◽  
Kaisheng Yao ◽  
Yoshua Bengio ◽  
Li Deng ◽  
...  

2020 ◽  
Vol 34 (05) ◽  
pp. 9539-9546
Author(s):  
Linhao Zhang ◽  
Dehong Ma ◽  
Xiaodong Zhang ◽  
Xiaohui Yan ◽  
Houfeng Wang

Much research in recent years has focused on spoken language understanding (SLU), which usually involves two tasks: intent detection and slot filling. Since Yao et al.(2013), almost all SLU systems are RNN-based, which have been shown to suffer various limitations due to their sequential nature. In this paper, we propose to tackle this task with Graph LSTM, which first converts text into a graph and then utilizes the message passing mechanism to learn the node representation. Not only the Graph LSTM addresses the limitations of sequential models, but it can also help to utilize the semantic correlation between slot and intent. We further propose a context-gated mechanism to make better use of context information for slot filling. Our extensive evaluation shows that the proposed model outperforms the state-of-the-art results by a large margin.


Symmetry ◽  
2020 ◽  
Vol 12 (6) ◽  
pp. 993
Author(s):  
Zhen Zhang ◽  
Hao Huang ◽  
Kai Wang

Modeling the context of a target word is of fundamental importance in predicting the semantic label for slot filling task in Spoken Language Understanding (SLU). Although Recurrent Neural Network (RNN) has shown to successfully achieve the state-of-the-art results for SLU, and Bidirectional RNN is capable of obtaining further improvement by modeling information not only from the past, but also from the future, they only consider limited contextual information of the target word. In order to make the network deeper and hence obtain longer contextual information, we propose to use a multi-layer Time Delay Neural Network (TDNN), which is prevalent in current large vocabulary continuous speech recognition tasks. In particular, we use a TDNN with symmetric time delay offset. To make the stacked TDNN easily trained, residual structures and skip concatenation are adopted. In addition, we further improve the model by introducing ResTDNN-BiLSTM, which combines the advantages of both the residual TDNN and BiLSTM. Experiments on slot filling tasks on the Air Travel Information System (ATIS) and Snips benchmark datasets show the proposed SC-TDNN-C achieves state-of-the-art results without any additional knowledge and data resources. Finally, we review and compare slot filling results by using a variety of existing models and methods.


Sign in / Sign up

Export Citation Format

Share Document