scholarly journals Sequential Convolutional Neural Networks for Slot Filling in Spoken Language Understanding

2016 ◽  
Author(s):  
Ngoc Thang Vu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Spoken Language ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Slot Filling

Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding

2015 ◽  
Vol 23 (3) ◽  
pp. 530-539 ◽  
Author(s):  
Gregoire Mesnil ◽  
Yann Dauphin ◽  
Kaisheng Yao ◽  
Yoshua Bengio ◽  
Li Deng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Spoken Language ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Slot Filling

scholarly journals Graph LSTM with Context-Gated Mechanism for Spoken Language Understanding

2020 ◽  
Vol 34 (05) ◽  
pp. 9539-9546
Author(s):  
Linhao Zhang ◽  
Dehong Ma ◽  
Xiaodong Zhang ◽  
Xiaohui Yan ◽  
Houfeng Wang
Keyword(s):  
Message Passing ◽  
Spoken Language ◽  
Language Understanding ◽  
Semantic Correlation ◽  
Spoken Language Understanding ◽  
Extensive Evaluation ◽  
Proposed Model ◽  
Sequential Models ◽  
Almost All ◽  
Slot Filling

Much research in recent years has focused on spoken language understanding (SLU), which usually involves two tasks: intent detection and slot filling. Since Yao et al.(2013), almost all SLU systems are RNN-based, which have been shown to suffer various limitations due to their sequential nature. In this paper, we propose to tackle this task with Graph LSTM, which first converts text into a graph and then utilizes the message passing mechanism to learn the node representation. Not only the Graph LSTM addresses the limitations of sequential models, but it can also help to utilize the semantic correlation between slot and intent. We further propose a context-gated mechanism to make better use of context information for slot filling. Our extensive evaluation shows that the proposed model outperforms the state-of-the-art results by a large margin.

scholarly journals Using Deep Time Delay Neural Network for Slot Filling in Spoken Language Understanding

Symmetry ◽  
2020 ◽  
Vol 12 (6) ◽  
pp. 993
Author(s):  
Zhen Zhang ◽  
Hao Huang ◽  
Kai Wang
Keyword(s):  
Neural Network ◽  
Time Delay ◽  
Target Word ◽  
State Of The Art ◽  
Contextual Information ◽  
Spoken Language ◽  
Continuous Speech Recognition ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Slot Filling

Modeling the context of a target word is of fundamental importance in predicting the semantic label for slot filling task in Spoken Language Understanding (SLU). Although Recurrent Neural Network (RNN) has shown to successfully achieve the state-of-the-art results for SLU, and Bidirectional RNN is capable of obtaining further improvement by modeling information not only from the past, but also from the future, they only consider limited contextual information of the target word. In order to make the network deeper and hence obtain longer contextual information, we propose to use a multi-layer Time Delay Neural Network (TDNN), which is prevalent in current large vocabulary continuous speech recognition tasks. In particular, we use a TDNN with symmetric time delay offset. To make the stacked TDNN easily trained, residual structures and skip concatenation are adopted. In addition, we further improve the model by introducing ResTDNN-BiLSTM, which combines the advantages of both the residual TDNN and BiLSTM. Experiments on slot filling tasks on the Air Travel Information System (ATIS) and Snips benchmark datasets show the proposed SC-TDNN-C achieves state-of-the-art results without any additional knowledge and data resources. Finally, we review and compare slot filling results by using a variety of existing models and methods.

Sign in / Sign up

Export Citation Format

Share Document