A New Pre-Training Method for Training Deep Learning Models with Application to Spoken Language Understanding

2016 ◽  
Author(s):  
Asli Celikyilmaz ◽  
Ruhi Sarikaya ◽  
Dilek Hakkani-Tür ◽  
Xiaohu Liu ◽  
Nikhil Ramesh ◽  
...  
Keyword(s):  
Deep Learning ◽  
Spoken Language ◽  
Training Method ◽  
Learning Models ◽  
Language Understanding ◽  
Spoken Language Understanding

scholarly journals Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents

2019 ◽  
Vol 33 ◽  
pp. 4959-4966 ◽  
Author(s):  
Aditya Siddhant ◽  
Anuj Goyal ◽  
Angeliki Metallinou
Keyword(s):  
Intelligent Agents ◽  
Language Model ◽  
Model Performance ◽  
User Interaction ◽  
Spoken Language ◽  
Training Method ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Improve Model ◽  
Improve Model Performance

User interaction with voice-powered agents generates large amounts of unlabeled utterances. In this paper, we explore techniques to efficiently transfer the knowledge from these unlabeled utterances to improve model performance on Spoken Language Understanding (SLU) tasks. We use Embeddings from Language Model (ELMo) to take advantage of unlabeled data by learning contextualized word representations. Additionally, we propose ELMo-Light (ELMoL), a faster and simpler unsupervised pre-training method for SLU. Our findings suggest unsupervised pre-training on a large corpora of unlabeled utterances leads to significantly better SLU performance compared to training from scratch and it can even outperform conventional supervised transfer. Additionally, we show that the gains from unsupervised transfer techniques can be further improved by supervised transfer. The improvements are more pronounced in low resource settings and when using only 1000 labeled in-domain samples, our techniques match the performance of training from scratch on 10-15x more labeled in-domain data.

scholarly journals Research on Spoken Language Understanding Based on Deep Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Hui Yanli
Keyword(s):  
Deep Learning ◽  
Spoken Language ◽  
Recognition Algorithm ◽  
Practical Application ◽  
Language Understanding ◽  
Spoken Language Understanding ◽  
Simulation Results ◽  
Gate Unit ◽  
Slot Filling ◽  
Cyclic Network

Aiming at solving the problem that the recognition effect of rare slot values in spoken language is poor, which affects the accuracy of oral understanding task, a spoken language understanding method is designed based on deep learning. The local features of semantic text are extracted and classified to make the classification results match the dialogue task. An intention recognition algorithm is designed for the classification results. Each datum has a corresponding intention label to complete the task of semantic slot filling. The attention mechanism is applied to the recognition of rare slot value information, the weight of hidden state and corresponding slot characteristics are obtained, and the updated slot value is used to represent the tracking state. An auxiliary gate unit is constructed between the upper and lower slots of historical dialogue, and the word vector is trained based on deep learning to complete the task of spoken language understanding. The simulation results show that the proposed method can realize multiple rounds of man-machine spoken language. Compared with the spoken language understanding methods based on cyclic network, context information, and label decomposition, it has higher accuracy and F1 value and has higher practical application value.

Integrating Syntax and Semantics into Spoken Language Understanding

1991 ◽  
Author(s):  
Lynette Hirschman ◽  
Stephanie Seneff ◽  
David Goodine ◽  
Michael Phillips
Keyword(s):  
Spoken Language ◽  
Language Understanding ◽  
Spoken Language Understanding

Spoken Language Understanding using Pretraining Technique

2020 ◽  
Author(s):  
Saad Ghojaria ◽  
Rahul Kotian ◽  
Yash Sawant ◽  
Suresh Mestry
Keyword(s):  
Spoken Language ◽  
Language Understanding ◽  
Spoken Language Understanding

scholarly journals RNN Based Incremental Online Spoken Language Understanding

2021 ◽  
Author(s):  
Prashanth Gurunath Shivakumar ◽  
Naveen Kumar ◽  
Panayiotis Georgiou ◽  
Shrikanth Narayanan
Keyword(s):  
Spoken Language ◽  
Language Understanding ◽  
Spoken Language Understanding
Sign in / Sign up

Export Citation Format

Share Document