scholarly journals A Hybrid Neural Network BERT-Cap Based on Pre-Trained Language Model and Capsule Network for User Intent Classification

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Hai Liu ◽  
Yuanxia Liu ◽  
Leung-Pun Wong ◽  
Lap-Kei Lee ◽  
Tianyong Hao
Keyword(s):  
Neural Network ◽  
Neural Network Model ◽  
Question Answering ◽  
Semantic Information ◽  
Language Model ◽  
Dialogue System ◽  
Hybrid Neural Network ◽  
Question Answering System ◽  
User Intent ◽  
Vital Component

User intent classification is a vital component of a question-answering system or a task-based dialogue system. In order to understand the goals of users’ questions or discourses, the system categorizes user text into a set of pre-defined user intent categories. User questions or discourses are usually short in length and lack sufficient context; thus, it is difficult to extract deep semantic information from these types of text and the accuracy of user intent classification may be affected. To better identify user intents, this paper proposes a BERT-Cap hybrid neural network model with focal loss for user intent classification to capture user intents in dialogue. The model uses multiple transformer encoder blocks to encode user utterances and initializes encoder parameters with a pre-trained BERT. Then, it extracts essential features using a capsule network with dynamic routing after utterances encoding. Experiment results on four publicly available datasets show that our model BERT-Cap achieves a F1 score of 0.967 and an accuracy of 0.967, outperforming a number of baseline methods, indicating its effectiveness in user intent classification.

Bayesian Probability and Tanimoto Based Recurrent Neural Network for Question Answering System

2021 ◽  
Author(s):  
Veeraraghavan Jagannathan
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Question Answering ◽  
Bayesian Probability ◽  
Significant Information ◽  
Continuous Increase ◽  
Question Answering System ◽  
Relevant Result ◽  
Proper Nouns ◽  
Different Types

Question Answering (QA) has become one of the most significant information retrieval applications. Despite that, most of the question answering system focused to increase the user experience in finding the relevant result. Due to the continuous increase of web content, retrieving the relevant result faces a challenging issue in the Question Answering System (QAS). Thus, an effective Question Classification (QC), and retrieval approach named Bayesian probability and Tanimoto-based Recurrent Neural Network (RNN) are proposed in this research to differentiate the types of questions more efficiently. This research presented an analysis of different types of questions with respect to the grammatical structures. Various patterns are identified from the questions and the RNN classifier is used to classify the questions. The results obtained by the proposed Bayesian probability and Tanimoto-based RNN showed that the syntactic categories related to the domain-specific types of proper nouns, numeral numbers, and the common nouns enable the RNN classifier to reveal better result for different types of questions. However, the proposed approach obtained better performance in terms of precision, recall, and F-measure with the values of 90.14, 86.301, and 90.936 using dataset-2.

scholarly journals PV Power Prediction, Using CNN-LSTM Hybrid Neural Network Model. Case of Study: Temixco-Morelos, México

Energies ◽  
2020 ◽  
Vol 13 (24) ◽  
pp. 6512
Author(s):  
Mario Tovar ◽  
Miguel Robles ◽  
Felipe Rashid
Keyword(s):  
Neural Network ◽  
Hybrid Model ◽  
Network Model ◽  
Neural Network Model ◽  
Short Term Memory ◽  
Lasso Regression ◽  
Neural Network Models ◽  
Model Case ◽  
Hybrid Neural Network ◽  
Photovoltaic Power

Due to the intermittent nature of solar energy, accurate photovoltaic power predictions are very important for energy integration into existing energy systems. The evolution of deep learning has also opened the possibility to apply neural network models to predict time series, achieving excellent results. In this paper, a five layer CNN-LSTM model is proposed for photovoltaic power predictions using real data from a location in Temixco, Morelos in Mexico. In the proposed hybrid model, the convolutional layer acts like a filter, extracting local features of the data; then the temporal features are extracted by the long short-term memory network. Finally, the performance of the hybrid model with five layers is compared with a single model (a single LSTM), a CNN-LSTM hybrid model with two layers and two well known popular benchmarks. The results also shows that the hybrid neural network model has better prediction effect than the two layer hybrid model, the single prediction model, the Lasso regression or the Ridge regression.

scholarly journals Physical Hybrid Neural Network Model to Forecast Typhoon Floods

Water ◽  
2018 ◽  
Vol 10 (5) ◽  
pp. 632 ◽  
Author(s):  
You-Da Jhong ◽  
Chang-Shian Chen ◽  
Hsin-Ping Lin ◽  
Shien-Tsung Chen
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Hybrid Neural Network
Sign in / Sign up

Export Citation Format

Share Document