Deep Neural Models and Retrofitting for Arabic Text Categorization

2020 ◽  
Vol 16 (2) ◽  
pp. 74-86 ◽  
Author(s):  
Fatima-Zahra El-Alami ◽  
Said Ouatik El Alaoui ◽  
Noureddine En-Nahnahi
Keyword(s):  
Text Categorization ◽  
Short Term Memory ◽  
Network Models ◽  
Arabic Text ◽  
Short Term ◽  
Lexical Resources ◽  
Neural Network Models ◽  
Online Data ◽  
Long Short Term Memory ◽  
Vector Representations

Arabic text categorization is an important task in text mining particularly with the fast-increasing quantity of the Arabic online data. Deep neural network models have shown promising performance and indicated great data modeling capacities in managing large and substantial datasets. This article investigates convolution neural networks (CNNs), long short-term memory (LSTM) and their combination for Arabic text categorization. This work additionally handles the morphological variety of Arabic words by exploring the word embeddings model using position weights and subword information. To guarantee the nearest vector representations for connected words, this article adopts a strategy for refining Arabic vector space representations using semantic information embedded in lexical resources. Several experiments utilizing different architectures have been conducted on the OSAC dataset. The obtained results show the effectiveness of CNN-LSTM without and with retrofitting for Arabic text categorization in comparison with major competing methods.

scholarly journals Residual Life Prediction for Induction Furnace by Sequential Encoder with s-Convolutional LSTM

Processes ◽  
2021 ◽  
Vol 9 (7) ◽  
pp. 1121
Author(s):  
Yulim Choi ◽  
Hyeonho Kwun ◽  
Dohee Kim ◽  
Eunju Lee ◽  
Hyerim Bae
Keyword(s):  
Life Prediction ◽  
Induction Furnace ◽  
Short Term Memory ◽  
Residual Life ◽  
Network Models ◽  
Sensor Data ◽  
Short Term ◽  
Neural Network Models ◽  
Long Short Term Memory ◽  
Convolutional Lstm

Induction furnaces are widely used for melting scrapped steel in small foundries and their use has recently become more frequent. The maintenance of induction furnaces is usually based on empirical decisions of the operator and an explosion can occur through operator error. To prevent an explosion, previous studies have utilized statistical models but have been unable to generalize the problem and have achieved a low accuracy. Herein, we propose a data-driven method for induction furnaces by proposing a novel 2D matrix called a sequential feature matrix(s-encoder) and multi-channel convolutional long short-term memory (s-ConLSTM). First, the sensor data and operation data are converted into sequential feature matrices. Then, N-sequential feature matrices are imported into the convolutional LSTM model to predict the residual life of the induction furnace wall. Based on our experimental results, our method outperforms general neural network models and enhances the safe use of induction furnaces.

Categorizing Natural Language-Based Customer Satisfaction: An Implementation Method Using Support Vector Machine and Long Short-Term Memory Neural Network

2021 ◽  
Vol 13 (4) ◽  
Author(s):  
Ralph Sherwin A. Corpuz ◽  
Keyword(s):  
Neural Network ◽  
Support Vector Machine ◽  
Natural Language ◽  
Text Categorization ◽  
Short Term Memory ◽  
Support Vector ◽  
Feature Engineering ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Analyzing natural language-based Customer Satisfaction (CS) is a tedious process. This issue is practically true if one is to manually categorize large datasets. Fortunately, the advent of supervised machine learning techniques has paved the way toward the design of efficient categorization systems used for CS. This paper presents the feasibility of designing a text categorization model using two popular and robust algorithms – the Support Vector Machine (SVM) and Long Short-Term Memory (LSTM) Neural Network, in order to automatically categorize complaints, suggestions, feedbacks, and commendations. The study found that, in terms of training accuracy, SVM has best rating of 98.63% while LSTM has best rating of 99.32%. Such results mean that both SVM and LSTM algorithms are at par with each other in terms of training accuracy, but SVM is significantly faster than LSTM by approximately 35.47s. The training performance results of both algorithms are attributed on the limitations of the dataset size, high-dimensionality of both English and Tagalog languages, and applicability of the feature engineering techniques used. Interestingly, based on the results of actual implementation, both algorithms are found to be 100% effective in accurately predicting the correct CS categories. Hence, the extent of preference between the two algorithms boils down on the available dataset and the skill in optimizing these algorithms through feature engineering techniques and in implementing them toward actual text categorization applications.

Multi‐dimensional long short‐term memory networks for artificial Arabic text recognition in news video

2018 ◽  
Vol 12 (5) ◽  
pp. 710-719 ◽  
Author(s):  
Oussama Zayene ◽  
Sameh Masmoudi Touj ◽  
Jean Hennebert ◽  
Rolf Ingold ◽  
Najoua Essoukri Ben Amara
Keyword(s):  
Short Term Memory ◽  
Text Recognition ◽  
Arabic Text ◽  
Short Term ◽  
Term Memory ◽  
News Video ◽  
Long Short Term Memory

scholarly journals Identification of Gender based on Blogs using Attention-based Recurrent Neural Network

2019 ◽  
Vol 8 (4) ◽  
pp. 3152-3158
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Absolute Error ◽  
Digital Content ◽  
Human Beings ◽  
Short Term ◽  
Neural Network Models ◽  
Term Memory ◽  
Long Short Term Memory

With the digitization, the importance of content writing is being increased. This is due to the huge improvement in accessibility and the major impact of digital content on human beings. Due to veracity and huge demand for digital content, author profiling becomes a necessity to identify the correct person for particular content writing. This paper works on deep neural network models to identify the gender of author for any particular content. The analysis has been done on the corpus dataset by using artificial neural networks with different number of layers, long short term memory based Recurrent Neural Network (RNN), bidirectional long short term memory based RNN and attention-based RNN models using mean absolute error, root mean square error, accuracy, and loss as analysis parameters. The results of different epochs show the significance of each model.

scholarly journals Abstractive Arabic Text Summarization Based on Deep Learning

2022 ◽  
Vol 2022 ◽  
pp. 1-14
Author(s):  
Y.M. Wazery ◽  
Marwa E. Saleh ◽  
Abdullah Alharbi ◽  
Abdelmgeid A. Ali
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Arabic Text ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Arabic Text Summarization ◽  
Abstractive Summarization ◽  
Research Studies

Text summarization (TS) is considered one of the most difficult tasks in natural language processing (NLP). It is one of the most important challenges that stand against the modern computer system’s capabilities with all its new improvement. Many papers and research studies address this task in literature but are being carried out in extractive summarization, and few of them are being carried out in abstractive summarization, especially in the Arabic language due to its complexity. In this paper, an abstractive Arabic text summarization system is proposed, based on a sequence-to-sequence model. This model works through two components, encoder and decoder. Our aim is to develop the sequence-to-sequence model using several deep artificial neural networks to investigate which of them achieves the best performance. Different layers of Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM), and Bidirectional Long Short-Term Memory (BiLSTM) have been used to develop the encoder and the decoder. In addition, the global attention mechanism has been used because it provides better results than the local attention mechanism. Furthermore, AraBERT preprocess has been applied in the data preprocessing stage that helps the model to understand the Arabic words and achieves state-of-the-art results. Moreover, a comparison between the skip-gram and the continuous bag of words (CBOW) word2Vec word embedding models has been made. We have built these models using the Keras library and run-on Google Colab Jupiter notebook to run seamlessly. Finally, the proposed system is evaluated through ROUGE-1, ROUGE-2, ROUGE-L, and BLEU evaluation metrics. The experimental results show that three layers of BiLSTM hidden states at the encoder achieve the best performance. In addition, our proposed system outperforms the other latest research studies. Also, the results show that abstractive summarization models that use the skip-gram word2Vec model outperform the models that use the CBOW word2Vec model.

scholarly journals Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems

2019 ◽  
Vol 10 (1) ◽  
pp. 1-19
Author(s):  
Matthieu Riou ◽  
Bassam Jabaian ◽  
Stéphane Huet ◽  
Fabrice Lefèvre
Keyword(s):  
Natural Language ◽  
Short Term Memory ◽  
Network Models ◽  
Dialogue Systems ◽  
Neural Network Models ◽  
Spoken Dialogue Systems ◽  
Spoken Dialogue ◽  
Automatic Transcription ◽  
Long Short Term Memory ◽  
Expected Payoffs

Following some recent propositions to handle natural language generation in spoken dialogue systems with long short-term memory recurrent neural network models~\citep{Wen2016a} we first investigate a variant thereof with the objective of a better integration of the attention subnetwork. Then our next objective is to propose and evaluate a framework to adapt the NLG module online through direct interactions with the users. When doing so the basic way is to ask the user to utter an alternative sentence to express a particular dialogue act. But then the system has to decide between using an automatic transcription or to ask for a manual transcription. To do so a reinforcement learning approach based on an adversarial bandit scheme is retained. We show that by defining appropriately the rewards as a linear combination of expected payoffs and costs of acquiring the new data provided by the user, a system design can balance between improving the system's performance towards a better match with the user's preferences and the burden associated with it. Then the actual benefits of this system is assessed with a human evaluation, showing that the addition of more diverse utterances allows to produce sentences more satisfying for the user.

scholarly journals Streamflow prediction using an integrated methodology based on convolutional neural network and long short-term memory networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Sujan Ghimire ◽  
Zaher Mundher Yaseen ◽  
Aitazaz A. Farooque ◽  
Ravinesh C. Deo ◽  
Ji Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Flood Control ◽  
Performance Metrics ◽  
Short Term Memory ◽  
Short Term ◽  
Neural Network Models ◽  
Time Intervals ◽  
Term Memory ◽  
Long Short Term Memory

AbstractStreamflow (Qflow) prediction is one of the essential steps for the reliable and robust water resources planning and management. It is highly vital for hydropower operation, agricultural planning, and flood control. In this study, the convolution neural network (CNN) and Long-Short-term Memory network (LSTM) are combined to make a new integrated model called CNN-LSTM to predict the hourly Qflow (short-term) at Brisbane River and Teewah Creek, Australia. The CNN layers were used to extract the features of Qflow time-series, while the LSTM networks use these features from CNN for Qflow time series prediction. The proposed CNN-LSTM model is benchmarked against the standalone model CNN, LSTM, and Deep Neural Network models and several conventional artificial intelligence (AI) models. Qflow prediction is conducted for different time intervals with the length of 1-Week, 2-Weeks, 4-Weeks, and 9-Months, respectively. With the help of different performance metrics and graphical analysis visualization, the experimental results reveal that with small residual error between the actual and predicted Qflow, the CNN-LSTM model outperforms all the benchmarked conventional AI models as well as ensemble models for all the time intervals. With 84% of Qflow prediction error below the range of 0.05 m3 s−1, CNN-LSTM demonstrates a better performance compared to 80% and 66% for LSTM and DNN, respectively. In summary, the results reveal that the proposed CNN-LSTM model based on the novel framework yields more accurate predictions. Thus, CNN-LSTM has significant practical value in Qflow prediction.

scholarly journals Exploring the Sensitivity of Recurrent Neural Network Models for Forecasting Land Cover Change

Land ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 282
Author(s):  
Alysha van Duynhoven ◽  
Suzana Dragićević
Keyword(s):  
Land Cover ◽  
Short Term Memory ◽  
Network Models ◽  
Optimization Techniques ◽  
Neural Network Models ◽  
Method Performance ◽  
Timeseries Analysis ◽  
Method Optimization ◽  
Long Short Term Memory ◽  
Real World Datasets

Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) architectures, have obtained successful outcomes in timeseries analysis tasks. While RNNs demonstrated favourable performance for Land Cover (LC) change analyses, few studies have explored or quantified the geospatial data characteristics required to utilize this method. Likewise, many studies utilize overall measures of accuracy rather than metrics accounting for the slow or sparse changes of LC that are typically observed. Therefore, the main objective of this study is to evaluate the performance of LSTM models for forecasting LC changes by conducting a sensitivity analysis involving hypothetical and real-world datasets. The intent of this assessment is to explore the implications of varying temporal resolutions and LC classes. Additionally, changing these input data characteristics impacts the number of timesteps and LC change rates provided to the respective models. Kappa variants are selected to explore the capacity of LSTM models for forecasting transitions or persistence of LC. Results demonstrate the adverse effects of coarser temporal resolutions and high LC class cardinality on method performance, despite method optimization techniques applied. This study suggests various characteristics of geospatial datasets that should be present before considering LSTM methods for LC change forecasting.

Sign in / Sign up

Export Citation Format

Share Document