scholarly journals An Improved Double Channel Long Short-Term Memory Model for Medical Text Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Shengbin Liang ◽  
Xinan Chen ◽  
Jixin Ma ◽  
Wencai Du ◽  
Huawei Ma
Keyword(s):  
Text Classification ◽  
Short Term Memory ◽  
Short Term ◽  
Model Learning ◽  
Current Time ◽  
Term Memory ◽  
Medical Text ◽  
Word Level ◽  
Long Short Term Memory ◽  
Double Channel

There are a large number of symptom consultation texts in medical and healthcare Internet communities, and Chinese health segmentation is more complex, which leads to the low accuracy of the existing algorithms for medical text classification. The deep learning model has advantages in extracting abstract features of text effectively. However, for a large number of samples of complex text data, especially for words with ambiguous meanings in the field of Chinese medical diagnosis, the word-level neural network model is insufficient. Therefore, in order to solve the triage and precise treatment of patients, we present an improved Double Channel (DC) mechanism as a significant enhancement to Long Short-Term Memory (LSTM). In this DC mechanism, two channels are used to receive word-level and char-level embedding, respectively, at the same time. Hybrid attention is proposed to combine the current time output with the current time unit state and then using attention to calculate the weight. By calculating the probability distribution of each timestep input data weight, the weight score is obtained, and then weighted summation is performed. At last, the data input by each timestep is subjected to trade-off learning to improve the generalization ability of the model learning. Moreover, we conduct an extensive performance evaluation on two different datasets: cMedQA and Sentiment140. The experimental results show that the DC-LSTM model proposed in this paper has significantly superior accuracy and ROC compared with the basic CNN-LSTM model.

scholarly journals Major–Minor Long Short-Term Memory for Word-Level Language Model

2020 ◽  
Vol 31 (10) ◽  
pp. 3932-3946
Author(s):  
Kai Shuang ◽  
Rui Li ◽  
Mengyu Gu ◽  
Jonathan Loo ◽  
Sen Su
Keyword(s):  
Short Term Memory ◽  
Language Model ◽  
Short Term ◽  
Term Memory ◽  
Word Level ◽  
Long Short Term Memory

scholarly journals Chinese Text Classification Model Based on Deep Learning

2018 ◽  
Vol 10 (11) ◽  
pp. 113 ◽  
Author(s):  
Yue Li ◽  
Xutao Wang ◽  
Pengjian Xu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Language Processing ◽  
Chinese Text ◽  
Text Classification ◽  
Short Term Memory ◽  
Classification Model ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning model which achieves better classification results in Chinese text than those of other researchers’ models. After comparing different methods, long short-term memory (LSTM) and convolutional neural network (CNN) methods were selected as deep learning methods to classify Chinese text. LSTM is a special kind of recurrent neural network (RNN), which is capable of processing serialized information through its recurrent structure. By contrast, CNN has shown its ability to extract features from visual imagery. Therefore, two layers of LSTM and one layer of CNN were integrated to our new model: the BLSTM-C model (BLSTM stands for bi-directional long short-term memory while C stands for CNN.) LSTM was responsible for obtaining a sequence output based on past and future contexts, which was then input to the convolutional layer for extracting features. In our experiments, the proposed BLSTM-C model was evaluated in several ways. In the results, the model exhibited remarkable performance in text classification, especially in Chinese texts.

scholarly journals Analysis of Public Opinion Sentiment on Instagram regarding Covid-19 with SVM

2021 ◽  
Vol 1 (1) ◽  
pp. 67
Author(s):  
Iin Kurniasari ◽  
Kusrini Kusrini ◽  
Hanif Al Fatta
Keyword(s):  
Support Vector Machine ◽  
Public Opinion ◽  
Short Term Memory ◽  
Support Vector ◽  
Response Ratio ◽  
Short Term ◽  
Term Memory ◽  
Social Distancing ◽  
Word Level ◽  
Long Short Term Memory

Perkembangan teknologi dewasa ini mendorong masyarakat untuk selalu tanggap teknologi, terlebih di era pandemi covid-19 yang selalu mengedepankan social distancing. Media sosial digunakan sebagai suatu alat untuk menyampaikan opini masyarakat kepada khalayak. Dalam penelitian ini, penulis melakukan penelitian tentang opini masyaraat pada media sosial instagram dengan mengguakan Support Vector Machine. Setelah dilakukan uji akurasi dan presisi ternyata SVM belum sesuai digunakan sebagai algoritma yang dapat menangkap urutan karena susunan kata yang dibolak-balik meskipun maknanya berbeda tetap bermakna sama oleh mesin SVM, hal ini dibuktikan juga dengan jumlah akurasi yang kecil.yaitu 59%. Sehingga diperlukan langkah untuk bisa diteliti dengan algoritma lain misalnya algoritma HRRN (Highest Response Ratio Next) atau LSTM (Long Short-Term Memory) yang memperhatikan urutan dan proses dengan rasio respon paling tinggi. Jika berdasarkan pendekatan ekstraksi fitur SVM dengan pendekatan count vector, tf-idf word level, tf-idf ngram level dan tf-idf char level. Dalam skenario ini nilai akurasi tertinggi terdapat pada perhitungan dengan menggunakan ekstraksi fitur count vector dan tf-idf ngram level.

scholarly journals A Comparison of Power Quality Disturbance Detection and Classification Methods Using CNN, LSTM and CNN-LSTM

2020 ◽  
Vol 10 (19) ◽  
pp. 6755
Author(s):  
Carlos Iturrino Garcia ◽  
Francesco Grasso ◽  
Antonio Luchetta ◽  
Maria Cristina Piccirilli ◽  
Libero Paolucci ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Data Augmentation ◽  
Short Term Memory ◽  
Machine Learning Algorithms ◽  
Validation Dataset ◽  
Short Term ◽  
Current Time ◽  
Term Memory ◽  
Long Short Term Memory

The use of electronic loads has improved many aspects of everyday life, permitting more efficient, precise and automated process. As a drawback, the nonlinear behavior of these systems entails the injection of electrical disturbances on the power grid that can cause distortion of voltage and current. In order to adopt countermeasures, it is important to detect and classify these disturbances. To do this, several Machine Learning Algorithms are currently exploited. Among them, for the present work, the Long Short Term Memory (LSTM), the Convolutional Neural Networks (CNN), the Convolutional Neural Networks Long Short Term Memory (CNN-LSTM) and the CNN-LSTM with adjusted hyperparameters are compared. As a preliminary stage of the research, the voltage and current time signals are simulated using MATLAB Simulink. Thanks to the simulation results, it is possible to acquire a current and voltage dataset with which the identification algorithms are trained, validated and tested. These datasets include simulations of several disturbances such as Sag, Swell, Harmonics, Transient, Notch and Interruption. Data Augmentation techniques are used in order to increase the variability of the training and validation dataset in order to obtain a generalized result. After that, the networks are fed with an experimental dataset of voltage and current field measurements containing the disturbances mentioned above. The networks have been compared, resulting in a 79.14% correct classification rate with the LSTM network versus a 84.58% for the CNN, 84.76% for the CNN-LSTM and a 83.66% for the CNN-LSTM with adjusted hyperparameters. All of these networks are tested using real measurements.

scholarly journals Structured Sparsification of Gated Recurrent Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 4989-4996
Author(s):  
Ekaterina Lobacheva ◽  
Nadezhda Chirkova ◽  
Alexander Markovich ◽  
Dmitry Vetrov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Text Classification ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Language Modeling ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Network Compression

One of the most popular approaches for neural network compression is sparsification — learning sparse weight matrices. In structured sparsification, weights are set to zero by groups corresponding to structure units, e. g. neurons. We further develop the structured sparsification approach for the gated recurrent neural networks, e. g. Long Short-Term Memory (LSTM). Specifically, in addition to the sparsification of individual weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies an LSTM structure. We test our approach on the text classification and language modeling tasks. Our method improves the neuron-wise compression of the model in most of the tasks. We also observe that the resulting structure of gate sparsity depends on the task and connect the learned structures to the specifics of the particular tasks.

scholarly journals Vietnamese Text Classification Algorithm using Long Short Term Memory and Word2Vec

2020 ◽  
Vol 19 (6) ◽  
pp. 1255-1279
Author(s):  
Huu Nguyen Phat ◽  
Nguyen Thi Minh Anh
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Text Classification ◽  
Industrial Revolution ◽  
Short Term Memory ◽  
Text Processing ◽  
Research Effort ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

In the context of the ongoing forth industrial revolution and fast computer science development the amount of textual information becomes huge. So, prior to applying the seemingly appropriate methodologies and techniques to the above data processing their nature and characteristics should be thoroughly analyzed and understood. At that, automatic text processing incorporated in the existing systems may facilitate many procedures. So far, text classification is one of the basic applications to natural language processing accounting for such factors as emotions’ analysis, subject labeling etc. In particular, the existing advancements in deep learning networks demonstrate that the proposed methods may fit the documents’ classifying, since they possess certain extra efficiency; for instance, they appeared to be effective for classifying texts in English. The thorough study revealed that practically no research effort was put into an expertise of the documents in Vietnamese language. In the scope of our study, there is not much research for documents in Vietnamese. The development of deep learning models for document classification has demonstrated certain improvements for texts in Vietnamese. Therefore, the use of long short term memory network with Word2vec is proposed to classify text that improves both performance and accuracy. The here developed approach when compared with other traditional methods demonstrated somewhat better results at classifying texts in Vietnamese language. The evaluation made over datasets in Vietnamese shows an accuracy of over 90%; also the proposed approach looks quite promising for real applications.

Sign in / Sign up

Export Citation Format

Share Document