scholarly journals A Bidirectional LSTM Language Model for Code Evaluation and Repair

Symmetry ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 247
Author(s):  
Md. Mostafizer Rahman ◽  
Yutaka Watanobe ◽  
Keita Nakamura
Keyword(s):  
Error Detection ◽  
Short Term Memory ◽  
Language Model ◽  
Source Code ◽  
Short Term ◽  
Science And Engineering ◽  
Term Memory ◽  
Source Codes ◽  
Long Short Term Memory ◽  
Logical Errors

Programming is a vital skill in computer science and engineering-related disciplines. However, developing source code is an error-prone task. Logical errors in code are particularly hard to identify for both students and professionals, and a single error is unexpected to end-users. At present, conventional compilers have difficulty identifying many of the errors (especially logical errors) that can occur in code. To mitigate this problem, we propose a language model for evaluating source codes using a bidirectional long short-term memory (BiLSTM) neural network. We trained the BiLSTM model with a large number of source codes with tuning various hyperparameters. We then used the model to evaluate incorrect code and assessed the model’s performance in three principal areas: source code error detection, suggestions for incorrect code repair, and erroneous code classification. Experimental results showed that the proposed BiLSTM model achieved 50.88% correctness in identifying errors and providing suggestions. Moreover, the model achieved an F-score of approximately 97%, outperforming other state-of-the-art models (recurrent neural networks (RNNs) and long short-term memory (LSTM)).

scholarly journals Major–Minor Long Short-Term Memory for Word-Level Language Model

2020 ◽  
Vol 31 (10) ◽  
pp. 3932-3946
Author(s):  
Kai Shuang ◽  
Rui Li ◽  
Mengyu Gu ◽  
Jonathan Loo ◽  
Sen Su
Keyword(s):  
Short Term Memory ◽  
Language Model ◽  
Short Term ◽  
Term Memory ◽  
Word Level ◽  
Long Short Term Memory

scholarly journals Learning molecular dynamics with simple language model built upon long short-term memory neural network

2020 ◽  
Vol 11 (1) ◽  
Author(s):  
Sun-Ting Tsai ◽  
En-Jui Kuo ◽  
Pratyush Tiwary
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Language Model ◽  
Short Term ◽  
Term Memory ◽  
Molecular Systems ◽  
Simple Language ◽  
Long Short Term Memory

Abstract Recurrent neural networks have led to breakthroughs in natural language processing and speech recognition. Here we show that recurrent networks, specifically long short-term memory networks can also capture the temporal evolution of chemical/biophysical trajectories. Our character-level language model learns a probabilistic model of 1-dimensional stochastic trajectories generated from higher-dimensional dynamics. The model captures Boltzmann statistics and also reproduces kinetics across a spectrum of timescales. We demonstrate how training the long short-term memory network is equivalent to learning a path entropy, and that its embedding layer, instead of representing contextual meaning of characters, here exhibits a nontrivial connectivity between different metastable states in the underlying physical system. We demonstrate our model’s reliability through different benchmark systems and a force spectroscopy trajectory for multi-state riboswitch. We anticipate that our work represents a stepping stone in the understanding and use of recurrent neural networks for understanding the dynamics of complex stochastic molecular systems.

scholarly journals Post Text Processing of Chinese Speech Recognition Based on Bidirectional LSTM Networks and CRF

Electronics ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. 1248 ◽  
Author(s):  
Li Yang ◽  
Ying Li ◽  
Jin Wang ◽  
Zhuo Tang
Keyword(s):  
Speech Recognition ◽  
Error Detection ◽  
Short Term Memory ◽  
Text Processing ◽  
Conditional Random Field ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Lstm Network ◽  
Two Stages

With the rapid development of Internet of Things Technology, speech recognition has been applied more and more widely. Chinese Speech Recognition is a complex process. In the process of speech-to-text conversion, due to the influence of dialect, environmental noise, and context, the accuracy of speech-to-text in multi-round dialogues and specific contexts is still not high. After the general speech recognition technology, the text after speech recognition can be detected and corrected in the specific context, which is helpful to improve the robustness of text comprehension and is a beneficial supplement to the speech recognition technology. In this paper, a text processing model after Chinese Speech Recognition is proposed, which combines a bidirectional long short-term memory (LSTM) network with a conditional random field (CRF) model. The task is divided into two stages: text error detection and text error correction. In this paper, a bidirectional long short-term memory (Bi-LSTM) network and conditional random field are used in two stages of text error detection and text error correction respectively. Through verification and system test on the SIGHAN 2013 Chinese Spelling Check (CSC) dataset, the experimental results show that the model can effectively improve the accuracy of text after speech recognition.

Learning Syllables Using Conv-LSTM Model for Swahili Word Representation and Part-of-speech Tagging

2021 ◽  
Vol 20 (4) ◽  
pp. 1-25
Author(s):  
Casper Shikali Shivachi ◽  
Refuoe Mokhosi ◽  
Zhou Shijie ◽  
Liu Qihe
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Language Model ◽  
Word Embeddings ◽  
Short Term ◽  
Term Memory ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Word Representation ◽  
Long Short Term Memory

The need to capture intra-word information in natural language processing (NLP) tasks has inspired research in learning various word representations at word, character, or morpheme levels, but little attention has been given to syllables from a syllabic alphabet. Motivated by the success of compositional models in morphological languages, we present a Convolutional-long short term memory (Conv-LSTM) model for constructing Swahili word representation vectors from syllables. The unified architecture addresses the word agglutination and polysemous nature of Swahili by extracting high-level syllable features using a convolutional neural network (CNN) and then composes quality word embeddings with a long short term memory (LSTM). The word embeddings are then validated using a syllable-aware language model ( 31.267 ) and a part-of-speech (POS) tagging task ( 98.78 ), both yielding very competitive results to the state-of-art models in their respective domains. We further validate the language model using Xhosa and Shona, which are syllabic-based languages. The novelty of the study is in its capability to construct quality word embeddings from syllables using a hybrid model that does not use max-over-pool common in CNN and then the exploitation of these embeddings in POS tagging. Therefore, the study plays a crucial role in the processing of agglutinative and syllabic-based languages by contributing quality word embeddings from syllable embeddings, a robust Conv–LSTM model that learns syllables for not only language modeling and POS tagging, but also for other downstream NLP tasks.

Evolutionary optimization of long short-term memory neural network language model

2016 ◽  
Vol 140 (4) ◽  
pp. 3062-3062
Author(s):  
Tomohiro Tanaka ◽  
Takafumi Moriya ◽  
Takahiro Shinozaki ◽  
Shinji Watanabe ◽  
Takaaki Hori ◽  
...  
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Language Model ◽  
Evolutionary Optimization ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Network Language

scholarly journals NEWLSTM: An Optimized Long Short-Term Memory Language Model for Sequence Prediction

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 65395-65401
Author(s):  
Qing Wang ◽  
Rong-Qun Peng ◽  
Jia-Qiang Wang ◽  
Zhi Li ◽  
Han-Bing Qu
Keyword(s):  
Short Term Memory ◽  
Language Model ◽  
Short Term ◽  
Term Memory ◽  
Sequence Prediction ◽  
Long Short Term Memory
Sign in / Sign up

Export Citation Format

Share Document