An experimental comparison of recurrent neural network for natural language production

2004 ◽  
Author(s):  
H. Nakagama ◽  
S. Tanaka
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Recurrent Neural Network ◽  
Language Production ◽  
Experimental Comparison

scholarly journals Natural language description of images using hybrid recurrent neural network

2019 ◽  
Vol 9 (4) ◽  
pp. 2932
Author(s):  
Md. Asifuzzaman Jishan ◽  
Khan Raqib Mahmud ◽  
Abul Kalam Al Azad
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Text Line ◽  
Short Term ◽  
Word Representation ◽  
Benchmark Datasets ◽  
Long Short Term Memory ◽  
Language Description

We presented a learning model that generated natural language description of images. The model utilized the connections between natural language and visual data by produced text line based contents from a given image. Our Hybrid Recurrent Neural Network model is based on the intricacies of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Bi-directional Recurrent Neural Network (BRNN) models. We conducted experiments on three benchmark datasets, e.g., Flickr8K, Flickr30K, and MS COCO. Our hybrid model utilized LSTM model to encode text line or sentences independent of the object location and BRNN for word representation, this reduced the computational complexities without compromising the accuracy of the descriptor. The model produced better accuracy in retrieving natural language based description on the dataset.

scholarly journals Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference

2017 ◽  
Author(s):  
Qian Chen ◽  
Xiaodan Zhu ◽  
Zhen-Hua Ling ◽  
Si Wei ◽  
Hui Jiang ◽  
...  
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Recurrent Neural Network

scholarly journals Smart Project Management: Interactive Platform Using Natural Language Processing Technology

2021 ◽  
Vol 11 (4) ◽  
pp. 1597
Author(s):  
Jieh-Haur Chen ◽  
Mu-Chun Su ◽  
Vidya Trisandini Azzizi ◽  
Ting-Kwei Wang ◽  
Wei-Jen Lin
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Project Management ◽  
Natural Language ◽  
Real Time ◽  
Construction Industry ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Processing Technology ◽  
Management Platform

Technological developments have made the construction industry efficient. The aim of this research is to solve communication interaction problems to build a project management platform using the interactive concept of natural language processing technology. A comprehensive literature review and expert interviews associated with techniques dealing with natural languages suggests the proposed system containing the Progressive Scale Expansion Network (PSENet), Convolutional Recurrent Neural Network (CRNN), and Bi-directional Recurrent Neutral Networks Convolutional Recurrent Neural Network (BRNN-CNN) toolboxes to extract the key words for construction projects contracts. The results show that a fully automatic platform facilitating contract management is achieved. For academic domains, the Contract Keyword Detection (CKD) mechanism integrating PSENet, CRNN, and BRNN-CNN approaches to cope with real-time massive document flows is novel in the construction industry. For practice, the proposed approach brings significant reduction for manpower and human error, an alternative for settling down misunderstanding or disputes due to real-time and precise communication, and a solution for efficient documentary management. It connects all contract stakeholders proficiently.

scholarly journals Malicious URL Detection Algorithm Based on Multi Neural Network Series

CONVERTER ◽  
2021 ◽  
pp. 579-590
Author(s):  
Weirong Xiu
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Convolutional Neural Network ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Detection Algorithm ◽  
Attention Mechanism ◽  
Global Features ◽  
Multi Neural Network

Convolutional neural network based on attention mechanism and a bidirectional independent recurrent neural network tandem joint algorithm (CATIR) are proposed. In natural language processing related technologies, word vector features are extracted based on URLs, and the extracted URL information features and host information features are merged. The proposed CATIR algorithm uses CNN (Convolutional Neural Network) to obtain the deep local features in the data, uses the Attention mechanism to adjust the weights, and uses IndRNN (Independent Recurrent Neural Network) to obtain the global features in the data. The experimental results shows that the CATIR algorithm has significantly improved the accuracy of malicious URL detection based on traditional algorithms to 96.9%.

scholarly journals Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts

2017 ◽  
Vol 24 (4) ◽  
pp. 813-821 ◽  
Author(s):  
Anne Cocos ◽  
Alexander G Fiks ◽  
Aaron J Masino
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
State Of The Art ◽  
Detection Performance ◽  
Word Embeddings

Abstract Objective Social media is an important pharmacovigilance data source for adverse drug reaction (ADR) identification. Human review of social media data is infeasible due to data quantity, thus natural language processing techniques are necessary. Social media includes informal vocabulary and irregular grammar, which challenge natural language processing methods. Our objective is to develop a scalable, deep-learning approach that exceeds state-of-the-art ADR detection performance in social media. Materials and Methods We developed a recurrent neural network (RNN) model that labels words in an input sequence with ADR membership tags. The only input features are word-embedding vectors, which can be formed through task-independent pretraining or during ADR detection training. Results Our best-performing RNN model used pretrained word embeddings created from a large, non–domain-specific Twitter dataset. It achieved an approximate match F-measure of 0.755 for ADR identification on the dataset, compared to 0.631 for a baseline lexicon system and 0.65 for the state-of-the-art conditional random field model. Feature analysis indicated that semantic information in pretrained word embeddings boosted sensitivity and, combined with contextual awareness captured in the RNN, precision. Discussion Our model required no task-specific feature engineering, suggesting generalizability to additional sequence-labeling tasks. Learning curve analysis showed that our model reached optimal performance with fewer training examples than the other models. Conclusions ADR detection performance in social media is significantly improved by using a contextually aware model and word embeddings formed from large, unlabeled datasets. The approach reduces manual data-labeling requirements and is scalable to large social media datasets.

scholarly journals Ensemble Model for Chunking

2021 ◽  
Author(s):  
Nilamadhaba Mohapatra ◽  
Namrata Sarraf ◽  
Swapna sarit Sahu
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Recurrent Neural Network ◽  
Neural Network Model ◽  
Statistical Models ◽  
Neural Nets ◽  
Language Models ◽  
Ensemble Model ◽  
Neural Net ◽  
Transformer Model

Transformer Models have taken over most of the Natural language Inference tasks. In recent times they have proved to beat several benchmarks. Chunking means splitting the sentences into tokens and then grouping them in a meaningful way. Chunking is a task that has gradually moved from POS tag-based statistical models to neural nets using Language models such as LSTM, Bidirectional LSTMs, attention models, etc. Deep neural net Models are deployed indirectly for classifying tokens as different tags defined under Named Recognition Tasks. Later these tags are used in conjunction with pointer frameworks for the final chunking task. In our paper, we propose an Ensemble Model using a fine-tuned Transformer Model and a recurrent neural network model together to predict tags and chunk substructures of a sentence. We analyzed the shortcomings of the transformer models in predicting different tags and then trained the BILSTM+CNN accordingly to compensate for the same.

scholarly journals Tackling Graphical Natural Language Processing’s Problems with Recurrent Neural Networks

2019 ◽  
Vol 54 (5) ◽  
Author(s):  
Ali Sami Sosa ◽  
Saja Majeed Mohammed ◽  
Haider Hadi Abbas ◽  
Israa Al Barazanchi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Question Answering ◽  
Structural Information ◽  
Sequential Data

Recent years have witnessed the success of artificial intelligence–based automated systems that use deep learning, especially recurrent neural network-based models, on many natural language processing problems, including machine translation and question answering. Besides, recurrent neural networks and their variations have been extensively studied with respect to several graph problems and have shown preliminary success. Despite these successes, recurrent neural network -based models continue to suffer from several major drawbacks. First, they can only consume sequential data; thus, linearization is required to serialize input graphs, resulting in the loss of important structural information. In particular, graph nodes that are originally located closely to each other can be very far away after linearization, and this introduces great challenges for recurrent neural networks to model their relation. Second, the serialization results are usually very long, so it takes a long time for recurrent neural networks to encode them. In the methodology of this study, we made the resulting graphs more densely connected so that more useful facts could be inferred, and the problem of graphical natural language processing could be easily decoded with graph recurrent neural network. As a result, the performances with single-typed edges were significantly better than the Local baseline, whereas the combination of all types of edges achieved a much better accuracy than just that of the Local using recurrent neural network. In this paper, we propose a novel graph neural network, named graph recurrent network.

Research on behavior recognition based on feature fusion of automatic coder and recurrent neural network

2020 ◽  
Vol 39 (6) ◽  
pp. 8927-8935
Author(s):  
Bing Zheng ◽  
Dawei Yun ◽  
Yan Liang
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Behavior Pattern ◽  
Rapid Development ◽  
Video Data ◽  
Support Vector ◽  
Behavior Recognition ◽  
Learning Methods ◽  
The Impact ◽  
Internet Of Things Technology

Under the impact of COVID-19, research on behavior recognition are highly needed. In this paper, we combine the algorithm of self-adaptive coder and recurrent neural network to realize the research of behavior pattern recognition. At present, most of the research of human behavior recognition is focused on the video data, which is based on the video number. At the same time, due to the complexity of video image data, it is easy to violate personal privacy. With the rapid development of Internet of things technology, it has attracted the attention of a large number of experts and scholars. Researchers have tried to use many machine learning methods, such as random forest, support vector machine and other shallow learning methods, which perform well in the laboratory environment, but there is still a long way to go from practical application. In this paper, a recursive neural network algorithm based on long and short term memory (LSTM) is proposed to realize the recognition of behavior patterns, so as to improve the accuracy of human activity behavior recognition.

Sign in / Sign up

Export Citation Format

Share Document