Hybridization between Neural Computing and Nature-Inspired Algorithms for a Sentence Similarity Model Based on the Attention Mechanism

2021 ◽  
Vol 20 (1) ◽  
pp. 1-21
Author(s):  
Peiying Zhang ◽  
Xingzhe Huang ◽  
Maozhen Li ◽  
Yu Xue
Keyword(s):  
Neural Networks ◽  
Customer Service ◽  
Language Processing ◽  
Question Answering ◽  
Hybrid Algorithms ◽  
Attention Mechanism ◽  
Similarity Analysis ◽  
Model Based ◽  
Sentence Similarity ◽  
Nature Inspired Algorithms

Sentence similarity analysis has been applied in many fields, such as machine translation, the question answering system, and voice customer service. As a basic task of natural language processing, sentence similarity analysis plays an important role in many fields. The task of sentence similarity analysis is to establish a sentence similarity scoring model through multi-features. In previous work, researchers proposed a variety of models to deal with the calculation of sentence similarity. But these models do not consider the association information of sentence pairs, but only input sentence pairs into the model. In this article, we propose a sentence feature extraction model based on multi-feature attention. In addition, with the development of deep learning and the application of nature-inspired algorithms, researchers have proposed various hybrid algorithms that combine nature-inspired algorithms with neural networks. The hybrid algorithms not only solve the problem of decision-making based on multiple features but also improve the performance of the model. In the model, we use the attention mechanism to extract sentence features and assign weight. Then, the convolutional neural network is used to reduce the dimension of the matrix. In the training process, we integrate the firefly algorithm in the neural networks. The experimental results show that the accuracy of our model is 74.21%.

Sentence Similarity Algorithm Based on Fused Bi-Channel Dependency Matching Feature

2019 ◽  
Vol 34 (07) ◽  
pp. 2050019
Author(s):  
Xiaohan Guan ◽  
Jianhui Han ◽  
Zhi Liu ◽  
Mengmeng Zhang
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Question Answering ◽  
Calculation Methods ◽  
Matching Problem ◽  
Position Information ◽  
The Past ◽  
Vector Interaction ◽  
Sentence Similarity ◽  
Similarity Algorithm

Many tasks of natural language processing such as information retrieval, intelligent question answering, and machine translation require the calculation of sentence similarity. The traditional calculation methods used in the past could not solve semantic understanding problems well. First, the model structure based on Siamese lack of interaction between sentences; second, it has matching problem which contains lacking position information and only using partial matching factor based on the matching model. In this paper, a combination of word and word’s dependence is proposed to calculate the sentence similarity. This combination can extract the word features and word’s dependency features. To extract more matching features, a bi-directional multi-interaction matching sequence model is proposed by using word2vec and dependency2vec. This model obtains matching features by convolving and pooling the word-granularity (word vector, dependency vector) interaction sequences in two directions. Next, the model aggregates the bi-direction matching features. The paper evaluates the model on two tasks: paraphrase identification and natural language inference. The experimental results show that the combination of word and word’s dependence can enhance the ability of extracting matching features between two sentences. The results also show that the model with dependency can achieve higher accuracy than these models without using dependency.

scholarly journals English Machine Translation Model Based on an Improved Self-Attention Technology

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Wenxia Pan
Keyword(s):  
Machine Translation ◽  
Language Processing ◽  
Extraction Methods ◽  
Attention Mechanism ◽  
The Self ◽  
Position Information ◽  
Directional Information ◽  
Translation Model ◽  
Model Based ◽  
Phrase Extraction

English machine translation is a natural language processing research direction that has important scientific research value and practical value in the current artificial intelligence boom. The variability of language, the limited ability to express semantic information, and the lack of parallel corpus resources all limit the usefulness and popularity of English machine translation in practical applications. The self-attention mechanism has received a lot of attention in English machine translation tasks because of its highly parallelizable computing ability, which reduces the model’s training time and allows it to capture the semantic relevance of all words in the context. The efficiency of the self-attention mechanism, however, differs from that of recurrent neural networks because it ignores the position and structure information between context words. The English machine translation model based on the self-attention mechanism uses sine and cosine position coding to represent the absolute position information of words in order to enable the model to use position information between words. This method, on the other hand, can reflect relative distance but does not provide directionality. As a result, a new model of English machine translation is proposed, which is based on the logarithmic position representation method and the self-attention mechanism. This model retains the distance and directional information between words, as well as the efficiency of the self-attention mechanism. Experiments show that the nonstrict phrase extraction method can effectively extract phrase translation pairs from the n-best word alignment results and that the extraction constraint strategy can improve translation quality even further. Nonstrict phrase extraction methods and n-best alignment results can significantly improve the quality of translation translations when compared to traditional phrase extraction methods based on single alignment.

scholarly journals VR-Enabled Chatbot System Supporting Transformer Mass-Customization Services

2020 ◽  
Author(s):  
Finch C.-T. Wu ◽  
Oscar N.-J. Hong ◽  
Amy J.C. Trappey ◽  
Charles V. Trappey
Keyword(s):  
Virtual Reality ◽  
Natural Language ◽  
Customer Service ◽  
Language Processing ◽  
Mass Customization ◽  
Cost Estimation ◽  
Question Answering ◽  
Intelligent System ◽  
Language Communication ◽  
Customer Services

Chatbot is a conversational question answering (Q&A) system capable of natural language communication between a computer system and a person. The use of chatbots for 24-hour customer service provides quick responses that solve problems online. This approach is quickly becoming a convenient way for companies to enhance their customer services without location or knowledgeable staff limitations. This research proposes a system framework and develops a prototype virtual reality (VR) enabled transformer mass-customization consultation chatbot. The chatbot technique is a retrieval-based intelligent system. First, thousands of transformer specific frequently asked questions (FAQs) are collected as a Q&A dataset for technical supports retrieval. More than 1.2 million engineering Wikipedia pages and engineering technical papers are used to train a word embedding model used for natural language processing and question-answer retrieval. The chatbot is integrated into a virtual reality (VR) immersive user interface (UI) environment enabling users to make transformer design changes while querying the system about specifications and standards while interacting with 3D models from the company’s knowledge base archive. The system provides two unique UIs for personal computer (PC) and a helmet-based immersive interface. The system supports real-time consultation of mass-customized transformers and their bills of materials (BOM) for design review, analysis and cost estimation.

scholarly journals Investigating the role of Word Embeddings in sentiment analysis

2020 ◽  
pp. 18-24
Author(s):  
Samuele Martinelli ◽  
Gloria Gonella ◽  
Dario Bertolino
Keyword(s):  
Sentiment Analysis ◽  
Customer Service ◽  
Language Processing ◽  
Question Answering ◽  
Classification Task ◽  
Mining Machine ◽  
Computational Techniques ◽  
Human Language ◽  
Simple Classification ◽  
Learning Techniques

During decades, Natural language processing (NLP) expanded its range of tasks, from document classification to automatic text summarization, sentiment analysis, text mining, machine translation, automatic question answering and others. In 2018, T. Young described NLP as a theory-motivated range of computational techniques for the automatic analysis and representation of human language. Outside and before AI, human language has been studied by specialists from various disciplines: linguistics, philosophy, logic, psychology. The aim of this work is to build a neural network to perform a sentiment analysis on Italian reviews from the chatbot customer service. Sentiment analysis is a data mining process which identifies and extracts subjective information from text. It could help to understand the social sentiment of clients, respect a business product or service. It could be a simple classification task that analyses a sentence and tells whether the underlying sentiment is positive or negative. The potentiality of deep learning techniques made this simple classification task evolve, creating new, more complex sentiment analysis, e.g. Intent Analysis and Contextual Semantic Search.

scholarly journals Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge

2022 ◽  
Vol 40 (1) ◽  
pp. 1-33
Author(s):  
Yang Deng ◽  
Yuexiang Xie ◽  
Yaliang Li ◽  
Min Yang ◽  
Wai Lam ◽  
...  
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Question Answering ◽  
Representation Learning ◽  
Attention Mechanism ◽  
Convolutional Network ◽  
Structure Information ◽  
Dialog Systems ◽  
External Knowledge ◽  
Knowledge Based

Answer selection, which is involved in many natural language processing applications, such as dialog systems and question answering (QA), is an important yet challenging task in practice, since conventional methods typically suffer from the issues of ignoring diverse real-world background knowledge. In this article, we extensively investigate approaches to enhancing the answer selection model with external knowledge from knowledge graph (KG). First, we present a context-knowledge interaction learning framework, Knowledge-aware Neural Network, which learns the QA sentence representations by considering a tight interaction with the external knowledge from KG and the textual information. Then, we develop two kinds of knowledge-aware attention mechanism to summarize both the context-based and knowledge-based interactions between questions and answers. To handle the diversity and complexity of KG information, we further propose a Contextualized Knowledge-aware Attentive Neural Network, which improves the knowledge representation learning with structure information via a customized Graph Convolutional Network and comprehensively learns context-based and knowledge-based sentence representation via the multi-view knowledge-aware attention mechanism. We evaluate our method on four widely used benchmark QA datasets, including WikiQA, TREC QA, InsuranceQA, and Yahoo QA. Results verify the benefits of incorporating external knowledge from KG and show the robust superiority and extensive applicability of our method.

scholarly journals Prediction of Answer Keywords using Char-RNN

2019 ◽  
Vol 9 (3) ◽  
pp. 2164
Author(s):  
Pratheek I ◽  
Joy Paulose
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Question Answering ◽  
Context Aware ◽  
Question Answering System ◽  
Novel Approach ◽  
Current Affairs ◽  
Generating Sequences ◽  
Gated Recurrent Unit

<p>Generating sequences of characters using a Recurrent Neural Network (RNN) is a tried and tested method for creating unique and context aware words, and is fundamental in Natural Language Processing tasks. These type of Neural Networks can also be used a question-answering system. The main drawback of most of these systems is that they work from a factoid database of information, and when queried about new and current information, the responses are usually bleak. In this paper, the author proposes a novel approach to finding answer keywords from a given body of news text or headline, based on the query that was asked, where the query would be of the nature of current affairs or recent news, with the use of Gated Recurrent Unit (GRU) variant of RNNs. Thus, this ensures that the answers provided are relevant to the content of query that was put forth.</p>

scholarly journals A multi-label text classification model based on ELMo and attention

2020 ◽  
Vol 309 ◽  
pp. 03015
Author(s):  
Wenbin Liu ◽  
Bojian Wen ◽  
Shang Gao ◽  
Jiesheng Zheng ◽  
Yinlong Zheng
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Attention Mechanism ◽  
Sentiment Classification ◽  
Classification Model ◽  
Data Set ◽  
Model Based ◽  
Related Information ◽  
Related Text ◽  
Common Application

Text classification is a common application in natural language processing. We proposed a multi-label text classification model based on ELMo and attention mechanism which help solve the problem for the sentiment classification task that there is no grammar or writing convention in power supply related text and the sentiment related information disperses in the text. Firstly, we use pre-trained word embedding vector to extract the feature of text from the Internet. Secondly, the analyzed deep information features are weighted according to the attention mechanism. Finally, an improved ELMo model in which we replace the LSTM module with GRU module is used to characterize the text and information is classified. The experimental results on Kaggle’s toxic comment classification data set show that the accuracy of sentiment classification is as high as 98%.

Sign in / Sign up

Export Citation Format

Share Document