scholarly journals A Dynamic Emotional Session Generation Model Based on Seq2Seq and a Dictionary-Based Attention Mechanism

2020 ◽  
Vol 10 (6) ◽  
pp. 1967
Author(s):  
Qiangqaing Guo ◽  
Zhenfang Zhu ◽  
Qiang Lu ◽  
Dianyuan Zhang ◽  
Wenqing Wu
Keyword(s):  
Deep Learning ◽  
Emotional Response ◽  
Large Scale ◽  
Attention Mechanism ◽  
Generation Model ◽  
Open Domain ◽  
Dialogue System ◽  
Output Sequence ◽  
Emotional Factors

With the development of deep learning, the method of large-scale dialogue generation based on deep learning has received extensive attention. The current research has aimed to solve the problem of the quality of generated dialogue content, but has failed to fully consider the emotional factors of generated dialogue content. In order to solve the problem of emotional response in the open domain dialogue system, we proposed a dynamic emotional session generation model (DESG). On the basis of the Seq2Seq (sequence-to-sequence) framework, the model abbreviation incorporates a dictionary-based attention mechanism that encourages the substitution of words in response with synonyms in emotion dictionaries. Meanwhile, in order to improve the model, internal emotion regulator and emotion classifier mechanisms are introduced in order to build a large-scale emotion-session generation model. Experimental results show that our DESG model can not only produce an appropriate output sequence in terms of content (related grammar) for a given post and emotion category, but can also express the expected emotional response explicitly or implicitly.

scholarly journals Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification

2019 ◽  
Vol 9 (18) ◽  
pp. 3717 ◽  
Author(s):  
Wenkuan Li ◽  
Dongyuan Li ◽  
Hongxia Yin ◽  
Lindong Zhang ◽  
Zhenfang Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Attention Mechanism ◽  
Sentiment Classification ◽  
Classification Model ◽  
Linguistic Knowledge ◽  
Great Success ◽  
Text Representation ◽  
Attention Network ◽  
Sentiment Lexicon

Text representation learning is an important but challenging issue for various natural language processing tasks. Recently, deep learning-based representation models have achieved great success for sentiment classification. However, these existing models focus on more semantic information rather than sentiment linguistic knowledge, which provides rich sentiment information and plays a key role in sentiment analysis. In this paper, we propose a lexicon-enhanced attention network (LAN) based on text representation to improve the performance of sentiment classification. Specifically, we first propose a lexicon-enhanced attention mechanism by combining the sentiment lexicon with an attention mechanism to incorporate sentiment linguistic knowledge into deep learning methods. Second, we introduce a multi-head attention mechanism in the deep neural network to interactively capture the contextual information from different representation subspaces at different positions. Furthermore, we stack a LAN model to build a hierarchical sentiment classification model for large-scale text. Extensive experiments are conducted to evaluate the effectiveness of the proposed models on four popular real-world sentiment classification datasets at both the sentence level and the document level. The experimental results demonstrate that our proposed models can achieve comparable or better performance than the state-of-the-art methods.

scholarly journals Commonsense Knowledge Aware Conversation Generation with Graph Attention

2018 ◽  
Author(s):  
Hao Zhou ◽  
Tom Young ◽  
Minlie Huang ◽  
Haizhou Zhao ◽  
Jingfang Xu ◽  
...  
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Semantic Information ◽  
Attention Mechanism ◽  
Generation Model ◽  
Dynamic Graph ◽  
Commonsense Knowledge ◽  
Word Generation ◽  
Proposed Model ◽  
Knowledge Graphs

Commonsense knowledge is vital to many natural language processing tasks. In this paper, we present a novel open-domain conversation generation model to demonstrate how large-scale commonsense knowledge can facilitate language understanding and generation. Given a user post, the model retrieves relevant knowledge graphs from a knowledge base and then encodes the graphs with a static graph attention mechanism, which augments the semantic information of the post and thus supports better understanding of the post. Then, during word generation, the model attentively reads the retrieved knowledge graphs and the knowledge triples within each graph to facilitate better generation through a dynamic graph attention mechanism. This is the first attempt that uses large-scale commonsense knowledge in conversation generation. Furthermore, unlike existing models that use knowledge triples (entities) separately and independently, our model treats each knowledge graph as a whole, which encodes more structured, connected semantic information in the graphs. Experiments show that the proposed model can generate more appropriate and informative responses than state-of-the-art baselines. 

Generating Emotional Response by Conditional Variational Auto-Encoder in Open-domain Dialogue System

2021 ◽  
Author(s):  
Mengjuan Liu ◽  
Xiaoming Bao ◽  
Jiang Liu ◽  
Pei Zhao ◽  
Yuchen Shen
Keyword(s):  
Emotional Response ◽  
Open Domain ◽  
Dialogue System

scholarly journals Remote sensing image description based on word embedding and end-to-end deep learning

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yuan Wang ◽  
Hongbing Ma ◽  
Kuerban Alifu ◽  
Yalong Lv
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Large Scale ◽  
Populus Euphratica ◽  
Word Embedding ◽  
Generation Model ◽  
Image Description ◽  
Remote Sensing Images ◽  
End To End ◽  
Occurrence Matrix

AbstractThis study proposes an end-to-end image description generation model based on word embedding technology to realise the classification and identification of Populus euphratica and Tamarix in complex remote sensing images by providing descriptions in precise and concise natural sentences. First, category ambiguity over large-scale regions in remote sensing images is addressed by introducing the co-occurrence matrix and global vectors for word representation to generate the word vector features of the object to be identified. Second, a new multi-level end-to-end model is employed to further describe the content of remote sensing images and to better advance the description tasks for P. euphratica and Tamarix in remote sensing images. Experimental results reveal that the natural language sentences generated using this method can better describe P. euphratica and Tamarix in remote sensing images compared with conventional deep learning methods.

scholarly journals Fingerprint Classification Based on Deep Learning Approaches: Experimental Findings and Comparisons

Symmetry ◽  
2021 ◽  
Vol 13 (5) ◽  
pp. 750
Author(s):  
Carmelo Militello ◽  
Leonardo Rundo ◽  
Salvatore Vitabile ◽  
Vincenzo Conti
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Learning Approaches ◽  
Biometric Recognition ◽  
Fingerprint Classification ◽  
Model Efficiency ◽  
Experimental Findings ◽  
Processing Techniques

Biometric classification plays a key role in fingerprint characterization, especially in the identification process. In fact, reducing the number of comparisons in biometric recognition systems is essential when dealing with large-scale databases. The classification of fingerprints aims to achieve this target by splitting fingerprints into different categories. The general approach of fingerprint classification requires pre-processing techniques that are usually computationally expensive. Deep Learning is emerging as the leading field that has been successfully applied to many areas, such as image processing. This work shows the performance of pre-trained Convolutional Neural Networks (CNNs), tested on two fingerprint databases—namely, PolyU and NIST—and comparisons to other results presented in the literature in order to establish the type of classification that allows us to obtain the best performance in terms of precision and model efficiency, among approaches under examination, namely: AlexNet, GoogLeNet, and ResNet. We present the first study that extensively compares the most used CNN architectures by classifying the fingerprints into four, five, and eight classes. From the experimental results, the best performance was obtained in the classification of the PolyU database by all the tested CNN architectures due to the higher quality of its samples. To confirm the reliability of our study and the results obtained, a statistical analysis based on the McNemar test was performed.

scholarly journals Novel Linguistic Steganography Based on Character-Level Text Generation

Mathematics ◽  
2020 ◽  
Vol 8 (9) ◽  
pp. 1558 ◽  
Author(s):  
Lingyun Xiang ◽  
Shuanghui Yang ◽  
Yuhang Liu ◽  
Qian Li ◽  
Chengzhang Zhu
Keyword(s):  
Large Scale ◽  
Experimental Results ◽  
Selection Strategy ◽  
Generation Model ◽  
Text Generation ◽  
Embedding Capacity ◽  
Secret Information ◽  
Prediction Probability ◽  
Linguistic Steganography

With the development of natural language processing, linguistic steganography has become a research hotspot in the field of information security. However, most existing linguistic steganographic methods may suffer from the low embedding capacity problem. Therefore, this paper proposes a character-level linguistic steganographic method (CLLS) to embed the secret information into characters instead of words by employing a long short-term memory (LSTM) based language model. First, the proposed method utilizes the LSTM model and large-scale corpus to construct and train a character-level text generation model. Through training, the best evaluated model is obtained as the prediction model of generating stego text. Then, we use the secret information as the control information to select the right character from predictions of the trained character-level text generation model. Thus, the secret information is hidden in the generated text as the predicted characters having different prediction probability values can be encoded into different secret bit values. For the same secret information, the generated stego texts vary with the starting strings of the text generation model, so we design a selection strategy to find the highest quality stego text from a number of candidate stego texts as the final stego text by changing the starting strings. The experimental results demonstrate that compared with other similar methods, the proposed method has the fastest running speed and highest embedding capacity. Moreover, extensive experiments are conducted to verify the effect of the number of candidate stego texts on the quality of the final stego text. The experimental results show that the quality of the final stego text increases with the number of candidate stego texts increasing, but the growth rate of the quality will slow down.

scholarly journals Pedestrian Reidentification Algorithm Based on Deconvolution Network Feature Extraction-Multilayer Attention Mechanism Convolutional Neural Network

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Feng-Ping An ◽  
Jun-e Liu ◽  
Lei Bai
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Deep Learning ◽  
Information Transfer ◽  
Large Scale ◽  
Recognition Accuracy ◽  
Attention Mechanism ◽  
Convolution Kernel ◽  
Network Feature ◽  
Adaptive Ability

Pedestrian reidentification is a key technology in large-scale distributed camera systems. It can quickly and efficiently detect and track target people in large-scale distributed surveillance networks. The existing traditional pedestrian reidentification methods have problems such as low recognition accuracy, low calculation efficiency, and weak adaptive ability. Pedestrian reidentification algorithms based on deep learning have been widely used in the field of pedestrian reidentification due to their strong adaptive ability and high recognition accuracy. However, the pedestrian recognition method based on deep learning has the following problems: first, during the learning process of the deep learning model, the initial value of the convolution kernel is usually randomly assigned, which makes the model learning process easily fall into a local optimum. The second is that the model parameter learning method based on the gradient descent method exhibits gradient dispersion. The third is that the information transfer of pedestrian reidentification sequence images is not considered. In view of these issues, this paper first examines the feature map matrix from the original image through a deconvolution neural network, uses it as a convolution kernel, and then performs layer-by-layer convolution and pooling operations. Then, the second derivative information of the error function is directly obtained without calculating the Hessian matrix, and the momentum coefficient is used to improve the convergence of the backpropagation, thereby suppressing the gradient dispersion phenomenon. At the same time, to solve the problem of information transfer of pedestrian reidentification sequence images, this paper proposes a memory network model based on a multilayer attention mechanism, which uses the network to effectively store image visual information and pedestrian behavior information, respectively. It can solve the problem of information transmission. Based on the above ideas, this paper proposes a pedestrian reidentification algorithm based on deconvolution network feature extraction-multilayer attention mechanism convolutional neural network. Experiments are performed on the related data sets using this algorithm and other major popular human reidentification algorithms. The results show that the pedestrian reidentification method proposed in this paper not only has strong adaptive ability but also has significantly improved average recognition accuracy and rank-1 matching rate compared with other mainstream methods.

scholarly journals Social Multimedia Security and Suspicious Activity Detection in SDN using Hybrid Deep Learning Technique

2020 ◽  
Vol 2 (2) ◽  
pp. 108-115
Author(s):  
Dr. Joy Iong Zong Chen ◽  
Dr. Smys S.
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Large Scale ◽  
Multimedia Data ◽  
Support Vector ◽  
Data Delivery ◽  
Energy Aware ◽  
Detection Scheme ◽  
Social Multimedia

Social multimedia traffic is growing exponentially with the increased usage and continuous development of services and applications based on multimedia. Quality of Service (QoS), Quality of Information (QoI), scalability, reliability and such factors that are essential for social multimedia networks are realized by secure data transmission. For delivering actionable and timely insights in order to meet the growing demands of the user, multimedia analytics is performed by means of a trust-based paradigm. Efficient management and control of the network is facilitated by limiting certain capabilities such as energy-aware networking and runtime security in Software Defined Networks. In social multimedia context, suspicious flow detection is performed by a hybrid deep learning based anomaly detection scheme in order to enhance the SDN reliability. The entire process is divided into two modules namely – Abnormal activities detection using support vector machine based on Gradient descent and improved restricted Boltzmann machine which facilitates the anomaly detection module, and satisfying the strict requirements of QoS like low latency and high bandwidth in SDN using end-to-end data delivery module. In social multimedia, data delivery and anomaly detection services are essential in order to improve the efficiency and effectiveness of the system. For this purpose, we use benchmark datasets as well as real time evaluation to experimentally evaluate the proposed scheme. Detection of malicious events like confidential data collection, profile cloning and identity theft are performed to analyze the performance of the system using CMU-based insider threat dataset for large scale analysis.

scholarly journals Revisiting Text Guide, a Truncation Method for Long Text Classification

2021 ◽  
Vol 11 (18) ◽  
pp. 8554
Author(s):  
Krzysztof Fiok ◽  
Waldemar Karwowski ◽  
Edgar Gutierrez ◽  
Mohammad Reza Davahli ◽  
Maciej Wilamowski ◽  
...  
Keyword(s):  
Deep Learning ◽  
Text Classification ◽  
Computational Cost ◽  
Attention Mechanism ◽  
Common Method ◽  
Improve Performance ◽  
Truncation Method ◽  
High Computational Cost ◽  
Naive Approach

The quality of text classification has greatly improved with the introduction of deep learning, and more recently, models using attention mechanism. However, to address the problem of classifying text instances that are longer than the length limit adopted by most of the best performing transformer models, the most common method is to naively truncate the text so that it meets the model limit. Researchers have proposed other approaches, but they do not appear to be popular, because of their high computational cost and implementation complexity. Recently, another method called Text Guide has been proposed, which allows for text truncation that outperforms the naive approach and simultaneously is less complex and costly than earlier proposed solutions. Our study revisits Text Guide by testing the influence of certain modifications on the method’s performance. We found that some aspects of the method can be altered to further improve performance and confirmed several assumptions regarding the dependence of the method’s quality on certain factors.

scholarly journals Conversational Intelligence Challenge: Accelerating Research with Crowd Science and Open Source

AI Magazine ◽  
2020 ◽  
Vol 41 (3) ◽  
pp. 18-27
Author(s):  
Mikhail Burtsev ◽  
Varvara Logacheva
Keyword(s):  
Open Source ◽  
Language Processing ◽  
Large Scale ◽  
Training Data ◽  
Automatic Evaluation ◽  
Dialogue Systems ◽  
Open Domain ◽  
Short Text ◽  
Human Evaluation

Development of conversational systems is one of the most challenging tasks in natural language processing, and it is especially hard in the case of open-domain dialogue. The main factors that hinder progress in this area are lack of training data and difficulty of automatic evaluation. Thus, to reliably evaluate the quality of such models, one needs to resort to time-consuming and expensive human evaluation. We tackle these problems by organizing the Conversational Intelligence Challenge (ConvAI) — open competition of dialogue systems. Our goals are threefold: to work out a good design for human evaluation of open-domain dialogue, to grow open-source code base for conversational systems, and to harvest and publish new datasets. Over the course of ConvAI1 and ConvAI2 competitions, we developed a framework for evaluation of chatbots in messaging platforms and used it to evaluate over 30 dialogue systems in two conversational tasks — discussion of short text snippets from Wikipedia and personalized small talk. These large-scale evaluation experiments were performed by recruiting volunteers as well as paid workers. As a result, we succeeded in collecting a dataset of around 5,000 long meaningful human-to-bot dialogues and got many insights into the organization of human evaluation. This dataset can be used to train an automatic evaluation model or to improve the quality of dialogue systems. Our analysis of ConvAI1 and ConvAI2 competitions shows that the future work in this area should be centered around the more active participation of volunteers in the assessment of dialogue systems. To achieve that, we plan to make the evaluation setup more engaging.

Sign in / Sign up

Export Citation Format

Share Document