scholarly journals Chinese Event Extraction Based on Attention and Semantic Features: A Bidirectional Circular Neural Network

2018 ◽  
Vol 10 (10) ◽  
pp. 95 ◽  
Author(s):  
Yue Wu ◽  
Junyi Zhang
Keyword(s):  
Neural Network ◽  
Attention Mechanism ◽  
Semantic Feature ◽  
Word Embedding ◽  
Event Extraction ◽  
Semantic Features ◽  
Improve Performance

Chinese event extraction uses word embedding to capture similarity, but suffers when handling previously unseen or rare words. From the test, we know that characters may provide some information that we cannot obtain in words, so we propose a novel architecture for combining word representations: character–word embedding based on attention and semantic features. By using an attention mechanism, our method is able to dynamically decide how much information to use from word or character level embedding. With the semantic feature, we can obtain some more information about a word from the sentence. We evaluate different methods on the CEC Corpus, and this method is found to improve performance.

scholarly journals A Novel Approach for Malicious URL Detection Based on the Joint Model

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
JianTing Yuan ◽  
YiPeng Liu ◽  
Long Yu
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Attention Mechanism ◽  
Semantic Features ◽  
Uniform Resource Locator ◽  
Word Level ◽  
Network Algorithm ◽  
Model Combining ◽  
Novel Approach ◽  
Neural Network Algorithm

The number of malicious websites is increasing yearly, and many companies and individuals worldwide have suffered losses. Therefore, the detection of malicious websites is a task that needs continuous development. In this study, a joint neural network algorithm model combining the attention mechanism, bidirectional independent recurrent neural network (Bi-IndRNN), and capsule network (CapsNet) is proposed. The word vector tool word2vec trains the character- and word-level uniform resource locator (URL) static embedding vector features. At the same time, the algorithm will also extract texture fingerprint features that can compare the content differences of different malicious web URL binary files. Then, the extracted features are fused and input into the joint neural network algorithm model. First, the multihead attention mechanism is used to extract contextual semantic features by adjusting weights and Bi-IndRNN. Second, CapsNet with dynamic routing is used to extract deep semantic information. Finally, the sigmoid classifier is used for classification. This study uses different methods from different angles to extract more comprehensive features. From the experimental results, the method proposed in this study improves the classification accuracy of malicious web page detection compared with other researchers.

scholarly journals A Multichannel Model for Microbial Key Event Extraction Based on Feature Fusion and Attention Mechanism

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Peng Li ◽  
Qian Wang
Keyword(s):  
Sentiment Analysis ◽  
Feature Fusion ◽  
Attention Mechanism ◽  
Event Extraction ◽  
Semantic Features ◽  
Analysis Model ◽  
Global Features ◽  
Public Health Emergencies ◽  
Multimodal Information ◽  
Local And Global Features

In order to further mine the deep semantic information of the microbial text of public health emergencies, this paper proposes a multichannel microbial sentiment analysis model MCMF-A. Firstly, we use word2vec and fastText to generate word vectors in the feature vector embedding layer and fuse them with lexical and location feature vectors; secondly, we build a multichannel layer based on CNN and BiLSTM to extract local and global features of the microbial text; then we build an attention mechanism layer to extract the important semantic features of the microbial text; thirdly, we merge the multichannel output in the fusion layer and use soft; finally, the results are merged in the fusion layer, and a surtax function is used in the output layer for sentiment classification. The results show that the F1 value of the MCMF-A sentiment analysis model reaches 90.21%, which is 9.71% and 9.14% higher than the benchmark CNN and BiLSTM models, respectively. The constructed dataset is small in size, and the multimodal information such as images and speech has not been considered.

scholarly journals Attention-Mechanism-Containing Neural Networks for High-Resolution Remote Sensing Image Classification

2018 ◽  
Vol 10 (10) ◽  
pp. 1602 ◽  
Author(s):  
Rudong Xu ◽  
Yiting Tao ◽  
Zhongyuan Lu ◽  
Yanfei Zhong
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Feature Extraction ◽  
High Resolution ◽  
Deep Neural Network ◽  
Remote Sensing Image ◽  
Attention Mechanism ◽  
Semantic Features ◽  
Feature Maps ◽  
Class Differences

A deep neural network is suitable for remote sensing image pixel-wise classification because it effectively extracts features from the raw data. However, remote sensing images with higher spatial resolution exhibit smaller inter-class differences and greater intra-class differences; thus, feature extraction becomes more difficult. The attention mechanism, as a method that simulates the manner in which humans comprehend and perceive images, is useful for the quick and accurate acquisition of key features. In this study, we propose a novel neural network that incorporates two kinds of attention mechanisms in its mask and trunk branches; i.e., control gate (soft) and feedback attention mechanisms, respectively, based on the branches’ primary roles. Thus, a deep neural network can be equipped with an attention mechanism to perform pixel-wise classification for very high-resolution remote sensing (VHRRS) images. The control gate attention mechanism in the mask branch is utilized to build pixel-wise masks for feature maps, to assign different priorities to different locations on different channels for feature extraction recalibration, to apply stress to the effective features, and to weaken the influence of other profitless features. The feedback attention mechanism in the trunk branch allows for the retrieval of high-level semantic features. Hence, additional aids are provided for lower layers to re-weight the focus and to re-update higher-level feature extraction in a target-oriented manner. These two attention mechanisms are fused to form a neural network module. By stacking various modules with different-scale mask branches, the network utilizes different attention-aware features under different local spatial structures. The proposed method is tested on the VHRRS images from the BJ-02, GF-02, Geoeye, and Quickbird satellites, and the influence of the network structure and the rationality of the network design are discussed. Compared with other state-of-the-art methods, our proposed method achieves competitive accuracy, thereby proving its effectiveness.

scholarly journals An Efficient Module for Instance Segmentation Based on Multi-Level Features and Attention Mechanisms

2021 ◽  
Vol 11 (3) ◽  
pp. 968
Author(s):  
Yingchun Sun ◽  
Wang Gao ◽  
Shuguo Pan ◽  
Tao Zhao ◽  
Yahui Peng
Keyword(s):  
Feature Extraction ◽  
Spatial Structure ◽  
Semantic Feature ◽  
Semantic Features ◽  
Segmentation Method ◽  
Spatial Dimensions ◽  
Feature Pyramid ◽  
Multi Level ◽  
High Level ◽  
Instance Segmentation

Recently, multi-level feature networks have been extensively used in instance segmentation. However, because not all features are beneficial to instance segmentation tasks, the performance of networks cannot be adequately improved by synthesizing multi-level convolutional features indiscriminately. In order to solve the problem, an attention-based feature pyramid module (AFPM) is proposed, which integrates the attention mechanism on the basis of a multi-level feature pyramid network to efficiently and pertinently extract the high-level semantic features and low-level spatial structure features; for instance, segmentation. Firstly, we adopt a convolutional block attention module (CBAM) into feature extraction, and sequentially generate attention maps which focus on instance-related features along the channel and spatial dimensions. Secondly, we build inter-dimensional dependencies through a convolutional triplet attention module (CTAM) in lateral attention connections, which is used to propagate a helpful semantic feature map and filter redundant informative features irrelevant to instance objects. Finally, we construct branches for feature enhancement to strengthen detailed information to boost the entire feature hierarchy of the network. The experimental results on the Cityscapes dataset manifest that the proposed module outperforms other excellent methods under different evaluation metrics and effectively upgrades the performance of the instance segmentation method.

Sign in / Sign up

Export Citation Format

Share Document