scholarly journals Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations

2016 ◽  
Vol 4 ◽  
pp. 313-327 ◽  
Author(s):  
Eliyahu Kiperwasser ◽  
Yoav Goldberg
Keyword(s):  
State Of The Art ◽  
The State ◽  
Dependency Parsing ◽  
Sentential Context ◽  
Feature Vectors ◽  
Feature Representations ◽  
Effective Scheme ◽  
Bidirectional Lstm

We present a simple and effective scheme for dependency parsing which is based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with a BiLSTM vector representing the token in its sentential context, and feature vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is trained jointly with the parser objective, resulting in very effective feature extractors for parsing. We demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser as well as to a globally optimized graph-based parser. The resulting parsers have very simple architectures, and match or surpass the state-of-the-art accuracies on English and Chinese.

scholarly journals TET-GAN: Text Effects Transfer via Stylization and Destylization

2019 ◽  
Vol 33 ◽  
pp. 1238-1245 ◽  
Author(s):  
Shuai Yang ◽  
Jiaying Liu ◽  
Wenjing Wang ◽  
Zongming Guo
Keyword(s):  
Network Design ◽  
State Of The Art ◽  
The State ◽  
High Quality ◽  
Style Transfer ◽  
Feature Representations ◽  
Art Methods ◽  
Matching Criteria

Text effects transfer technology automatically makes the text dramatically more impressive. However, previous style transfer methods either study the model for general style, which cannot handle the highly-structured text effects along the glyph, or require manual design of subtle matching criteria for text effects. In this paper, we focus on the use of the powerful representation abilities of deep neural features for text effects transfer. For this purpose, we propose a novel Texture Effects Transfer GAN (TET-GAN), which consists of a stylization subnetwork and a destylization subnetwork. The key idea is to train our network to accomplish both the objective of style transfer and style removal, so that it can learn to disentangle and recombine the content and style features of text effects images. To support the training of our network, we propose a new text effects dataset with as much as 64 professionally designed styles on 837 characters. We show that the disentangled feature representations enable us to transfer or remove all these styles on arbitrary glyphs using one network. Furthermore, the flexible network design empowers TET-GAN to efficiently extend to a new text style via oneshot learning where only one example is required. We demonstrate the superiority of the proposed method in generating high-quality stylized text over the state-of-the-art methods.

scholarly journals Efficient Structured Inference for Transition-Based Parsing with Neural Networks and Error States

2016 ◽  
Vol 4 ◽  
pp. 183-196 ◽  
Author(s):  
Ashish Vaswani ◽  
Kenji Sagae
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
The State ◽  
Local Models ◽  
Dependency Parsing ◽  
New Approach ◽  
Local Classification

Transition-based approaches based on local classification are attractive for dependency parsing due to their simplicity and speed, despite producing results slightly below the state-of-the-art. In this paper, we propose a new approach for approximate structured inference for transition-based parsing that produces scores suitable for global scoring using local models. This is accomplished with the introduction of error states in local training, which add information about incorrect derivation paths typically left out completely in locally-trained models. Using neural networks for our local classifiers, our approach achieves 93.61% accuracy for transition-based dependency parsing in English.

scholarly journals Relation-Guided Spatial Attention and Temporal Refinement for Video-Based Person Re-Identification

2020 ◽  
Vol 34 (07) ◽  
pp. 11434-11441
Author(s):  
Xingze Li ◽  
Wengang Zhou ◽  
Yun Zhou ◽  
Houqiang Li
Keyword(s):  
Spatial Attention ◽  
Video Surveillance ◽  
State Of The Art ◽  
The State ◽  
Temporal Information ◽  
Feature Representations ◽  
Art Performance ◽  
The Individual ◽  
Significant Application

Video-based person re-identification has received considerable attention in recent years due to its significant application in video surveillance. Compared with image-based person re-identification, video-based person re-identification is characterized by a much richer context, which raises the significance of identifying informative regions and fusing the temporal information across frames. In this paper, we propose two relation-guided modules to learn reinforced feature representations for effective re-identification. First, a relation-guided spatial attention (RGSA) module is designed to explore the discriminative regions globally. The weight at each position is determined by its feature as well as the relation features from other positions, revealing the dependence between local and global contents. Based on the adaptively weighted frame-level feature, then, a relation-guided temporal refinement (RGTR) module is proposed to further refine the feature representations across frames. The learned relation information via the RGTR module enables the individual frames to complement each other in an aggregation manner, leading to robust video-level feature representations. Extensive experiments on four prevalent benchmarks verify the state-of-the-art performance of the proposed method.

scholarly journals An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 12613-12620 ◽  
Author(s):  
Jihan Yang ◽  
Ruijia Xu ◽  
Ruiyu Li ◽  
Xiaojuan Qi ◽  
Xiaoyong Shen ◽  
...  
Keyword(s):  
Marginal Distribution ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Feature Space ◽  
Semantic Segmentation ◽  
The State ◽  
Object Size ◽  
Feature Maps ◽  
Feature Representations ◽  
Unsupervised Domain Adaptation

We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 → Cityscapes and SYNTHIA → Cityscapes.

scholarly journals Are Noisy Sentences Useless for Distant Supervised Relation Extraction?

2020 ◽  
Vol 34 (05) ◽  
pp. 8799-8806
Author(s):  
Yuming Shang ◽  
He-Yan Huang ◽  
Xian-Ling Mao ◽  
Xin Sun ◽  
Wei Wei
Keyword(s):  
Selective Attention ◽  
State Of The Art ◽  
Relation Extraction ◽  
The State ◽  
Benchmark Dataset ◽  
Experimental Results ◽  
Training Data ◽  
High Confidence ◽  
Feature Representations ◽  
Novel Method

The noisy labeling problem has been one of the major obstacles for distant supervised relation extraction. Existing approaches usually consider that the noisy sentences are useless and will harm the model's performance. Therefore, they mainly alleviate this problem by reducing the influence of noisy sentences, such as applying bag-level selective attention or removing noisy sentences from sentence-bags. However, the underlying cause of the noisy labeling problem is not the lack of useful information, but the missing relation labels. Intuitively, if we can allocate credible labels for noisy sentences, they will be transformed into useful training data and benefit the model's performance. Thus, in this paper, we propose a novel method for distant supervised relation extraction, which employs unsupervised deep clustering to generate reliable labels for noisy sentences. Specifically, our model contains three modules: a sentence encoder, a noise detector and a label generator. The sentence encoder is used to obtain feature representations. The noise detector detects noisy sentences from sentence-bags, and the label generator produces high-confidence relation labels for noisy sentences. Extensive experimental results demonstrate that our model outperforms the state-of-the-art baselines on a popular benchmark dataset, and can indeed alleviate the noisy labeling problem.

scholarly journals Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement

2021 ◽  
Vol 9 ◽  
pp. 120-138
Author(s):  
Alireza Mohammadshahi ◽  
James Henderson
Keyword(s):  
State Of The Art ◽  
The State ◽  
Iterative Refinement ◽  
Dependency Parsing ◽  
Arbitrary Graphs ◽  
Syntactic Dependency

We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing. We demonstrate the power and effectiveness of RNGTr on several dependency corpora, using a refinement model pre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a non-recursive parser similar to our refinement model. RNGTr can improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks, English and Chinese Penn Treebanks, and the German CoNLL2009 corpus, even improving over the new state-of-the-art results achieved by SynTr, significantly improving the state-of-the-art for all corpora tested.

scholarly journals Protein active site prediction for early drug discovery and designing

2021 ◽  
Vol 13 (1) ◽  
pp. 98-105
Author(s):  
Aqsa Yousaf ◽  
Tahira Shehzadi ◽  
Aqeel Farooq ◽  
Komal Ilyas
Keyword(s):  
Active Sites ◽  
State Of The Art ◽  
Research Work ◽  
Living Cells ◽  
The State ◽  
Feature Vectors ◽  
Protein Active Sites ◽  
A Cell ◽  
Scoring Matrix ◽  
Living Organisms

Abstract Adenosine triphosphate (ATP) is an energy compound present in living organisms and is required by living cells for performing operations such as replication, molecules transportation, chemical synthesis, etc. ATP connects with living cells through specialized sites called ATP-sites. ATP-sites are present in various proteins of a living cell. The life span of a cell can be controlled by controlling ATP compounds and without the provision of energy to ATP compounds, cells cannot survive. Countless diseases treatment (such as cancer, diabetes) can be possible once protein active sites are predicted. Considering the need for an algorithm that predicts ATP-sites with higher accuracy and effectiveness, this research work predicts protein ATP sites in a very novel way. Till now Position-specific scoring matrix (PSSM) along with many physicochemical properties have been used as features with deep neural networks in order to create a model that predicts the ATP-sites. To overcome this problem of complex computation, this exertion proposes k-mer feature vectors with simple machine learning (ML) models to attain the same or even better performance with less computation required. Using 2-mer as feature vectors, this research work trained and tested five different models including KNN, Conv1D, XGBoost, SVM and Random Forest. SVM gave the best performance on k-mer features. The accuracy of the created model is 96%, MCC 90% and ROC-AUC is 99%, which are the same or even better in some aspects than the state-of-the-art results. The state-of-the-art results have an accuracy of 97%, MCC 78% and ROC-AUC is 92%. One of the benefits of the created model is that it is much simpler and more accurate.

scholarly journals A Dual Input-aware Factorization Machine for CTR Prediction

2020 ◽  
Author(s):  
Wantong Lu ◽  
Yantao Yu ◽  
Yongzhe Chang ◽  
Zhen Wang ◽  
Chenhui Li ◽  
...  
Keyword(s):  
Real World ◽  
Predictive Power ◽  
State Of The Art ◽  
Model Parameters ◽  
Feature Vectors ◽  
Feature Representations ◽  
Factorization Machine ◽  
Click Through Rate ◽  
Novel Model ◽  
Original Feature

Factorization Machines (FMs) refer to a class of general predictors working with real valued feature vectors, which are well-known for their ability to estimate model parameters under significant sparsity and have found successful applications in many areas such as the click-through rate (CTR) prediction. However, standard FMs only produce a single fixed representation for each feature across different input instances, which may limit the CTR model’s expressive and predictive power. Inspired by the success of Input-aware Factorization Machines (IFMs), which aim to learn more flexible and informative representations of a given feature according to different input instances, we propose a novel model named Dual Input-aware Factorization Machines (DIFMs) that can adaptively reweight the original feature representations at the bit-wise and vector-wise levels simultaneously. Furthermore, DIFMs strategically integrate various components including Multi-Head Self-Attention, Residual Networks and DNNs into a unified end-to-end model. Comprehensive experiments on two real-world CTR prediction datasets show that the DIFM model can outperform several state-of-the-art models consistently.

scholarly journals Deep Bidirectional LSTM Network Learning-Based Sentiment Analysis for Arabic Text

2020 ◽  
Vol 30 (1) ◽  
pp. 395-412
Author(s):  
Hanane Elfaik ◽  
El Habib Nfaoui
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Computational Linguistics ◽  
State Of The Art ◽  
Contextual Information ◽  
The State ◽  
Great Success ◽  
Bidirectional Lstm ◽  
Lstm Network ◽  
Arabic Sentiment Analysis

Abstract Sentiment analysis aims to predict sentiment polarities (positive, negative or neutral) of a given piece of text. It lies at the intersection of many fields such as Natural Language Processing (NLP), Computational Linguistics, and Data Mining. Sentiments can be expressed explicitly or implicitly. Arabic Sentiment Analysis presents a challenge undertaking due to its complexity, ambiguity, various dialects, the scarcity of resources, the morphological richness of the language, the absence of contextual information, and the absence of explicit sentiment words in an implicit piece of text. Recently, deep learning has obviously shown a great success in the field of sentiment analysis and is considered as the state-of-the-art model in Arabic Sentiment Analysis. However, the state-of-the-art accuracy for Arabic sentiment analysis still needs improvements regarding contextual information and implicit sentiment expressed in different real cases. In this paper, an efficient Bidirectional LSTM Network (BiLSTM) is investigated to enhance Arabic Sentiment Analysis, by applying Forward-Backward encapsulate contextual information from Arabic feature sequences. The experimental results on six benchmark sentiment analysis datasets demonstrate that our model achieves significant improvements over the state-of-art deep learning models and the baseline traditional machine learning methods.

Sign in / Sign up

Export Citation Format

Share Document