Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00358 ◽

2021 ◽

Vol 9 ◽

pp. 120-138

Author(s):

Alireza Mohammadshahi ◽

James Henderson

Keyword(s):

State Of The Art ◽

The State ◽

Iterative Refinement ◽

Dependency Parsing ◽

Arbitrary Graphs ◽

Syntactic Dependency

We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing. We demonstrate the power and effectiveness of RNGTr on several dependency corpora, using a refinement model pre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a non-recursive parser similar to our refinement model. RNGTr can improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks, English and Chinese Penn Treebanks, and the German CoNLL2009 corpus, even improving over the new state-of-the-art results achieved by SynTr, significantly improving the state-of-the-art for all corpora tested.

Download Full-text

Efficient Structured Inference for Transition-Based Parsing with Neural Networks and Error States

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00092 ◽

2016 ◽

Vol 4 ◽

pp. 183-196 ◽

Cited By ~ 1

Author(s):

Ashish Vaswani ◽

Kenji Sagae

Keyword(s):

Neural Networks ◽

State Of The Art ◽

The State ◽

Local Models ◽

Dependency Parsing ◽

New Approach ◽

Local Classification

Transition-based approaches based on local classification are attractive for dependency parsing due to their simplicity and speed, despite producing results slightly below the state-of-the-art. In this paper, we propose a new approach for approximate structured inference for transition-based parsing that produces scores suitable for global scoring using local models. This is accomplished with the introduction of error states in local training, which add information about incorrect derivation paths typically left out completely in locally-trained models. Using neural networks for our local classifiers, our approach achieves 93.61% accuracy for transition-based dependency parsing in English.

Download Full-text

Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00101 ◽

2016 ◽

Vol 4 ◽

pp. 313-327 ◽

Cited By ~ 83

Author(s):

Eliyahu Kiperwasser ◽

Yoav Goldberg

Keyword(s):

State Of The Art ◽

The State ◽

Dependency Parsing ◽

Sentential Context ◽

Feature Vectors ◽

Feature Representations ◽

Effective Scheme ◽

Bidirectional Lstm

We present a simple and effective scheme for dependency parsing which is based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with a BiLSTM vector representing the token in its sentential context, and feature vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is trained jointly with the parser objective, resulting in very effective feature extractors for parsing. We demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser as well as to a globally optimized graph-based parser. The resulting parsers have very simple architectures, and match or surpass the state-of-the-art accuracies on English and Chinese.

Download Full-text

Practical Picture Processing

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100051700 ◽

1974 ◽

Vol 32 ◽

pp. 338-339

Author(s):

T. A. Welton

Keyword(s):

Radiation Damage ◽

Coherence Length ◽

Spatial Information ◽

State Of The Art ◽

Coherent Radiation ◽

The State ◽

Energy Spread ◽

Electron Micrograph ◽

Picture Processing ◽

Molecular Skeleton

Various authors have emphasized the spatial information resident in an electron micrograph taken with adequately coherent radiation. In view of the completion of at least one such instrument, this opportunity is taken to summarize the state of the art of processing such micrographs. We use the usual symbols for the aberration coefficients, and supplement these with £ and 6 for the transverse coherence length and the fractional energy spread respectively. He also assume a weak, biologically interesting sample, with principal interest lying in the molecular skeleton remaining after obvious hydrogen loss and other radiation damage has occurred.

Download Full-text