scholarly journals Hierarchical Transfer Learning Architecture for Low-Resource Neural Machine Translation

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 154157-154166 ◽  
Author(s):  
Gongxu Luo ◽  
Yating Yang ◽  
Yang Yuan ◽  
Zhanheng Chen ◽  
Aizimaiti Ainiwaer
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Neural Machine Translation ◽  
Low Resource

scholarly journals A Joint Back-Translation and Transfer Learning Method for Low-Resource Neural Machine Translation

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Gong-Xu Luo ◽  
Ya-Ting Yang ◽  
Rui Dong ◽  
Yan-Hong Chen ◽  
Wen-Bo Zhang
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Large Scale ◽  
Data Augmentation ◽  
Training Methods ◽  
Learning Method ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
Back Translation

Neural machine translation (NMT) for low-resource languages has drawn great attention in recent years. In this paper, we propose a joint back-translation and transfer learning method for low-resource languages. It is widely recognized that data augmentation methods and transfer learning methods are both straight forward and effective ways for low-resource problems. However, existing methods, which utilize one of these methods alone, limit the capacity of NMT models for low-resource problems. In order to make full use of the advantages of existing methods and further improve the translation performance of low-resource languages, we propose a new method to perfectly integrate the back-translation method with mainstream transfer learning architectures, which can not only initialize the NMT model by transferring parameters of the pretrained models, but also generate synthetic parallel data by translating large-scale monolingual data of the target side to boost the fluency of translations. We conduct experiments to explore the effectiveness of the joint method by incorporating back-translation into the parent-child and the hierarchical transfer learning architecture. In addition, different preprocessing and training methods are explored to get better performance. Experimental results on Uygur-Chinese and Turkish-English translation demonstrate the superiority of the proposed method over the baselines that use single methods.

scholarly journals Transfer Learning for Low-Resource Neural Machine Translation

2016 ◽  
Author(s):  
Barret Zoph ◽  
Deniz Yuret ◽  
Jonathan May ◽  
Kevin Knight
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Neural Machine Translation ◽  
Low Resource

scholarly journals Machine Translation in Low-Resource Languages by an Adversarial Neural Network

2021 ◽  
Vol 11 (22) ◽  
pp. 10860
Author(s):  
Mengtao Sun ◽  
Hao Wang ◽  
Mark Pasquine ◽  
Ibrahim A. Hameed
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Grammatical Structure ◽  
Neural Machine Translation ◽  
Low Resource ◽  
High Resource ◽  
Learning Techniques ◽  
Good Potential ◽  
Target Languages ◽  
Cross Lingual

Existing Sequence-to-Sequence (Seq2Seq) Neural Machine Translation (NMT) shows strong capability with High-Resource Languages (HRLs). However, this approach poses serious challenges when processing Low-Resource Languages (LRLs), because the model expression is limited by the training scale of parallel sentence pairs. This study utilizes adversary and transfer learning techniques to mitigate the lack of sentence pairs in LRL corpora. We propose a new Low resource, Adversarial, Cross-lingual (LAC) model for NMT. In terms of the adversary technique, LAC model consists of a generator and discriminator. The generator is a Seq2Seq model that produces the translations from source to target languages, while the discriminator measures the gap between machine and human translations. In addition, we introduce transfer learning on LAC model to help capture the features in rare resources because some languages share the same subject-verb-object grammatical structure. Rather than using the entire pretrained LAC model, we separately utilize the pretrained generator and discriminator. The pretrained discriminator exhibited better performance in all experiments. Experimental results demonstrate that the LAC model achieves higher Bilingual Evaluation Understudy (BLEU) scores and has good potential to augment LRL translations.

scholarly journals Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8854-8861 ◽  
Author(s):  
Aditya Siddhant ◽  
Melvin Johnson ◽  
Henry Tsai ◽  
Naveen Ari ◽  
Jason Riesa ◽  
...  
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Single Model ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Sequence Labeling ◽  
Transfer Capability ◽  
Learning Scenarios ◽  
The Cross ◽  
Cross Lingual

The recently proposed massively multilingual neural machine translation (NMT) system has been shown to be capable of translating over 100 languages to and from English within a single model (Aharoni, Johnson, and Firat 2019). Its improved translation performance on low resource languages hints at potential cross-lingual transfer capability for downstream tasks. In this paper, we evaluate the cross-lingual effectiveness of representations from the encoder of a massively multilingual NMT model on 5 downstream classification and sequence labeling tasks covering a diverse set of over 50 languages. We compare against a strong baseline, multilingual BERT (mBERT) (Devlin et al. 2018), in different cross-lingual transfer learning scenarios and show gains in zero-shot transfer in 4 out of these 5 tasks.

Neural machine translation with a polysynthetic low resource language

2020 ◽  
Vol 34 (4) ◽  
pp. 325-346
Author(s):  
John E. Ortega ◽  
Richard Castro Mamani ◽  
Kyunghyun Cho
Keyword(s):  
Machine Translation ◽  
Neural Machine Translation ◽  
Low Resource

Analyzing Subword Techniques to Improve English to Sinhala Neural Machine Translation

2021 ◽  
pp. 2050017
Author(s):  
Rashmini Naranpanawa ◽  
Ravinga Perera ◽  
Thilakshi Fonseka ◽  
Uthayasanker Thayasivam
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Rare Word ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Word Level ◽  
Morphologically Rich Languages

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

Improving thai-lao neural machine translation with similarity lexicon

2021 ◽  
pp. 1-10
Author(s):  
Zhiqiang Yu ◽  
Yuxin Huang ◽  
Junjun Guo
Keyword(s):  
Machine Translation ◽  
Semantic Information ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Translation Quality ◽  
Decoder Architecture ◽  
Baseline System ◽  
Input Sentence ◽  
Resource Conditions ◽  
Language Pair

It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions. Thai-Lao is a typical low-resource language pair of tiny parallel corpus, leading to suboptimal NMT performance on it. However, Thai and Lao have considerable similarities in linguistic morphology and have bilingual lexicon which is relatively easy to obtain. To use this feature, we first build a bilingual similarity lexicon composed of pairs of similar words. Then we propose a novel NMT architecture to leverage the similarity between Thai and Lao. Specifically, besides the prevailing sentence encoder, we introduce an extra similarity lexicon encoder into the conventional encoder-decoder architecture, by which the semantic information carried by the similarity lexicon can be represented. We further provide a simple mechanism in the decoder to balance the information representations delivered from the input sentence and the similarity lexicon. Our approach can fully exploit linguistic similarity carried by the similarity lexicon to improve translation quality. Experimental results demonstrate that our approach achieves significant improvements over the state-of-the-art Transformer baseline system and previous similar works.

Improved neural machine translation for low-resource English–Assamese pair

2021 ◽  
pp. 1-12
Author(s):  
Sahinur Rahman Laskar ◽  
Abdullah Faiz Ur Rahman Khilji ◽  
Partha Pakray ◽  
Sivaji Bandyopadhyay
Keyword(s):  
Machine Translation ◽  
Data Augmentation ◽  
Language Translation ◽  
Linguistically Diverse ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
The World ◽  
Translation Accuracy ◽  
Vocabulary Problems

Language translation is essential to bring the world closer and plays a significant part in building a community among people of different linguistic backgrounds. Machine translation dramatically helps in removing the language barrier and allows easier communication among linguistically diverse communities. Due to the unavailability of resources, major languages of the world are accounted as low-resource languages. This leads to a challenging task of automating translation among various such languages to benefit indigenous speakers. This article investigates neural machine translation for the English–Assamese resource-poor language pair by tackling insufficient data and out-of-vocabulary problems. We have also proposed an approach of data augmentation-based NMT, which exploits synthetic parallel data and shows significantly improved translation accuracy for English-to-Assamese and Assamese-to-English translation and obtained state-of-the-art results.

Sign in / Sign up

Export Citation Format

Share Document