scholarly journals Improving Candidate Generation for Low-resource Cross-lingual Entity Linking

2020 ◽  
Vol 8 ◽  
pp. 109-124
Author(s):  
Shuyan Zhou ◽  
Shruti Rijhwani ◽  
John Wieting ◽  
Jaime Carbonell ◽  
Graham Neubig
Keyword(s):  
State Of The Art ◽  
Target Language ◽  
Entity Linking ◽  
Average Gain ◽  
Source Language ◽  
Low Resource ◽  
High Resource ◽  
Language Knowledge ◽  
Cross Lingual ◽  
Improved Model

Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts. The first step of (X)EL is candidate generation, which retrieves a list of plausible candidate entities from the target-language KB for each mention. Approaches based on resources from Wikipedia have proven successful in the realm of relatively high-resource languages, but these do not extend well to low-resource languages with few, if any, Wikipedia pages. Recently, transfer learning methods have been shown to reduce the demand for resources in the low-resource languages by utilizing resources in closely related languages, but the performance still lags far behind their high-resource counterparts. In this paper, we first assess the problems faced by current entity candidate generation methods for low-resource XEL, then propose three improvements that (1) reduce the disconnect between entity mentions and KB entries, and (2) improve the robustness of the model to low-resource scenarios. The methods are simple, but effective: We experiment with our approach on seven XEL datasets and find that they yield an average gain of 16.9% in Top-30 gold candidate recall, compared with state-of-the-art baselines. Our improved model also yields an average gain of 7.9% in in-KB accuracy of end-to-end XEL. 1

scholarly journals Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

2018 ◽  
Vol 6 ◽  
pp. 557-570 ◽  
Author(s):  
Xilun Chen ◽  
Yu Sun ◽  
Ben Athiwaratkun ◽  
Claire Cardie ◽  
Kilian Weinberger
Keyword(s):  
State Of The Art ◽  
Classification Problem ◽  
Sentiment Classification ◽  
Great Success ◽  
Source Language ◽  
Shared Feature ◽  
Low Resource ◽  
Feature Extractor ◽  
Cross Lingual ◽  
Averaging Network

In recent years great success has been achieved in sentiment classification for English, thanks in part to the availability of copious annotated resources. Unfortunately, most languages do not enjoy such an abundance of labeled data. To tackle the sentiment classification problem in low-resource languages without adequate annotated data, we propose an Adversarial Deep Averaging Network (ADAN 1 ) to transfer the knowledge learned from labeled data on a resource-rich source language to low-resource languages where only unlabeled data exist. ADAN has two discriminative branches: a sentiment classifier and an adversarial language discriminator. Both branches take input from a shared feature extractor to learn hidden representations that are simultaneously indicative for the classification task and invariant across languages. Experiments on Chinese and Arabic sentiment classification demonstrate that ADAN significantly outperforms state-of-the-art systems.

scholarly journals Cross-Lingual Syntactic Transfer with Limited Resources

2017 ◽  
Vol 5 ◽  
pp. 279-293 ◽  
Author(s):  
Mohammad Sadegh Rasooli ◽  
Michael Collins
Keyword(s):  
Projection Method ◽  
State Of The Art ◽  
The State ◽  
Target Language ◽  
Limited Resources ◽  
Lexical Information ◽  
Source Language ◽  
The Bible ◽  
Cross Lingual

We describe a simple but effective method for cross-lingual syntactic transfer of dependency parsers, in the scenario where a large amount of translation data is not available. This method makes use of three steps: 1) a method for deriving cross-lingual word clusters, which can then be used in a multilingual parser; 2) a method for transferring lexical information from a target language to source language treebanks; 3) a method for integrating these steps with the density-driven annotation projection method of Rasooli and Collins (2015). Experiments show improvements over the state-of-the-art in several languages used in previous work, in a setting where the only source of translation data is the Bible, a considerably smaller corpus than the Europarl corpus used in previous work. Results using the Europarl corpus as a source of translation data show additional improvements over the results of Rasooli and Collins (2015). We conclude with results on 38 datasets from the Universal Dependencies corpora.

scholarly journals Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification

2020 ◽  
Vol 34 (05) ◽  
pp. 9547-9554
Author(s):  
Mozhi Zhang ◽  
Yoshinari Fujinuma ◽  
Jordan Boyd-Graber
Keyword(s):  
Knowledge Transfer ◽  
Text Classification ◽  
Document Classification ◽  
Training Data ◽  
Target Language ◽  
Source Language ◽  
Low Resource ◽  
Classification Framework ◽  
Related Language ◽  
Cross Lingual

Text classification must sometimes be applied in a low-resource language with no labeled training data. However, training data may be available in a related language. We investigate whether character-level knowledge transfer from a related language helps text classification. We present a cross-lingual document classification framework (caco) that exploits cross-lingual subword similarity by jointly training a character-based embedder and a word-based classifier. The embedder derives vector representations for input words from their written forms, and the classifier makes predictions based on the word vectors. We use a joint character representation for both the source language and the target language, which allows the embedder to generalize knowledge about source language words to target language words with similar forms. We propose a multi-task objective that can further improve the model if additional cross-lingual or monolingual resources are available. Experiments confirm that character-level knowledge transfer is more data-efficient than word-level transfer between related languages.

scholarly journals Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce

2020 ◽  
Vol 34 (05) ◽  
pp. 8212-8219
Author(s):  
Juntao Li ◽  
Chang Liu ◽  
Jian Wang ◽  
Lidong Bing ◽  
Hongsong Li ◽  
...  
Keyword(s):  
Model Development ◽  
Target Language ◽  
Retrieval Task ◽  
Product Attribute ◽  
Source Language ◽  
Low Resource ◽  
Cross Border ◽  
Resource Setting ◽  
Context Dependent ◽  
Cross Lingual

With the prosperous of cross-border e-commerce, there is an urgent demand for designing intelligent approaches for assisting e-commerce sellers to offer local products for consumers from all over the world. In this paper, we explore a new task of cross-lingual information retrieval, i.e., cross-lingual set-to-description retrieval in cross-border e-commerce, which involves matching product attribute sets in the source language with persuasive product descriptions in the target language. We manually collect a new and high-quality paired dataset, where each pair contains an unordered product attribute set in the source language and an informative product description in the target language. As the dataset construction process is both time-consuming and costly, the new dataset only comprises of 13.5k pairs, which is a low-resource setting and can be viewed as a challenging testbed for model development and evaluation in cross-border e-commerce. To tackle this cross-lingual set-to-description retrieval task, we propose a novel cross-lingual matching network (CLMN) with the enhancement of context-dependent cross-lingual mapping upon the pre-trained monolingual BERT representations. Experimental results indicate that our proposed CLMN yields impressive results on the challenging task and the context-dependent cross-lingual mapping on BERT yields noticeable improvement over the pre-trained multi-lingual BERT model.

scholarly journals Zero-Shot Neural Transfer for Cross-Lingual Entity Linking

2019 ◽  
Vol 33 ◽  
pp. 6924-6931 ◽  
Author(s):  
Shruti Rijhwani ◽  
Jiateng Xie ◽  
Graham Neubig ◽  
Jaime Carbonell
Keyword(s):  
Target Language ◽  
Entity Linking ◽  
Lexical Resources ◽  
Phonological Representations ◽  
Low Resource ◽  
Average Accuracy ◽  
Structured Knowledge ◽  
Target Languages ◽  
Cross Lingual ◽  
Language Universal

Cross-lingual entity linking maps an entity mention in a source language to its corresponding entry in a structured knowledge base that is in a different (target) language. While previous work relies heavily on bilingual lexical resources to bridge the gap between the source and the target languages, these resources are scarce or unavailable for many low-resource languages. To address this problem, we investigate zero-shot cross-lingual entity linking, in which we assume no bilingual lexical resources are available in the source low-resource language. Specifically, we propose pivot-basedentity linking, which leverages information from a highresource “pivot” language to train character-level neural entity linking models that are transferred to the source lowresource language in a zero-shot manner. With experiments on 9 low-resource languages and transfer through a total of54 languages, we show that our proposed pivot-based framework improves entity linking accuracy 17% (absolute) on average over the baseline systems, for the zero-shot scenario.1 Further, we also investigate the use of language-universal phonological representations which improves average accuracy (absolute) by 36% when transferring between languages that use different scripts.

scholarly journals Learning to Learn Morphological Inflection for Resource-Poor Languages

2020 ◽  
Vol 34 (05) ◽  
pp. 8058-8065
Author(s):  
Katharina Kann ◽  
Samuel R. Bowman ◽  
Kyunghyun Cho
Keyword(s):  
Fine Tuning ◽  
Target Language ◽  
Model Parameters ◽  
Transfer Model ◽  
Absolute Accuracy ◽  
Learning To Learn ◽  
Suggested Approach ◽  
High Resource ◽  
Resource Poor ◽  
Cross Lingual

We propose to cast the task of morphological inflection—mapping a lemma to an indicated inflected form—for resource-poor languages as a meta-learning problem. Treating each language as a separate task, we use data from high-resource source languages to learn a set of model parameters that can serve as a strong initialization point for fine-tuning on a resource-poor target language. Experiments with two model architectures on 29 target languages from 3 families show that our suggested approach outperforms all baselines. In particular, it obtains a 31.7% higher absolute accuracy than a previously proposed cross-lingual transfer model and outperforms the previous state of the art by 1.7% absolute accuracy on average over languages.

scholarly journals GlobalTrait: Personality Alignment of Multilingual Word Embeddings

2019 ◽  
Vol 33 ◽  
pp. 7015-7022
Author(s):  
Farhad Bin Siddique ◽  
Dario Bertero ◽  
Pascale Fung
Keyword(s):  
Personality Traits ◽  
Big Five ◽  
Target Language ◽  
Big Five Personality ◽  
Word Embeddings ◽  
Alignment Method ◽  
Semantic Meaning ◽  
Text Data ◽  
Source Language ◽  
High Resource

We propose a multilingual model to recognize Big Five Personality traits from text data in four different languages: English, Spanish, Dutch and Italian. Our analysis shows that words having a similar semantic meaning in different languages do not necessarily correspond to the same personality traits. Therefore, we propose a personality alignment method, GlobalTrait, which has a mapping for each trait from the source language to the target language (English), such that words that correlate positively to each trait are close together in the multilingual vector space. Using these aligned embeddings for training, we can transfer personality related training features from high-resource languages such as English to other low-resource languages, and get better multilingual results, when compared to using simple monolingual and unaligned multilingual embeddings. We achieve an average F-score increase (across all three languages except English) from 65 to 73.4 (+8.4), when comparing our monolingual model to multilingual using CNN with personality aligned embeddings. We also show relatively good performance in the regression tasks, and better classification results when evaluating our model on a separate Chinese dataset.

Deep Persian sentiment analysis: Cross-lingual training for low-resource languages

2020 ◽  
pp. 016555152096278
Author(s):  
Rouzbeh Ghasemi ◽  
Seyed Arad Ashrafi Asli ◽  
Saeedeh Momtazi
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Training Data ◽  
Target Language ◽  
Low Resource ◽  
Proposed Model ◽  
Significant Difference ◽  
Cross Lingual

With the advent of deep neural models in natural language processing tasks, having a large amount of training data plays an essential role in achieving accurate models. Creating valid training data, however, is a challenging issue in many low-resource languages. This problem results in a significant difference between the accuracy of available natural language processing tools for low-resource languages compared with rich languages. To address this problem in the sentiment analysis task in the Persian language, we propose a cross-lingual deep learning framework to benefit from available training data of English. We deployed cross-lingual embedding to model sentiment analysis as a transfer learning model which transfers a model from a rich-resource language to low-resource ones. Our model is flexible to use any cross-lingual word embedding model and any deep architecture for text classification. Our experiments on English Amazon dataset and Persian Digikala dataset using two different embedding models and four different classification networks show the superiority of the proposed model compared with the state-of-the-art monolingual techniques. Based on our experiment, the performance of Persian sentiment analysis improves 22% in static embedding and 9% in dynamic embedding. Our proposed model is general and language-independent; that is, it can be used for any low-resource language, once a cross-lingual embedding is available for the source–target language pair. Moreover, by benefitting from word-aligned cross-lingual embedding, the only required data for a reliable cross-lingual embedding is a bilingual dictionary that is available between almost all languages and the English language, as a potential source language.

Sign in / Sign up

Export Citation Format

Share Document