An approach to improving the quality of part-of-speech tagging of Chinese text

2004 ◽  
Author(s):  
Yi-li Qian ◽  
Jia-heng Zheng
Keyword(s):  
Chinese Text ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Speech Tagging ◽  
Quality Of Part

scholarly journals Modernising historical Slovene words

2015 ◽  
Vol 22 (6) ◽  
pp. 881-905 ◽  
Author(s):  
YVES SCHERRER ◽  
TOMAŽ ERJAVEC
Keyword(s):  
Statistical Machine Translation ◽  
Training Data ◽  
Original Text ◽  
Step Method ◽  
Part Of Speech Tagging ◽  
Text Annotation ◽  
Part Of Speech ◽  
Speech Tagging ◽  
Normalisation Method

AbstractWe propose a language-independent word normalisation method and exemplify it on modernising historical Slovene words. Our method relies on character-level statistical machine translation (CSMT) and uses only shallow knowledge. We present relevant data on historical Slovene, consisting of two (partially) manually annotated corpora and the lexicons derived from these corpora, containing historical word–modern word pairs. The two lexicons are disjoint, with one serving as the training set containing 40,000 entries, and the other as a test set with 20,000 entries. The data spans the years 1750–1900, and the lexicons are split into fifty-year slices, with all the experiments carried out separately on the three time periods. We perform two sets of experiments. In the first one – a supervised setting – we build a CSMT system using the lexicon of word pairs as training data. In the second one – an unsupervised setting – we simulate a scenario in which word pairs are not available. We propose a two-step method where we first extract a noisy list of word pairs by matching historical words with cognate modern words, and then train a CSMT system on these pairs. In both sets of experiments, we also optionally make use of a lexicon of modern words to filter the modernisation hypotheses. While we show that both methods produce significantly better results than the baselines, their accuracy and which method works best strongly correlates with the age of the texts, meaning that the choice of the best method will depend on the properties of the historical language which is to be modernised. As an extrinsic evaluation, we also compare the quality of part-of-speech tagging and lemmatisation directly on historical text and on its modernised words. We show that, depending on the age of the text, annotation on modernised words also produces significantly better results than annotation on the original text.

Lexical Rule and Lexicon Effect for Part of Speech Tagging Bahasa Madura

2018 ◽  
Vol 18 (1) ◽  
pp. 65-72
Author(s):  
Nindian Puspa Dewi ◽  
Ubaidi Ubaidi
Keyword(s):  
Text Processing ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Speech Tagging ◽  
Bahasa Indonesia

POS Tagging adalah dasar untuk pengembangan Text Processing suatu bahasa. Dalam penelitian ini kita meneliti pengaruh penggunaan lexicon dan perubahan morfologi kata dalam penentuan tagset yang tepat untuk suatu kata. Aturan dengan pendekatan morfologi kata seperti awalan, akhiran, dan sisipan biasa disebut sebagai lexical rule. Penelitian ini menerapkan lexical rule hasil learner dengan menggunakan algoritma Brill Tagger. Bahasa Madura adalah bahasa daerah yang digunakan di Pulau Madura dan beberapa pulau lainnya di Jawa Timur. Objek penelitian ini menggunakan Bahasa Madura yang memiliki banyak sekali variasi afiksasi dibandingkan dengan Bahasa Indonesia. Pada penelitian ini, lexicon selain digunakan untuk pencarian kata dasar Bahasa Madura juga digunakan sebagai salah satu tahap pemberian POS Tagging. Hasil ujicoba dengan menggunakan lexicon mencapai akurasi yaitu 86.61% sedangkan jika tidak menggunakan lexicon hanya mencapai akurasi 28.95 %. Dari sini dapat disimpulkan bahwa ternyata lexicon sangat berpengaruh terhadap POS Tagging.

scholarly journals The Impact of Arabic Part of Speech Tagging on Sentiment Analysis: A New Corpus and Deep Learning Approach

2021 ◽  
Vol 184 ◽  
pp. 148-155
Author(s):  
Abdul Munem Nerabie ◽  
Manar AlKhatib ◽  
Sujith Samuel Mathew ◽  
May El Barachi ◽  
Farhad Oroumchian
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Learning Approach ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
The Impact ◽  
Speech Tagging
Sign in / Sign up

Export Citation Format

Share Document