scholarly journals Automatically extracted parallel corpora enriched with highly useful metadata? A Wikipedia case study combining machine learning and social technology

2020 ◽  
Author(s):  
Ahmad Aghaebrahimian ◽  
Andy Stauder ◽  
Michael Ustaszewski
Keyword(s):  
Language Processing ◽  
User Survey ◽  
Promising Candidate ◽  
Parallel Corpora ◽  
Web Resources ◽  
Metadata Enrichment ◽  
Metadata Quality ◽  
Parallel Text ◽  
Combination Of Methods

Abstract The extraction of large amounts of multilingual parallel text from web resources is a widely used technique in natural language processing. However, automatically collected parallel corpora usually lack precise metadata, which are crucial to accurate data analysis and interpretation. The combination of automated extraction procedures and manual metadata enrichment may help address this issue. Wikipedia is a promising candidate for the exploration of the potential of said combination of methods because it is a rich source of translations in a large number of language pairs and because its open and collaborative nature makes it possible to identify and contact the users who produce translations. This article tests to what extent translated texts automatically extracted from Wikipedia by means of neural networks can be enriched with pertinent metadata through a self-submission-based user survey. Special emphasis is placed on data usefulness, defined in terms of a catalogue of previously established assessment criteria, most prominently metadata quality. The results suggest that from a quantitative perspective, the proposed methodology is capable of capturing metadata otherwise not available. At the same time, the crowd-based collection of data and metadata may face important technical and social limitations.

scholarly journals The Web as a Parallel Corpus

2003 ◽  
Vol 29 (3) ◽  
pp. 349-380 ◽  
Author(s):  
Philip Resnik ◽  
Noah A. Smith
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Structural Features ◽  
Classification Performance ◽  
Internet Archive ◽  
Parallel Corpora ◽  
Parallel Corpus ◽  
Original Algorithm ◽  
Parallel Text ◽  
The Web

Parallel corpora have become an essential resource for work in multilingual natural language processing. In this article, we report on our work using the STRAND system for mining parallel text on the World Wide Web, first reviewing the original algorithm and results and then presenting a set of significant enhancements. These enhancements include the use of supervised learning based on structural features of documents to improve classification performance, a new content-based measure of translational equivalence, and adaptation of the system to take advantage of the Internet Archive for mining parallel text from the Web on a large scale. Finally, the value of these techniques is demonstrated in the construction of a significant parallel corpus for a low-density language pair.

scholarly journals Natural language processing (NLP) tools in extracting biomedical concepts from research articles: a case study on autism spectrum disorder

2020 ◽  
Vol 20 (S11) ◽  
Author(s):  
Jacqueline Peng ◽  
Mengge Zhao ◽  
James Havrilla ◽  
Cong Liu ◽  
Chunhua Weng ◽  
...  
Keyword(s):  
Autism Spectrum Disorder ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Clinical Manifestations ◽  
Predictive Performance ◽  
Autism Spectrum ◽  
Spectrum Disorder ◽  
Research Articles

Abstract Background Natural language processing (NLP) tools can facilitate the extraction of biomedical concepts from unstructured free texts, such as research articles or clinical notes. The NLP software tools CLAMP, cTAKES, and MetaMap are among the most widely used tools to extract biomedical concept entities. However, their performance in extracting disease-specific terminology from literature has not been compared extensively, especially for complex neuropsychiatric disorders with a diverse set of phenotypic and clinical manifestations. Methods We comparatively evaluated these NLP tools using autism spectrum disorder (ASD) as a case study. We collected 827 ASD-related terms based on previous literature as the benchmark list for performance evaluation. Then, we applied CLAMP, cTAKES, and MetaMap on 544 full-text articles and 20,408 abstracts from PubMed to extract ASD-related terms. We evaluated the predictive performance using precision, recall, and F1 score. Results We found that CLAMP has the best performance in terms of F1 score followed by cTAKES and then MetaMap. Our results show that CLAMP has much higher precision than cTAKES and MetaMap, while cTAKES and MetaMap have higher recall than CLAMP. Conclusion The analysis protocols used in this study can be applied to other neuropsychiatric or neurodevelopmental disorders that lack well-defined terminology sets to describe their phenotypic presentations.

scholarly journals Citizen Participation and Machine Learning for a Better Democracy

2021 ◽  
Author(s):  
Robert Procter ◽  
Miguel Arana-Catania ◽  
Felix-Anselm van Lier ◽  
Nataliya Tkachenko ◽  
Yulan He ◽  
...  
Keyword(s):  
Machine Learning ◽  
Citizen Participation ◽  
Language Processing ◽  
Information Overload ◽  
City Council ◽  
Decision Making Processes ◽  
Development Goals ◽  
Democratic Systems ◽  
The City

The development of democratic systems is a crucial task as confirmed by its selection as one of the Millennium Sustainable Development Goals by the United Nations. In this article, we report on the progress of a project that aims to address barriers, one of which is information overload, to achieving effective direct citizen participation in democratic decision-making processes. The main objectives are to explore if the application of Natural Language Processing (NLP) and machine learning can improve citizens? experience of digital citizen participation platforms. Taking as a case study the ?Decide Madrid? Consul platform, which enables citizens to post proposals for policies they would like to see adopted by the city council, we used NLP and machine learning to provide new ways to (a) suggest to citizens proposals they might wish to support; (b) group citizens by interests so that they can more easily interact with each other; (c) summarise comments posted in response to proposals; (d) assist citizens in aggregating and developing proposals. Evaluation of the results confirms that NLP and machine learning have a role to play in addressing some of the barriers users of platforms such as Consul currently experience.

scholarly journals The Sentiments Analysis of Donald Trump and Jokowi’s Twitters on Covid-19 Policy Dissemination

Webology ◽  
2021 ◽  
Vol 18 (1) ◽  
pp. 389-405
Author(s):  
Rahmad Agus Dwianto ◽  
Achmad Nurmandi ◽  
Salahudin Salahudin
Keyword(s):  
Data Collection ◽  
Sentiment Analysis ◽  
Computational Linguistics ◽  
Language Processing ◽  
The Us ◽  
Twitter Account ◽  
Policy Dissemination ◽  
Collection Period ◽  
Source Materials

As Covid-19 spreads to other nations and governments attempt to minimize its effect by introducing countermeasures, individuals have often used social media outlets to share their opinions on the measures themselves, the leaders implementing them, and the ways in which their lives are shifting. Sentiment analysis refers to the application in source materials of natural language processing, computational linguistics, and text analytics to identify and classify subjective opinions. The reason why this research uses a sentiment case study towards Trump and Jokowi's policies is because Jokowi and Trump have similarities in handling Covid-19. Indonesia and the US are still low in the discipline in implementing health protocols. The data collection period was chosen on September 21 - October 21 2020 because during that period, the top 5 trending on Twitter included # covid19, #jokowi, #miglobal, #trump, and #donaldtrump. So, this period is most appropriate for taking data and discussing the handling of Covid-19 by Jokowi and Trump. The result shows both Jokowi and Trump have higher negative sentiments than positive sentiments during the period. Trump had issued a controversial statement regarding the handling of Covid-19. This research is limited to the sentiment generated by the policies conveyed by the US and Indonesian Governments via @jokowi and @realDonaldTrump Twitter Account. The dataset presented in this research is being collected and analyzed using the Brand24, a software-automated sentiment analysis. Further research can increase the scope of the data and increase the timeframe for data collection and develop tools for analyzing sentiment.

scholarly journals Ekstraksi Informasi Halaman Web Menggunakan Pendekatan Bootstrapping pada Ontology-Based Information Extraction

2015 ◽  
Vol 9 (2) ◽  
pp. 111 ◽  
Author(s):  
Erma Susanti ◽  
Khabib Mustofa
Keyword(s):  
Information Extraction ◽  
Language Processing ◽  
Semantic Content ◽  
Extraction Process ◽  
Web Pages ◽  
Structured Information ◽  
Improved Performance ◽  
Types Of Information ◽  
Unstructured Information

AbstrakEkstraksi  informasi  merupakan suatu bidang ilmu untuk pengolahan bahasa alami, dengan cara mengubah teks tidak terstruktur menjadi informasi dalam bentuk terstruktur. Berbagai jenis informasi di Internet ditransmisikan secara tidak terstruktur melalui website, menyebabkan munculnya kebutuhan akan suatu teknologi untuk menganalisa teks dan menemukan pengetahuan yang relevan dalam bentuk informasi terstruktur. Contoh informasi tidak terstruktur adalah informasi utama yang ada pada konten halaman web. Bermacam pendekatan untuk ekstraksi informasi telah dikembangkan oleh berbagai peneliti, baik menggunakan metode manual atau otomatis, namun masih perlu ditingkatkan kinerjanya terkait akurasi dan kecepatan ekstraksi. Pada penelitian ini diusulkan suatu penerapan pendekatan ekstraksi informasi dengan mengkombinasikan pendekatan bootstrapping dengan Ontology-based Information Extraction (OBIE). Pendekatan bootstrapping dengan menggunakan sedikit contoh data berlabel, digunakan untuk memimalkan keterlibatan manusia dalam proses ekstraksi informasi, sedangkan penggunakan panduan ontologi untuk mengekstraksi classes (kelas), properties dan instance digunakan untuk menyediakan konten semantik untuk web semantik. Pengkombinasian kedua pendekatan tersebut diharapkan dapat meningkatan kecepatan proses ekstraksi dan akurasi hasil ekstraksi. Studi kasus untuk penerapan sistem ekstraksi informasi menggunakan dataset “LonelyPlanet”. Kata kunci—Ekstraksi informasi, ontologi, bootstrapping, Ontology-Based Information Extraction, OBIE, kinerja Abstract Information extraction is a field study of natural language processing by converting unstructured text into structured information. Several types of information on the Internet is transmitted through unstructured information via websites, led to emergence of the need a technology to analyze text and found relevant knowledge into structured information. For example of unstructured information is existing main information on the content of web pages. Various approaches  for information extraction have been developed by many researchers, either using manual or automatic method, but still need to be improved performance related accuracy and speed of extraction. This research proposed an approach of information extraction that combines bootstrapping approach with Ontology-Based Information Extraction (OBIE). Bootstrapping approach using small seed of labelled data, is used to minimize human intervention on information extraction process, while the use of guide ontology for extracting classes, properties and instances, using for provide semantic content for semantic web. Combining both approaches expected to increase speed of extraction process and accuracy of extraction results. Case study to apply information extraction system using “LonelyPlanet” datasets. Keywords— Information extraction, ontology, bootstrapping, Ontology-Based Information Extraction, OBIE, performance

scholarly journals Natural Language Processing (NLP) and Its Impact across Industries – Unlocking the True Potential of Digital Healthcare (A Case Study Approach)

2021 ◽  
pp. 86-98
Author(s):  
Sourajit Roy ◽  
Pankaj Pathak ◽  
S. Nithya
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Computational Models ◽  
M2m Communication ◽  
Digital Revolution ◽  
Language Complexity ◽  
Study Approach ◽  
Data Variability

During the advent of the 21st century, technical breakthroughs and developments took place. Natural Language Processing or NLP is one of their promising disciplines that has been increasingly dynamic via groundbreaking findings on most computer networks. Because of the digital revolution the amounts of data generated by M2M communication across devices and platforms such as Amazon Alexa, Apple Siri, Microsoft Cortana, etc. were significantly increased. This causes a great deal of unstructured data to be processed that does not fit in with standard computational models. In addition, the increasing problems of language complexity, data variability and voice ambiguity make implementing models increasingly harder. The current study provides an overview of the potential and breadth of the NLP market and its acceptance in industry-wide, in particular after Covid-19. It also gives a macroscopic picture of progress in natural language processing research, development and implementation.

scholarly journals Pseudotext Injection and Advance Filtering of Low-Resource Corpus for Neural Machine Translation

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Michael Adjeisah ◽  
Guohua Liu ◽  
Douglas Omwenga Nyabuga ◽  
Richard Nuetey Nortey ◽  
Jinling Song
Keyword(s):  
Machine Translation ◽  
Language Processing ◽  
Training Data ◽  
Target Language ◽  
Similarity Metrics ◽  
Mahalanobis Distances ◽  
Parallel Corpora ◽  
Parallel Corpus ◽  
Low Resource ◽  
Sentence Level

Scaling natural language processing (NLP) to low-resourced languages to improve machine translation (MT) performance remains enigmatic. This research contributes to the domain on a low-resource English-Twi translation based on filtered synthetic-parallel corpora. It is often perplexing to learn and understand what a good-quality corpus looks like in low-resource conditions, mainly where the target corpus is the only sample text of the parallel language. To improve the MT performance in such low-resource language pairs, we propose to expand the training data by injecting synthetic-parallel corpus obtained by translating a monolingual corpus from the target language based on bootstrapping with different parameter settings. Furthermore, we performed unsupervised measurements on each sentence pair engaging squared Mahalanobis distances, a filtering technique that predicts sentence parallelism. Additionally, we extensively use three different sentence-level similarity metrics after round-trip translation. Experimental results on a diverse amount of available parallel corpus demonstrate that injecting pseudoparallel corpus and extensive filtering with sentence-level similarity metrics significantly improves the original out-of-the-box MT systems for low-resource language pairs. Compared with existing improvements on the same original framework under the same structure, our approach exhibits tremendous developments in BLEU and TER scores.

Sign in / Sign up

Export Citation Format

Share Document