A scalable approach to building a parallel corpus from the web

2011 ◽  
Author(s):  
Vivek Kumar Rangarajan Sridhar ◽  
Luciano Barbosa ◽  
Srinivas Bangalore
Keyword(s):  
Parallel Corpus ◽  
The Web

scholarly journals Constructing a Large-Scale English-Persian Parallel Corpus

2009 ◽  
Vol 54 (1) ◽  
pp. 181-188 ◽  
Author(s):  
Tayebeh Mosavi Miangah
Keyword(s):  
Large Scale ◽  
Target Language ◽  
Translation Memory ◽  
Web Documents ◽  
Parallel Corpus ◽  
Translation Quality ◽  
Text Corpora ◽  
Develop Software ◽  
General Translation ◽  
The Web

Abstract In recent years the exploitation of large text corpora in solving various kinds of linguistic problems, including those of translation, is commonplace. Yet a large-scale English-Persian corpus is still unavailable, because of certain difficulties and the amount of work required to overcome them. The project reported here is an attempt to constitute an English-Persian parallel corpus composed of digital texts and Web documents containing little or no noise. The Internet is useful because translations of existing texts are often published on the Web. The task is to find parallel pages in English and Persian, to judge their translation quality, and to download and align them. The corpus so created is of course open; that is, more material can be added as the need arises. One of the main activities associated with building such a corpus is to develop software for parallel concordancing, in which a user can enter a search string in one language and see all the citations for that string in it and corresponding sentences in the target language. Our intention is to construct general translation memory software using the present English-Persian parallel corpus.

scholarly journals The Web as a Parallel Corpus

2003 ◽  
Vol 29 (3) ◽  
pp. 349-380 ◽  
Author(s):  
Philip Resnik ◽  
Noah A. Smith
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Structural Features ◽  
Classification Performance ◽  
Internet Archive ◽  
Parallel Corpora ◽  
Parallel Corpus ◽  
Original Algorithm ◽  
Parallel Text ◽  
The Web

Parallel corpora have become an essential resource for work in multilingual natural language processing. In this article, we report on our work using the STRAND system for mining parallel text on the World Wide Web, first reviewing the original algorithm and results and then presenting a set of significant enhancements. These enhancements include the use of supervised learning based on structural features of documents to improve classification performance, a new content-based measure of translational equivalence, and adaptation of the system to take advantage of the Internet Archive for mining parallel text from the Web on a large scale. Finally, the value of these techniques is demonstrated in the construction of a significant parallel corpus for a low-density language pair.

The IJS-ELAN Slovene-English Parallel Corpus

2002 ◽  
Vol 7 (1) ◽  
pp. 1-20 ◽  
Author(s):  
Tomaž Erjavec
Keyword(s):  
Text Encoding ◽  
Language Engineering ◽  
Parallel Corpus ◽  
Feature Structures ◽  
The Web ◽  
The Eu

The paper presents an annotated parallel Slovene-English corpus developed in the scope of the EU ELAN project. The IJS-ELAN corpus was compiled to be a widely distributable dataset for language engineering and for translation and terminology studies. The corpus contains 1 million words from fifteen recent terminology-rich texts. The corpus is sentence aligned and word-tagged with context disambiguated morphosyntactic descriptions and lemmas. These descriptions model simple feature structures, the structure of which is shared between Slovene and English. The corpus is encoded according to the Guidelines for Text Encoding and Interchange and is freely available on the Web for downloading. Additionally, access to IJS-ELAN is available via a powerful Web concordancer.

scholarly journals The corpus, its users and their needs

2007 ◽  
Vol 12 (3) ◽  
pp. 335-374 ◽  
Author(s):  
Diana Santos ◽  
Ana Frankenberg-Garcia
Keyword(s):  
User Studies ◽  
Log Analysis ◽  
Web Interface ◽  
Language Resources ◽  
Parallel Corpus ◽  
Null Results ◽  
Search Modes ◽  
The Web

COMPARA is a bidirectional parallel corpus of English and Portuguese, currently with 3 million words. The corpus was launched in 2000 and at present it is possibly the largest edited parallel corpus publicly available on the Web, with roughly 6,000 corpus queries per month. This paper summarizes an analysis of six years of corpus use. We begin by looking at user studies for language resources, especially corpora, and then we provide a snapshot of COMPARA’s users and their behaviour based on log analysis. Particular emphasis is given to the language interface preferred by users (Portuguese and English are possible), the choice between the Simple and Complex Search modes, the reasons underlying null-results and behaviour after restricted output. The data has pointed us to cases where COMPARA’s Web interface can be improved, and provided insights about our users and the problems they face, although further studies that distinguish between different kinds of users remain necessary.

scholarly journals A large English–Thai parallel corpus from the web and machine-generated text

2021 ◽  
Author(s):  
Lalita Lowphansirikul ◽  
Charin Polpanumas ◽  
Attapol T. Rutherford ◽  
Sarana Nutanong
Keyword(s):  
Parallel Corpus ◽  
The Web
Sign in / Sign up

Export Citation Format

Share Document