Parallel sentence extraction to improve cross-language information retrieval from Wikipedia

2021 ◽  
pp. 016555152199275
Author(s):  
Juryong Cheon ◽  
Youngjoong Ko
Keyword(s):  
Information Retrieval ◽  
Language Resources ◽  
Query Translation ◽  
Factors Affecting ◽  
Parallel Corpora ◽  
Parallel Corpus ◽  
Bilingual Dictionary ◽  
Sentence Extraction ◽  
Cross Language Information Retrieval ◽  
Cross Language

Translation language resources, such as bilingual word lists and parallel corpora, are important factors affecting the effectiveness of cross-language information retrieval (CLIR) systems. In particular, when large domain-appropriate parallel corpora are not available, developing an effective CLIR system is particularly difficult. Furthermore, creating a large parallel corpus is costly and requires considerable effort. Therefore, we here demonstrate the construction of parallel corpora from Wikipedia as well as improved query translation, wherein the queries are used for a CLIR system. To do so, we first constructed a bilingual dictionary, termed WikiDic. Then, we evaluated individual language resources and combinations of them in terms of their ability to extract parallel sentences; the combinations of our proposed WikiDic with the translation probability from the Web’s bilingual example sentence pairs and WikiDic was found to be best suited to parallel sentence extraction. Finally, to evaluate the parallel corpus generated from this best combination of language resources, we compared its performance in query translation for CLIR to that of a manually created English–Korean parallel corpus. As a result, the corpus generated by our proposed method achieved a better performance than did the manually created corpus, thus demonstrating the effectiveness of the proposed method for automatic parallel corpus extraction. Not only can the method demonstrated herein be used to inform the construction of other parallel corpora from language resources that are readily available, but also, the parallel sentence extraction method will naturally improve as Wikipedia continues to be used and its content develops.

scholarly journals Embedding Web-Based Statistical Translation Models in Cross-Language Information Retrieval

2003 ◽  
Vol 29 (3) ◽  
pp. 381-419 ◽  
Author(s):  
Wessel Kraaij ◽  
Jian-Yun Nie ◽  
Michel Simard
Keyword(s):  
Information Retrieval ◽  
Low Cost ◽  
Standard Test ◽  
Query Translation ◽  
Web Based ◽  
Parallel Corpora ◽  
Cross Language Information Retrieval ◽  
Cross Language ◽  
Parallel Texts ◽  
The Web

Although more and more language pairs are covered by machine translation (MT) services, there are still many pairs that lack translation resources. Cross-language information retrieval (CLIR) is an application that needs translation functionality of a relatively low level of sophistication, since current models for information retrieval (IR) are still based on a bag of words. The Web provides a vast resource for the automatic construction of parallel corpora that can be used to train statistical translation models automatically. The resulting translation models can be embedded in several ways in a retrieval model. In this article, we will investigate the problem of automatically mining parallel texts from the Web and different ways of integrating the translation models within the retrieval process. Our experiments on standard test collections for CLIR show that the Web-based translation models can surpass commercial MT systems in CLIR tasks. These results open the perspective of constructing a fully automatic query translation device for CLIR at a very low cost.

Cross-language Ontology Learning

2011 ◽  
pp. 272-297
Author(s):  
Hans Hjelm ◽  
Martin Volk
Keyword(s):  
Information Retrieval ◽  
Learning Task ◽  
Formal Ontology ◽  
Ontology Learning ◽  
Lexical Knowledge ◽  
Automatic Learning ◽  
Parallel Corpora ◽  
Cross Language Information Retrieval ◽  
Language Data ◽  
Cross Language

A formal ontology does not contain lexical knowledge; it is by nature language-independent. Mappings can be added between the ontology and, arbitrarily, many lexica in any number of languages. The result of this operation is what is here referred to as a cross-language ontology. A cross-language ontology can be a useful resource for machine translation or cross-language information retrieval. This chapter focuses on ways of automatically building an ontology by exploiting cross-language information from parallel corpora. The goal is to improve the automatic learning results compared to learning an ontology from resources in a single language. The authors present a framework for cross-language ontology learning, providing a setting in which cross-language evidence (data) can be integrated and quantified. The aim is to investigate the following question: Can cross-language data teach us more than data from a single language for the ontology learning task?

Disambiguation of single noun translations extracted from bilingual comparable corpora

Terminology ◽  
2001 ◽  
Vol 7 (1) ◽  
pp. 63-83 ◽  
Author(s):  
Hiroshi Nakagawa
Keyword(s):  
Information Retrieval ◽  
Language Translation ◽  
Target Language ◽  
Comparable Corpora ◽  
Source Language ◽  
Bilingual Dictionary ◽  
Second Stage ◽  
Cross Language Information Retrieval ◽  
Machine Readable ◽  
Cross Language

Bilingual machine readable dictionaries are important and indispensable resources of information for cross-language information retrieval, and machine translation. Recently, these cross-language informational activities have begun to focus on specific academic or technological domains. In this paper, we describe a bilingual dictionary acquisition system which extracts translations from non-parallel but comparable corpora of a specific academic domain and disambiguates the extracted translations. The proposed method is two-fold. At the first stage, candidate terms are extracted from a Japanese and English corpus, respectively, and ranked according to their importance as terms. At the second stage, ambiguous translations are resolved by selecting the target language translation which is the nearest in rank to the source language term. Finally, we evaluate the proposed method in an experiment.

Sign in / Sign up

Export Citation Format

Share Document