scholarly journals A Minimally Supervised Approach for Synonym Extraction with Word Embeddings

2016 ◽  
Vol 105 (1) ◽  
pp. 111-142 ◽  
Author(s):  
Artuur Leeuwenberg ◽  
Mihaela Vela ◽  
Jon Dehdari ◽  
Josef van Genabith
Keyword(s):  
Machine Translation ◽  
Window Size ◽  
Cosine Similarity ◽  
Bag Of Words ◽  
Word Embeddings ◽  
Part Of Speech ◽  
Novel Approach ◽  
Evaluation Metric ◽  
Minimally Supervised ◽  
Final System

Abstract In this paper we present a novel approach to minimally supervised synonym extraction. The approach is based on the word embeddings and aims at presenting a method for synonym extraction that is extensible to various languages. We report experiments with word vectors trained by using both the continuous bag-of-words model (CBoW) and the skip-gram model (SG) investigating the effects of different settings with respect to the contextual window size, the number of dimensions and the type of word vectors. We analyze the word categories that are (cosine) similar in the vector space, showing that cosine similarity on its own is a bad indicator to determine if two words are synonymous. In this context, we propose a new measure, relative cosine similarity, for calculating similarity relative to other cosine-similar words in the corpus. We show that calculating similarity relative to other words boosts the precision of the extraction. We also experiment with combining similarity scores from differently-trained vectors and explore the advantages of using a part-of-speech tagger as a way of introducing some light supervision, thus aiding extraction. We perform both intrinsic and extrinsic evaluation on our final system: intrinsic evaluation is carried out manually by two human evaluators and we use the output of our system in a machine translation task for extrinsic evaluation, showing that the extracted synonyms improve the evaluation metric.

STD: An Automatic Evaluation Metric for Machine Translation Based on Word Embeddings

2019 ◽  
Vol 27 (10) ◽  
pp. 1497-1506 ◽  
Author(s):  
Pairui Li ◽  
Chuan Chen ◽  
Wujie Zheng ◽  
Yuetang Deng ◽  
Fanghua Ye ◽  
...  
Keyword(s):  
Machine Translation ◽  
Automatic Evaluation ◽  
Word Embeddings ◽  
Evaluation Metric

Designing High Accuracy Statistical Machine Translation for Sign Language Using Parallel Corpus

2019 ◽  
Vol 12 (2) ◽  
pp. 134-158 ◽  
Author(s):  
Achraf Othman ◽  
Mohamed Jemni
Keyword(s):  
American Sign Language ◽  
Sign Language ◽  
Machine Translation ◽  
Statistical Machine Translation ◽  
English Text ◽  
Parallel Corpus ◽  
Transcription System ◽  
Novel Approach ◽  
Alignment Algorithms ◽  
Evaluation Metric

In this article, the authors deal with the machine translation of written English text to sign language. They study the existing systems and issues in order to propose an implantation of a statistical machine translation from written English text to American Sign Language (English/ASL) taking care of several features of sign language. The work proposes a novel approach to build artificial corpus using grammatical dependencies rules owing to the lack of resources for sign language. The parallel corpus was the input of the statistical machine translation, which was used for creating statistical memory translation based on IBM alignment algorithms. These algorithms were enhanced and optimized by integrating the Jaro–Winkler distances in order to decrease training process. Subsequently, based on the constructed translation memory, a decoder was implemented for translating English text to the ASL using a novel proposed transcription system based on gloss annotation. The results were evaluated using the BLEU evaluation metric.

scholarly journals Learning Lexical Subspaces in a Distributional Vector Space

2020 ◽  
Vol 8 ◽  
pp. 311-329
Author(s):  
Kushal Arora ◽  
Aishik Chakraborty ◽  
Jackie C. K. Cheung
Keyword(s):  
Vector Space ◽  
Semantic Relations ◽  
Distributional Semantics ◽  
Word Embeddings ◽  
Word Similarity ◽  
Lexical Semantic ◽  
Novel Approach ◽  
Classification Tasks

In this paper, we propose LexSub, a novel approach towards unifying lexical and distributional semantics. We inject knowledge about lexical-semantic relations into distributional word embeddings by defining subspaces of the distributional vector space in which a lexical relation should hold. Our framework can handle symmetric attract and repel relations (e.g., synonymy and antonymy, respectively), as well as asymmetric relations (e.g., hypernymy and meronomy). In a suite of intrinsic benchmarks, we show that our model outperforms previous approaches on relatedness tasks and on hypernymy classification and detection, while being competitive on word similarity tasks. It also outperforms previous systems on extrinsic classification tasks that benefit from exploiting lexical relational cues. We perform a series of analyses to understand the behaviors of our model. 1 Code available at https://github.com/aishikchakraborty/LexSub .

scholarly journals Neuron Interaction Based Representation Composition for Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8204-8211
Author(s):  
Jian Li ◽  
Xing Wang ◽  
Baosong Yang ◽  
Shuming Shi ◽  
Michael R. Lyu ◽  
...  
Keyword(s):  
Machine Translation ◽  
Semantic Information ◽  
Low Rank ◽  
Strong Interactions ◽  
Low Rank Approximation ◽  
Neural Machine Translation ◽  
First Order ◽  
Complex Information ◽  
Novel Approach ◽  
Rank Approximation

Recent NLP studies reveal that substantial linguistic information can be attributed to single neurons, i.e., individual dimensions of the representation vectors. We hypothesize that modeling strong interactions among neurons helps to better capture complex information by composing the linguistic properties embedded in individual neurons. Starting from this intuition, we propose a novel approach to compose representations learned by different components in neural machine translation (e.g., multi-layer networks or multi-head attention), based on modeling strong interactions among neurons in the representation vectors. Specifically, we leverage bilinear pooling to model pairwise multiplicative interactions among individual neurons, and a low-rank approximation to make the model computationally feasible. We further propose extended bilinear pooling to incorporate first-order representations. Experiments on WMT14 English⇒German and English⇒French translation tasks show that our model consistently improves performances over the SOTA Transformer baseline. Further analyses demonstrate that our approach indeed captures more syntactic and semantic information as expected.

scholarly journals Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation

2017 ◽  
Vol 108 (1) ◽  
pp. 85-96 ◽  
Author(s):  
Eva Martínez Garcia ◽  
Carles Creus ◽  
Cristina España-Bonet ◽  
Lluís Màrquez
Keyword(s):  
Machine Translation ◽  
Evaluation Metrics ◽  
Automatic Evaluation ◽  
Word Embeddings ◽  
Standard Document ◽  
Sentence Level ◽  
Word Translation ◽  
Stochastic Mechanism ◽  
Document Level

Abstract We integrate new mechanisms in a document-level machine translation decoder to improve the lexical consistency of document translations. First, we develop a document-level feature designed to score the lexical consistency of a translation. This feature, which applies to words that have been translated into different forms within the document, uses word embeddings to measure the adequacy of each word translation given its context. Second, we extend the decoder with a new stochastic mechanism that, at translation time, allows to introduce changes in the translation oriented to improve its lexical consistency. We evaluate our system on English–Spanish document translation, and we conduct automatic and manual assessments of its quality. The automatic evaluation metrics, applied mainly at sentence level, do not reflect significant variations. On the contrary, the manual evaluation shows that the system dealing with lexical consistency is preferred over both a standard sentence-level and a standard document-level phrase-based MT systems.

Creation of annotated country-level dialectal Arabic resources: An unsupervised approach

2021 ◽  
pp. 1-42
Author(s):  
Maha J. Althobaiti
Keyword(s):  
Language Processing ◽  
Social Networking Sites ◽  
Linguistic Diversity ◽  
Poor Performance ◽  
Country Level ◽  
Part Of Speech ◽  
Novel Approach ◽  
Agreement Score ◽  
Arabic Dialects ◽  
Dialectal Arabic

Abstract The wide usage of multiple spoken Arabic dialects on social networking sites stimulates increasing interest in Natural Language Processing (NLP) for dialectal Arabic (DA). Arabic dialects represent true linguistic diversity and differ from modern standard Arabic (MSA). In fact, the complexity and variety of these dialects make it insufficient to build one NLP system that is suitable for all of them. In comparison with MSA, the available datasets for various dialects are generally limited in terms of size, genre and scope. In this article, we present a novel approach that automatically develops an annotated country-level dialectal Arabic corpus and builds lists of words that encompass 15 Arabic dialects. The algorithm uses an iterative procedure consisting of two main components: automatic creation of lists for dialectal words and automatic creation of annotated Arabic dialect identification corpus. To our knowledge, our study is the first of its kind to examine and analyse the poor performance of the MSA part-of-speech tagger on dialectal Arabic contents and to exploit that in order to extract the dialectal words. The pointwise mutual information association measure and the geographical frequency of word occurrence online are used to classify dialectal words. The annotated dialectal Arabic corpus (Twt15DA), built using our algorithm, is collected from Twitter and consists of 311,785 tweets containing 3,858,459 words in total. We randomly selected a sample of 75 tweets per country, 1125 tweets in total, and conducted a manual dialect identification task by native speakers. The results show an average inter-annotator agreement score equal to 64%, which reflects satisfactory agreement considering the overlapping features of the 15 Arabic dialects.

scholarly journals Tied Transformers: Neural Machine Translation with Shared Encoder and Decoder

2019 ◽  
Vol 33 ◽  
pp. 5466-5473 ◽  
Author(s):  
Yingce Xia ◽  
Tianyu He ◽  
Xu Tan ◽  
Fei Tian ◽  
Di He ◽  
...  
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
State Of The Art ◽  
Compact Model ◽  
Word Embeddings ◽  
Simple Method ◽  
Neural Machine Translation ◽  
German Translation ◽  
One Step ◽  
Target Side

Sharing source and target side vocabularies and word embeddings has been a popular practice in neural machine translation (briefly, NMT) for similar languages (e.g., English to French or German translation). The success of such wordlevel sharing motivates us to move one step further: we consider model-level sharing and tie the whole parts of the encoder and decoder of an NMT model. We share the encoder and decoder of Transformer (Vaswani et al. 2017), the state-of-the-art NMT model, and obtain a compact model named Tied Transformer. Experimental results demonstrate that such a simple method works well for both similar and dissimilar language pairs. We empirically verify our framework for both supervised NMT and unsupervised NMT: we achieve a 35.52 BLEU score on IWSLT 2014 German to English translation, 28.98/29.89 BLEU scores on WMT 2014 English to German translation without/with monolingual data, and a 22.05 BLEU score on WMT 2016 unsupervised German to English translation.

scholarly journals Multilingual Projection for Parsing Truly Low-Resource Languages

2016 ◽  
Vol 4 ◽  
pp. 301-312 ◽  
Author(s):  
Željko Agić ◽  
Anders Johannsen ◽  
Barbara Plank ◽  
Héctor Martínez Alonso ◽  
Natalie Schluter ◽  
...  
Keyword(s):  
Empirical Evaluation ◽  
Upper Bounds ◽  
Low Resource ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Novel Approach ◽  
Cross Lingual ◽  
Test Languages ◽  
Speech Tagging ◽  
Parallel Texts

We propose a novel approach to cross-lingual part-of-speech tagging and dependency parsing for truly low-resource languages. Our annotation projection-based approach yields tagging and parsing models for over 100 languages. All that is needed are freely available parallel texts, and taggers and parsers for resource-rich languages. The empirical evaluation across 30 test languages shows that our method consistently provides top-level accuracies, close to established upper bounds, and outperforms several competitive baselines.

Sign in / Sign up

Export Citation Format

Share Document