scholarly journals Enhancing Semantic Representations of Bilingual Word Embeddings with Syntactic Dependencies

2018 ◽  
Author(s):  
Linli Xu ◽  
Wenjun Ouyang ◽  
Xiaoying Ren ◽  
Yang Wang ◽  
Liang Jiang
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Real World ◽  
Bag Of Words ◽  
Word Embeddings ◽  
Word Level ◽  
Proposed Model ◽  
Parse Trees ◽  
Cross Lingual ◽  
Syntactic Dependencies

Cross-lingual representation is a technique that can both represent different languages in the same latent vector space and enable the knowledge transfer across languages. To learn such representations, most of existing works require parallel sentences with word-level alignments and assume that aligned words have similar Bag-of-Words (BoW) contexts.  However, due to differences in grammar structures among different languages, the contexts of aligned words in different languages may appear at different positions of the sentence. To address this issue of different syntactics across different languages, we propose a model of bilingual word embeddings integrating syntactic dependencies (DepBiWE) by producing dependency parse-trees which encode the accurate relative positions for the contexts of aligned words. In addition, a new method is proposed to learn bilingual word embeddings from dependency-based contexts and BoW contexts jointly. Extensive experimental results on a real world dataset clearly validate the superiority of the proposed model DepBiWE on various natural language processing (NLP) tasks.

Deep Persian sentiment analysis: Cross-lingual training for low-resource languages

2020 ◽  
pp. 016555152096278
Author(s):  
Rouzbeh Ghasemi ◽  
Seyed Arad Ashrafi Asli ◽  
Saeedeh Momtazi
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Training Data ◽  
Target Language ◽  
Low Resource ◽  
Proposed Model ◽  
Significant Difference ◽  
Cross Lingual

With the advent of deep neural models in natural language processing tasks, having a large amount of training data plays an essential role in achieving accurate models. Creating valid training data, however, is a challenging issue in many low-resource languages. This problem results in a significant difference between the accuracy of available natural language processing tools for low-resource languages compared with rich languages. To address this problem in the sentiment analysis task in the Persian language, we propose a cross-lingual deep learning framework to benefit from available training data of English. We deployed cross-lingual embedding to model sentiment analysis as a transfer learning model which transfers a model from a rich-resource language to low-resource ones. Our model is flexible to use any cross-lingual word embedding model and any deep architecture for text classification. Our experiments on English Amazon dataset and Persian Digikala dataset using two different embedding models and four different classification networks show the superiority of the proposed model compared with the state-of-the-art monolingual techniques. Based on our experiment, the performance of Persian sentiment analysis improves 22% in static embedding and 9% in dynamic embedding. Our proposed model is general and language-independent; that is, it can be used for any low-resource language, once a cross-lingual embedding is available for the source–target language pair. Moreover, by benefitting from word-aligned cross-lingual embedding, the only required data for a reliable cross-lingual embedding is a bilingual dictionary that is available between almost all languages and the English language, as a potential source language.

scholarly journals When Word Embeddings Become Endangered

2021 ◽  
pp. 275-288
Author(s):  
Khalid Alnajjar
Keyword(s):  
Natural Language Processing ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Endangered Languages ◽  
Word Embeddings ◽  
Analysis Model ◽  
Resource Poor ◽  
Semantic Spaces ◽  
Cross Lingual ◽  
Analysis Models

Big languages such as English and Finnish have many natural language processing (NLP) resources and models, but this is not the case for low-resourced and endangered languages as such resources are so scarce despite the great advantages they would provide for the language communities. The most common types of resources available for low-resourced and endangered languages are translation dictionaries and universal dependencies. In this paper, we present a method for constructing word embeddings for endangered languages using existing word embeddings of different resource-rich languages and the translation dictionaries of resource-poor languages. Thereafter, the embeddings are fine-tuned using the sentences in the universal dependencies and aligned to match the semantic spaces of the big languages; resulting in cross-lingual embeddings. The endangered languages we work with here are Erzya, Moksha, Komi-Zyrian and Skolt Sami. Furthermore, we build a universal sentiment analysis model for all the languages that are part of this study, whether endangered or not, by utilizing cross-lingual word embeddings. The evaluation conducted shows that our word embeddings for endangered languages are well-aligned with the resource-rich languages, and they are suitable for training task-specific models as demonstrated by our sentiment analysis models which achieved high accuracies. All our cross-lingual word embeddings and sentiment analysis models will be released openly via an easy-to-use Python library.

scholarly journals A Survey of Cross-lingual Word Embedding Models

2019 ◽  
Vol 65 ◽  
pp. 569-631 ◽  
Author(s):  
Sebastian Ruder ◽  
Ivan Vulić ◽  
Anders Søgaard
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Word Meaning ◽  
Word Embedding ◽  
Word Embeddings ◽  
Objective Functions ◽  
Future Challenges ◽  
Cross Lingual ◽  
Data Requirements

Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent, modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.

scholarly journals Leveraging Vector Space Similarity for Learning Cross-Lingual Word Embeddings: A Systematic Review

Digital ◽  
2021 ◽  
Vol 1 (3) ◽  
pp. 145-161
Author(s):  
Kowshik Bhowmik ◽  
Anca Ralescu
Keyword(s):  
Systematic Review ◽  
Natural Language Processing ◽  
Literature Review ◽  
Natural Language ◽  
Vector Space ◽  
Language Processing ◽  
Word Embedding ◽  
Word Embeddings ◽  
Low Resource ◽  
Cross Lingual

This article presents a systematic literature review on quantifying the proximity between independently trained monolingual word embedding spaces. A search was carried out in the broader context of inducing bilingual lexicons from cross-lingual word embeddings, especially for low-resource languages. The returned articles were then classified. Cross-lingual word embeddings have drawn the attention of researchers in the field of natural language processing (NLP). Although existing methods have yielded satisfactory results for resource-rich languages and languages related to them, some researchers have pointed out that the same is not true for low-resource and distant languages. In this paper, we report the research on methods proposed to provide better representation for low-resource and distant languages in the cross-lingual word embedding space.

1243-P: Novel Use of Natural Language Processing to Identify Reasons for Insulin Discontinuation in Patients with T2DM: A Real-World Evidence Study

Diabetes ◽  
2019 ◽  
Vol 68 (Supplement 1) ◽  
pp. 1243-P
Author(s):  
JIANMIN WU ◽  
FRITHA J. MORRISON ◽  
ZHENXIANG ZHAO ◽  
XUANYAO HE ◽  
MARIA SHUBINA ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Real World ◽  
Real World Evidence

A Hindi Image Caption Generation Framework Using Deep Learning

2021 ◽  
Vol 20 (2) ◽  
pp. 1-19
Author(s):  
Santosh Kumar Mishra ◽  
Rijul Dhir ◽  
Sriparna Saha ◽  
Pushpak Bhattacharyya
Keyword(s):  
Computer Vision ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
English Language ◽  
Image Captioning ◽  
Textual Description ◽  
Proposed Model ◽  
Hindi Language ◽  
The Given

Image captioning is the process of generating a textual description of an image that aims to describe the salient parts of the given image. It is an important problem, as it involves computer vision and natural language processing, where computer vision is used for understanding images, and natural language processing is used for language modeling. A lot of works have been done for image captioning for the English language. In this article, we have developed a model for image captioning in the Hindi language. Hindi is the official language of India, and it is the fourth most spoken language in the world, spoken in India and South Asia. To the best of our knowledge, this is the first attempt to generate image captions in the Hindi language. A dataset is manually created by translating well known MSCOCO dataset from English to Hindi. Finally, different types of attention-based architectures are developed for image captioning in the Hindi language. These attention mechanisms are new for the Hindi language, as those have never been used for the Hindi language. The obtained results of the proposed model are compared with several baselines in terms of BLEU scores, and the results show that our model performs better than others. Manual evaluation of the obtained captions in terms of adequacy and fluency also reveals the effectiveness of our proposed approach. Availability of resources : The codes of the article are available at https://github.com/santosh1821cs03/Image_Captioning_Hindi_Language ; The dataset will be made available: http://www.iitp.ac.in/∼ai-nlp-ml/resources.html .

scholarly journals Real World of Artificial Intelligence - A Review

2020 ◽  
Vol 11 (2) ◽  
pp. 41-47
Author(s):  
Amandeep Kaur ◽  
Madhu Dhiman ◽  
Mansi Tonk ◽  
Ramneet Kaur
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Face Recognition ◽  
Language Processing ◽  
Real World ◽  
Review Paper ◽  
General Concept ◽  
Research Trends ◽  
Human Intelligence

Artificial Intelligence is the combination of machine and human intelligence, which are in research trends from the last many years. Different Artificial Intelligence programs have become capable of challenging humans by providing Expert Systems, Neural Networks, Robotics, Natural Language Processing, Face Recognition and Speech Recognition. Artificial Intelligence brings a bright future for different technical inventions in various fields. This review paper shows the general concept of Artificial Intelligence and presents an impact of Artificial Intelligence in the present and future world.

scholarly journals Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation

2020 ◽  
Vol 34 (05) ◽  
pp. 7797-7804
Author(s):  
Goran Glavašš ◽  
Swapna Somasundaran
Keyword(s):  
State Of The Art ◽  
Language Transfer ◽  
Text Segmentation ◽  
Word Embeddings ◽  
Neural Architecture ◽  
Text Coherence ◽  
Sentence Level ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Cross Lingual

Breaking down the structure of long texts into semantically coherent segments makes the texts more readable and supports downstream applications like summarization and retrieval. Starting from an apparent link between text coherence and segmentation, we introduce a novel supervised model for text segmentation with simple but explicit coherence modeling. Our model – a neural architecture consisting of two hierarchically connected Transformer networks – is a multi-task learning model that couples the sentence-level segmentation objective with the coherence objective that differentiates correct sequences of sentences from corrupt ones. The proposed model, dubbed Coherence-Aware Text Segmentation (CATS), yields state-of-the-art segmentation performance on a collection of benchmark datasets. Furthermore, by coupling CATS with cross-lingual word embeddings, we demonstrate its effectiveness in zero-shot language transfer: it can successfully segment texts in languages unseen in training.

scholarly journals Multi-Sense Embeddings per Word

2020 ◽  
Author(s):  
Masashi Sugiyama
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Research Area ◽  
Word Embedding ◽  
The Other ◽  
Word Embeddings ◽  
Word Similarity ◽  
Better Than ◽  
Non Parametric

Recently, word embeddings have been used in many natural language processing problems successfully and how to train a robust and accurate word embedding system efficiently is a popular research area. Since many, if not all, words have more than one sense, it is necessary to learn vectors for all senses of word separately. Therefore, in this project, we have explored two multi-sense word embedding models, including Multi-Sense Skip-gram (MSSG) model and Non-parametric Multi-sense Skip Gram model (NP-MSSG). Furthermore, we propose an extension of the Multi-Sense Skip-gram model called Incremental Multi-Sense Skip-gram (IMSSG) model which could learn the vectors of all senses per word incrementally. We evaluate all the systems on word similarity task and show that IMSSG is better than the other models.

Sign in / Sign up

Export Citation Format

Share Document