scholarly journals Unseen Word Representation by Aligning Heterogeneous Lexical Semantic Spaces

2019 ◽  
Vol 33 ◽  
pp. 6900-6907 ◽  
Author(s):  
Victor Prokhorov ◽  
Mohammad Taher Pilehvar ◽  
Dimitri Kartsaklis ◽  
Pietro Lio ◽  
Nigel Collier
Keyword(s):  
Training Data ◽  
Lexical Resources ◽  
Rare Word ◽  
Word Similarity ◽  
Performance Improvements ◽  
Space Transformation ◽  
Word Representation ◽  
Consistent Performance ◽  
Cross Lingual

Word embedding techniques heavily rely on the abundance of training data for individual words. Given the Zipfian distribution of words in natural language texts, a large number of words do not usually appear frequently or at all in the training data. In this paper we put forward a technique that exploits the knowledge encoded in lexical resources, such as WordNet, to induce embeddings for unseen words. Our approach adapts graph embedding and cross-lingual vector space transformation techniques in order to merge lexical knowledge encoded in ontologies with that derived from corpus statistics. We show that the approach can provide consistent performance improvements across multiple evaluation benchmarks: in-vitro, on multiple rare word similarity datasets, and invivo, in two downstream text classification tasks.

scholarly journals Semantic Specialization of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints

2017 ◽  
Vol 5 ◽  
pp. 309-324 ◽  
Author(s):  
Nikola Mrkšić ◽  
Ivan Vulić ◽  
Diarmuid Ó Séaghdha ◽  
Ira Leviant ◽  
Roi Reichart ◽  
...  
Keyword(s):  
State Of The Art ◽  
Vector Spaces ◽  
Lexical Resources ◽  
High Quality ◽  
Performance Improvements ◽  
State Tracking ◽  
Cross Lingual ◽  
Semantic Transfer ◽  
Multiple Languages

We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialized cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialized vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.

scholarly journals Leveraging Pre-Trained Contextualized Word Embeddings to Enhance Sentiment Classification of Drug Reviews

2021 ◽  
Vol 35 (4) ◽  
pp. 307-314
Author(s):  
Redouane Karsi ◽  
Mounia Zaim ◽  
Jamila El Alami
Keyword(s):  
Sentiment Analysis ◽  
Classification Performance ◽  
Small Sample ◽  
Sentiment Classification ◽  
Training Data ◽  
Svm Classifier ◽  
Word Embeddings ◽  
Classification Problems ◽  
Lexical Resources ◽  
Word Representation

Traditionally, pharmacovigilance data are collected during clinical trials on a small sample of patients and are therefore insufficient to adequately assess drugs. Nowadays, consumers use online drug forums to share their opinions and experiences about medication. These feedbacks, which are widely available on the web, are automatically analyzed to extract relevant information for decision-making. Currently, sentiment analysis methods are being put forward to leverage consumers' opinions and produce useful drug monitoring indicators. However, these methods' effectiveness depends on the quality of word representation, which presents a real challenge because the information contained in user reviews is noisy and very subjective. Over time, several sentiment classification problems use machine learning methods based on the traditional bag of words model, sometimes enhanced with lexical resources. In recent years, word embedding models have significantly improved classification performance due to their ability to capture words' syntactic and semantic properties. Unfortunately, these latter models are weak in sentiment classification tasks because they are unable to encode sentiment information in the word representation. Indeed, two words with opposite polarities can have close word embeddings as they appear together in the same context. To overcome this drawback, some studies have proposed refining pre-trained word embeddings with lexical resources or learning word embeddings using training data. However, these models depend on external resources and are complex to implement. This work proposes a deep contextual word embeddings model called ELMo that inherently captures the sentiment information by providing separate vectors for words with opposite polarities. Different variants of our proposed model are compared with a benchmark of pre-trained word embeddings models using SVM classifier trained on Drug Review Dataset. Experimental results show that ELMo embeddings improve classification performance in sentiment analysis tasks on the pharmaceutical domain.

scholarly journals Alternative shoe closure designs alter biomechanical performance during agility-based movements

2020 ◽  
Author(s):  
Moira Pryhoda ◽  
Rachel Wathen ◽  
Jay Dicharry ◽  
Kevin Shelburne ◽  
Bradley Davidson
Keyword(s):  
Mechanical Properties ◽  
Performance Measures ◽  
Performance Measure ◽  
Male Athletes ◽  
Impaired Performance ◽  
Performance Improvements ◽  
Two Measures ◽  
Ground Contact Time ◽  
Improved Performance ◽  
Consistent Performance

The objective of this research was to determine if three alternative shoe upper closures improve biomechanical performance measures relative to a standard lace closure in court-based movements. NCAA Division 1 and club-level male athletes recruited from lacrosse, soccer, tennis, and rugby performed four court-based movements: Lateral Skater Jump repeats (LSJ), Countermovement Jump repeats (CMJ), Triangle Drop Step drill (TDS), and Anterior-Posterior drill (AP). Each athlete performed the movements in four shoe upper closures: Standard Closure, Lace Replacement, Y Wrap, and Tri Strap. Ground contact time, peak eccentric rate of force development (RFD), peak concentric GRF, peak concentric COM power, eccentric work, concentric work, and movement completion time were measured. Tri Strap saw improvements in four of seven biomechanical variables during CMJ and LSJ and one variable during TDS. Lace Replacement delivered improvements in one performance measure during CMJ, LSJ, and AP, and two variables in TDS. Y Wrap improved performance in three performance measures during LSJ and impaired performance in two measures during CMJ and three measures during AP. Tri Strap provided the most consistent performance improvements across all movements. This study allowed for the mechanical properties of the shoe lower to remain consistent across designs to examine if an alternative shoe upper closure could enhance performance. Our results indicate that increased proprioception and/or mechanical properties due to the alternative closures, especially Tri Strap, improves athlete performance, which concludes that the design of the shoe upper is an essential consideration in shoe design.

scholarly journals Antibody Complementarity Determining Region Design Using High-Capacity Machine Learning

2019 ◽  
Author(s):  
Ge Liu ◽  
Haoyang Zeng ◽  
Jonas Mueller ◽  
Brandon Carter ◽  
Ziheng Wang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Structural Information ◽  
High Capacity ◽  
Training Data ◽  
Proper Function ◽  
Integrative Approach ◽  
Machine Learning Method ◽  
Learning Method ◽  
Target Specificity

AbstractThe precise targeting of antibodies and other protein therapeutics is required for their proper function and the elimination of deleterious off-target effects. Often the molecular structure of a therapeutic target is unknown and randomized methods are used to design antibodies without a model that relates antibody sequence to desired properties. Here we present a machine learning method that can design human Immunoglobulin G (IgG) antibodies with target affinities that are superior to candidates from phage display panning experiments within a limited design budget. We also demonstrate that machine learning can improve target-specificity by the modular composition of models from different experimental campaigns, enabling a new integrative approach to improving target specificity. Our results suggest a new path for the discovery of therapeutic molecules by demonstrating that predictive and differentiable models of antibody binding can be learned from high-throughput experimental data without the need for target structural data.SignificanceAntibody based therapeutics must meet both affinity and specificity metrics, and existing in vitro methods for meeting these metrics are based upon randomization and empirical testing. We demonstrate that with sufficient target-specific training data machine learning can suggest novel antibody variable domain sequences that are superior to those observed during training. Our machine learning method does not require any target structural information. We further show that data from disparate antibody campaigns can be combined by machine learning to improve antibody specificity.

scholarly journals The data-driven Bulgarian WordNet: BTBWN

2018 ◽  
Author(s):  
Petya Osenova ◽  
Kiril Simov
Keyword(s):  
Information Retrieval ◽  
Machine Translation ◽  
Semantic Information ◽  
Data Driven ◽  
Lexical Resources ◽  
Multilingual Information Retrieval ◽  
Cross Lingual ◽  
Princeton Wordnet ◽  
Word Senses

The data-driven Bulgarian WordNet: BTBWNThe paper presents our work towards the simultaneous creation of a data-driven WordNet for Bulgarian and a manually annotated treebank with semantic information. Such an approach requires synchronization of the word senses in both - syntactic and lexical resources, without limiting the WordNet senses to the corpus or vice versa. Our strategy focuses on the identification of senses used in BulTreeBank, but the missing senses of a lemma also have been covered through exploration of bigger corpora. The identified senses have been organized in synsets for the Bulgarian WordNet. Then they have been aligned to the Princeton WordNet synsets. Various types of mappings are considered between both resources in a cross-lingual aspect and with respect to ensuring maximum connectivity and potential for incorporating the language specific concepts. The mapping between the two WordNets (English and Bulgarian) is a basis for applications such as machine translation and multilingual information retrieval. Oparty na danych WordNet bułgarski: BTBWNW artykule przedstawiono naszą pracę na rzecz jednoczesnej budowy opartego na danych wordnetu dla języka bułgarskiego oraz ręcznie oznaczonego informacjami semantycznymi banku drzew. Takie podejście wymaga uzgodnienia znaczeń słów zarówno w zasobach składniowych, jak i leksykalnych, bez ograniczania znaczeń umieszczanych w wordnecie do tych obecnych w korpusie, jak i odwrotnie. Nasza strategia koncentruje się na identyfikacji znaczeń stosowanych w BulTreeBank, przy czym brakujące znaczenia lematu zostały również zbadane przez zgłębienie większych korpusów. Zidentyfikowane znaczenia zostały zorganizowane w synsety bułgarskiego wordnetu, a następnie powiązane z synsetami Princeton WordNet. Rozmaite rodzaje rzutowań są rozpatrywane pomiędzy obydwoma zasobami w kontekście międzyjęzykowym, a także w odniesieniu do zapewnienia maksymalnej łączności i możliwości uwzględnienia pojęć specyficznych dla języka bułgarskiego. Rzutowanie między dwoma wordnetami (angielskim i bułgarskim) jest podstawą dla aplikacji, takich jak tłumaczenie maszynowe i wielojęzyczne wyszukiwanie informacji.

scholarly journals Learning Task-Specific Representation for Novel Words in Sequence Labeling

2019 ◽  
Author(s):  
Minlong Peng ◽  
Qi Zhang ◽  
Xiaoyu Xing ◽  
Tao Gui ◽  
Jinlan Fu ◽  
...  
Keyword(s):  
Empirical Studies ◽  
Named Entity Recognition ◽  
Learning Task ◽  
Training Data ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech Tagging ◽  
Sequence Labeling ◽  
Part Of Speech ◽  
Word Representation

Word representation is a key component in neural-network-based sequence labeling systems. However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance. This is commonly referred to as the out-of-vocabulary (OOV) problem. In this work, we address the OOV problem in sequence labeling using only training data of the task. To this end, we propose a novel method to predict representations for OOV words from their surface-forms (e.g., character sequence) and contexts. The method is specifically designed to avoid the error propagation problem suffered by existing approaches in the same paradigm. To evaluate its effectiveness, we performed extensive empirical studies on four part-of-speech tagging (POS) tasks and four named entity recognition (NER) tasks. Experimental results show that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.

scholarly journals Zero-Shot Learning for Cross-Lingual News Sentiment Classification

2020 ◽  
Vol 10 (17) ◽  
pp. 5993
Author(s):  
Andraž Pelicon ◽  
Marko Pranjić ◽  
Dragana Miljković ◽  
Blaž Škrlj ◽  
Senja Pollak
Keyword(s):  
Classification System ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Training Data ◽  
Test Set ◽  
Novel Technique ◽  
Analysis Task ◽  
Negative News ◽  
Cross Lingual ◽  
News Sentiment

In this paper, we address the task of zero-shot cross-lingual news sentiment classification. Given the annotated dataset of positive, neutral, and negative news in Slovene, the aim is to develop a news classification system that assigns the sentiment category not only to Slovene news, but to news in another language without any training data required. Our system is based on the multilingual BERTmodel, while we test different approaches for handling long documents and propose a novel technique for sentiment enrichment of the BERT model as an intermediate training step. With the proposed approach, we achieve state-of-the-art performance on the sentiment analysis task on Slovenian news. We evaluate the zero-shot cross-lingual capabilities of our system on a novel news sentiment test set in Croatian. The results show that the cross-lingual approach also largely outperforms the majority classifier, as well as all settings without sentiment enrichment in pre-training.

Deep Persian sentiment analysis: Cross-lingual training for low-resource languages

2020 ◽  
pp. 016555152096278
Author(s):  
Rouzbeh Ghasemi ◽  
Seyed Arad Ashrafi Asli ◽  
Saeedeh Momtazi
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Training Data ◽  
Target Language ◽  
Low Resource ◽  
Proposed Model ◽  
Significant Difference ◽  
Cross Lingual

With the advent of deep neural models in natural language processing tasks, having a large amount of training data plays an essential role in achieving accurate models. Creating valid training data, however, is a challenging issue in many low-resource languages. This problem results in a significant difference between the accuracy of available natural language processing tools for low-resource languages compared with rich languages. To address this problem in the sentiment analysis task in the Persian language, we propose a cross-lingual deep learning framework to benefit from available training data of English. We deployed cross-lingual embedding to model sentiment analysis as a transfer learning model which transfers a model from a rich-resource language to low-resource ones. Our model is flexible to use any cross-lingual word embedding model and any deep architecture for text classification. Our experiments on English Amazon dataset and Persian Digikala dataset using two different embedding models and four different classification networks show the superiority of the proposed model compared with the state-of-the-art monolingual techniques. Based on our experiment, the performance of Persian sentiment analysis improves 22% in static embedding and 9% in dynamic embedding. Our proposed model is general and language-independent; that is, it can be used for any low-resource language, once a cross-lingual embedding is available for the source–target language pair. Moreover, by benefitting from word-aligned cross-lingual embedding, the only required data for a reliable cross-lingual embedding is a bilingual dictionary that is available between almost all languages and the English language, as a potential source language.

scholarly journals Enzymatic Weight Update Algorithm for DNA-Based Molecular Learning

Molecules ◽  
2019 ◽  
Vol 24 (7) ◽  
pp. 1409 ◽  
Author(s):  
Christina Baek ◽  
Sang-Woo Lee ◽  
Beom-Jin Lee ◽  
Dong-Hyun Kwak ◽  
Byoung-Tak Zhang
Keyword(s):  
Machine Learning ◽  
Dna Sequences ◽  
Dna Nanotechnology ◽  
Search Space ◽  
Molecular Computing ◽  
Training Data ◽  
Molecular Systems ◽  
Novel Approach ◽  
Biological Substrates

Recent research in DNA nanotechnology has demonstrated that biological substrates can be used for computing at a molecular level. However, in vitro demonstrations of DNA computations use preprogrammed, rule-based methods which lack the adaptability that may be essential in developing molecular systems that function in dynamic environments. Here, we introduce an in vitro molecular algorithm that ‘learns’ molecular models from training data, opening the possibility of ‘machine learning’ in wet molecular systems. Our algorithm enables enzymatic weight update by targeting internal loop structures in DNA and ensemble learning, based on the hypernetwork model. This novel approach allows massively parallel processing of DNA with enzymes for specific structural selection for learning in an iterative manner. We also introduce an intuitive method of DNA data construction to dramatically reduce the number of unique DNA sequences needed to cover the large search space of feature sets. By combining molecular computing and machine learning the proposed algorithm makes a step closer to developing molecular computing technologies for future access to more intelligent molecular systems.

scholarly journals Genetic Programming for Evolving Similarity Functions for Clustering: Representations and Analysis

2020 ◽  
Vol 28 (4) ◽  
pp. 531-561 ◽  
Author(s):  
Andrew Lensen ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Genetic Programming ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Similarity Measures ◽  
Small Subset ◽  
Similarity Functions ◽  
New Approach ◽  
Performance Improvements ◽  
Consistent Performance ◽  
High Dimensional Datasets

Clustering is a difficult and widely studied data mining task, with many varieties of clustering algorithms proposed in the literature. Nearly all algorithms use a similarity measure such as a distance metric (e.g., Euclidean distance) to decide which instances to assign to the same cluster. These similarity measures are generally predefined and cannot be easily tailored to the properties of a particular dataset, which leads to limitations in the quality and the interpretability of the clusters produced. In this article, we propose a new approach to automatically evolving similarity functions for a given clustering algorithm by using genetic programming. We introduce a new genetic programming-based method which automatically selects a small subset of features (feature selection) and then combines them using a variety of functions (feature construction) to produce dynamic and flexible similarity functions that are specifically designed for a given dataset. We demonstrate how the evolved similarity functions can be used to perform clustering using a graph-based representation. The results of a variety of experiments across a range of large, high-dimensional datasets show that the proposed approach can achieve higher and more consistent performance than the benchmark methods. We further extend the proposed approach to automatically produce multiple complementary similarity functions by using a multi-tree approach, which gives further performance improvements. We also analyse the interpretability and structure of the automatically evolved similarity functions to provide insight into how and why they are superior to standard distance metrics.

Sign in / Sign up

Export Citation Format

Share Document