scholarly journals Using the Outlier Detection Task to Evaluate Distributional Semantic Models

2018 ◽  
Vol 1 (1) ◽  
pp. 211-223
Author(s):  
Pablo Gamallo
Keyword(s):  
Outlier Detection ◽  
Detection Task ◽  
Bag Of Words ◽  
Word Embeddings ◽  
Specific Task ◽  
Semantic Models ◽  
Distributional Semantic Models ◽  
Syntactic Dependencies ◽  
Better Than

In this article, we define the outlier detection task and use it to compare neural-based word embeddings with transparent count-based distributional representations. Using the English Wikipedia as a text source to train the models, we observed that embeddings outperform count-based representations when their contexts are made up of bag-of-words. However, there are no sharp differences between the two models if the word contexts are defined as syntactic dependencies. In general, syntax-based models tend to perform better than those based on bag-of-words for this specific task. Similar experiments were carried out for Portuguese with similar results. The test datasets we have created for the outlier detection task in English and Portuguese are freely available.

scholarly journals Decoding Brain Activity Associated with Literal and Metaphoric Sentence Comprehension Using Distributional Semantic Models

2020 ◽  
Vol 8 ◽  
pp. 231-246
Author(s):  
Vesna G. Djokic ◽  
Jean Maillard ◽  
Luana Bulat ◽  
Ekaterina Shutova
Keyword(s):  
Language Processing ◽  
Sentence Comprehension ◽  
Brain Activity ◽  
Activity Patterns ◽  
Word Embeddings ◽  
Semantic Models ◽  
Distributional Semantic Models ◽  
Metaphor Processing ◽  
Meaning Representation ◽  
The Brain

Recent years have seen a growing interest within the natural language processing (NLP) community in evaluating the ability of semantic models to capture human meaning representation in the brain. Existing research has mainly focused on applying semantic models to decode brain activity patterns associated with the meaning of individual words, and, more recently, this approach has been extended to sentences and larger text fragments. Our work is the first to investigate metaphor processing in the brain in this context. We evaluate a range of semantic models (word embeddings, compositional, and visual models) in their ability to decode brain activity associated with reading of both literal and metaphoric sentences. Our results suggest that compositional models and word embeddings are able to capture differences in the processing of literal and metaphoric sentences, providing support for the idea that the literal meaning is not fully accessible during familiar metaphor comprehension.

scholarly journals Enhancing Semantic Representations of Bilingual Word Embeddings with Syntactic Dependencies

2018 ◽  
Author(s):  
Linli Xu ◽  
Wenjun Ouyang ◽  
Xiaoying Ren ◽  
Yang Wang ◽  
Liang Jiang
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Real World ◽  
Bag Of Words ◽  
Word Embeddings ◽  
Word Level ◽  
Proposed Model ◽  
Parse Trees ◽  
Cross Lingual ◽  
Syntactic Dependencies

Cross-lingual representation is a technique that can both represent different languages in the same latent vector space and enable the knowledge transfer across languages. To learn such representations, most of existing works require parallel sentences with word-level alignments and assume that aligned words have similar Bag-of-Words (BoW) contexts.  However, due to differences in grammar structures among different languages, the contexts of aligned words in different languages may appear at different positions of the sentence. To address this issue of different syntactics across different languages, we propose a model of bilingual word embeddings integrating syntactic dependencies (DepBiWE) by producing dependency parse-trees which encode the accurate relative positions for the contexts of aligned words. In addition, a new method is proposed to learn bilingual word embeddings from dependency-based contexts and BoW contexts jointly. Extensive experimental results on a real world dataset clearly validate the superiority of the proposed model DepBiWE on various natural language processing (NLP) tasks.

Detecting linguistic idiosyncratic interests in autism using distributional semantic models

2014 ◽  
Author(s):  
Masoud Rouhizadeh ◽  
Emily Prud'hommeaux ◽  
Jan van Santen ◽  
Richard Sproat
Keyword(s):  
Semantic Models ◽  
Distributional Semantic Models

scholarly journals Unsupervised Compositionality Prediction of Nominal Compounds

2019 ◽  
Vol 45 (1) ◽  
pp. 1-57 ◽  
Author(s):  
Silvio Cordeiro ◽  
Aline Villavicencio ◽  
Marco Idiart ◽  
Carlos Ramisch
Keyword(s):  
Morphological Variation ◽  
Red Wine ◽  
Data Sets ◽  
High Agreement ◽  
Semantic Models ◽  
Corpus Size ◽  
Nominal Compounds ◽  
Distributional Semantic Models ◽  
The Impact

Nominal compounds such as red wine and nut case display a continuum of compositionality, with varying contributions from the components of the compound to its semantics. This article proposes a framework for compound compositionality prediction using distributional semantic models, evaluating to what extent they capture idiomaticity compared to human judgments. For evaluation, we introduce data sets containing human judgments in three languages: English, French, and Portuguese. The results obtained reveal a high agreement between the models and human predictions, suggesting that they are able to incorporate information about idiomaticity. We also present an in-depth evaluation of various factors that can affect prediction, such as model and corpus parameters and compositionality operations. General crosslingual analyses reveal the impact of morphological variation and corpus size in the ability of the model to predict compositionality, and of a uniform combination of the components for best results.

Distance Based Pattern Driven Mining for Outlier Detection in High Dimensional Big Dataset

2022 ◽  
Vol 13 (1) ◽  
pp. 1-17
Author(s):  
Ankit Kumar ◽  
Abhishek Kumar ◽  
Ali Kashif Bashir ◽  
Mamoon Rashid ◽  
V. D. Ambeth Kumar ◽  
...  
Keyword(s):  
Data Mining ◽  
Comparative Analysis ◽  
Outlier Detection ◽  
Credit Card ◽  
High Dimensional ◽  
Work Efficiency ◽  
Average Value ◽  
Novel Method ◽  
Detection Of Outliers ◽  
Better Than

Detection of outliers or anomalies is one of the vital issues in pattern-driven data mining. Outlier detection detects the inconsistent behavior of individual objects. It is an important sector in the data mining field with several different applications such as detecting credit card fraud, hacking discovery and discovering criminal activities. It is necessary to develop tools used to uncover the critical information established in the extensive data. This paper investigated a novel method for detecting cluster outliers in a multidimensional dataset, capable of identifying the clusters and outliers for datasets containing noise. The proposed method can detect the groups and outliers left by the clustering process, like instant irregular sets of clusters (C) and outliers (O), to boost the results. The results obtained after applying the algorithm to the dataset improved in terms of several parameters. For the comparative analysis, the accurate average value and the recall value parameters are computed. The accurate average value is 74.05% of the existing COID algorithm, and our proposed algorithm has 77.21%. The average recall value is 81.19% and 89.51% of the existing and proposed algorithm, which shows that the proposed work efficiency is better than the existing COID algorithm.

scholarly journals Extraction of Terms Related to Named Rivers

Languages ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. 46
Author(s):  
Juan ◽  
Faber
Keyword(s):  
Knowledge Base ◽  
Mississippi River ◽  
Environmental Science ◽  
Automatic Method ◽  
Statistical Procedures ◽  
Conceptual Structures ◽  
Semantic Models ◽  
Semantic Frames ◽  
Conceptual Network ◽  
Distributional Semantic Models

EcoLexicon is a terminological knowledge base on environmental science, whose design permits the geographic contextualization of data. For the geographic contextualization of landform concepts, this paper presents a semi-automatic method for extracting terms associated with named rivers (e.g., Mississippi River). Terms were extracted from a specialized corpus, where named rivers were automatically identified. Statistical procedures were applied for selecting both terms and rivers in distributional semantic models to construct the conceptual structures underlying the usage of named rivers. The rivers sharing associated terms were also clustered and represented in the same conceptual network. The results showed that the method successfully described the semantic frames of named rivers with explanatory adequacy, according to the premises of Frame-Based Terminology.

scholarly journals Outlier Detection Based on Residual Histogram Preference for Geometric Multi-Model Fitting

Sensors ◽  
2020 ◽  
Vol 20 (11) ◽  
pp. 3037
Author(s):  
Xi Zhao ◽  
Yun Zhang ◽  
Shoulie Xie ◽  
Qianqing Qin ◽  
Shiqian Wu ◽  
...  
Keyword(s):  
Outlier Detection ◽  
State Of The Art ◽  
Geometric Model ◽  
Model Fitting ◽  
Distribution Model ◽  
Data Sets ◽  
Analysis Method ◽  
Detection Scheme ◽  
The Impact ◽  
Better Than

Geometric model fitting is a fundamental issue in computer vision, and the fitting accuracy is affected by outliers. In order to eliminate the impact of the outliers, the inlier threshold or scale estimator is usually adopted. However, a single inlier threshold cannot satisfy multiple models in the data, and scale estimators with a certain noise distribution model work poorly in geometric model fitting. It can be observed that the residuals of outliers are big for all true models in the data, which makes the consensus of the outliers. Based on this observation, we propose a preference analysis method based on residual histograms to study the outlier consensus for outlier detection in this paper. We have found that the outlier consensus makes the outliers gather away from the inliers on the designed residual histogram preference space, which is quite convenient to separate outliers from inliers through linkage clustering. After the outliers are detected and removed, a linkage clustering with permutation preference is introduced to segment the inliers. In addition, in order to make the linkage clustering process stable and robust, an alternative sampling and clustering framework is proposed in both the outlier detection and inlier segmentation processes. The experimental results also show that the outlier detection scheme based on residual histogram preference can detect most of the outliers in the data sets, and the fitting results are better than most of the state-of-the-art methods in geometric multi-model fitting.

scholarly journals A structured distributional model of sentence meaning and processing

2019 ◽  
Vol 25 (4) ◽  
pp. 483-502 ◽  
Author(s):  
E. Chersoni ◽  
E. Santus ◽  
L. Pannitto ◽  
A. Lenci ◽  
P. Blache ◽  
...  
Keyword(s):  
Semantic Representation ◽  
Formal Semantics ◽  
Formal Structure ◽  
Data Sets ◽  
Word Embeddings ◽  
Discourse Representation Theory ◽  
Event Knowledge ◽  
Sentence Meaning ◽  
Generalized Knowledge ◽  
Distributional Semantic Models

AbstractMost compositional distributional semantic models represent sentence meaning with a single vector. In this paper, we propose a structured distributional model (SDM) that combines word embeddings with formal semantics and is based on the assumption that sentences represent events and situations. The semantic representation of a sentence is a formal structure derived from discourse representation theory and containing distributional vectors. This structure is dynamically and incrementally built by integrating knowledge about events and their typical participants, as they are activated by lexical items. Event knowledge is modelled as a graph extracted from parsed corpora and encoding roles and relationships between participants that are represented as distributional vectors. SDM is grounded on extensive psycholinguistic research showing that generalized knowledge about events stored in semantic memory plays a key role in sentence comprehension.We evaluate SDMon two recently introduced compositionality data sets, and our results show that combining a simple compositionalmodel with event knowledge constantly improves performances, even with dif ferent types of word embeddings.

scholarly journals Word Vector Models Approach to Text Regression of Financial Risk Prediction

Symmetry ◽  
2020 ◽  
Vol 12 (1) ◽  
pp. 89 ◽  
Author(s):  
Hsiang-Yuan Yeh ◽  
Yu-Ching Yeh ◽  
Da-Bai Shen
Keyword(s):  
Financial Risk ◽  
Information Economics ◽  
Bag Of Words ◽  
Word Embeddings ◽  
Textual Information ◽  
Financial Reports ◽  
Domain Specific ◽  
Regulatory Changes ◽  
Wide Range ◽  
Vector Representations

Linking textual information in finance reports to the stock return volatility provides a perspective on exploring useful insights for risk management. We introduce different kinds of word vector representations in the modeling of textual information: bag-of-words, pre-trained word embeddings, and domain-specific word embeddings. We apply linear and non-linear methods to establish a text regression model for volatility prediction. A large number of collected annually-published financial reports in the period from 1996 to 2013 is used in the experiments. We demonstrate that the domain-specific word vector learned from data not only captures lexical semantics, but also has better performance than the pre-trained word embeddings and traditional bag-of-words model. Our approach significantly outperforms with smaller prediction error in the regression task and obtains a 4%–10% improvement in the ranking task compared to state-of-the-art methods. These improvements suggest that the textual information may provide measurable effects on long-term volatility forecasting. In addition, we also find that the variations and regulatory changes in reports make older reports less relevant for volatility prediction. Our approach opens a new method of research into information economics and can be applied to a wide range of financial-related applications.

Sign in / Sign up

Export Citation Format

Share Document