scholarly journals AutoSense Model for Word Sense Induction

2019 ◽  
Vol 33 ◽  
pp. 6212-6219 ◽  
Author(s):  
Reinald Kim Amplayo ◽  
Seung-won Hwang ◽  
Min Song
Keyword(s):  
Latent Variable ◽  
Word Sense ◽  
Name Disambiguation ◽  
Variable Model ◽  
Fine Grained ◽  
Word Sense Induction ◽  
Author Name Disambiguation ◽  
Competing Models ◽  
Word Senses ◽  
Better Than

Word sense induction (WSI), or the task of automatically discovering multiple senses or meanings of a word, has three main challenges: domain adaptability, novel sense detection, and sense granularity flexibility. While current latent variable models are known to solve the first two challenges, they are not flexible to different word sense granularities, which differ very much among words, from aardvark with one sense, to play with over 50 senses. Current models either require hyperparameter tuning or nonparametric induction of the number of senses, which we find both to be ineffective. Thus, we aim to eliminate these requirements and solve the sense granularity problem by proposing AutoSense, a latent variable model based on two observations: (1) senses are represented as a distribution over topics, and (2) senses generate pairings between the target word and its neighboring word. These observations alleviate the problem by (a) throwing garbage senses and (b) additionally inducing fine-grained word senses. Results show great improvements over the stateof-the-art models on popular WSI datasets. We also show that AutoSense is able to learn the appropriate sense granularity of a word. Finally, we apply AutoSense to the unsupervised author name disambiguation task where the sense granularity problem is more evident and show that AutoSense is evidently better than competing models. We share our data and code here: https://github.com/rktamplayo/AutoSense.

scholarly journals A Flow-Based Deep Latent Variable Model for Speech Spectrogram Modeling and Enhancement

2020 ◽  
Author(s):  
Aditya Arie Nugraha ◽  
Kouhei Sekiguchi ◽  
Kazuyoshi Yoshii
Keyword(s):  
Speech Enhancement ◽  
Latent Variables ◽  
Latent Variable ◽  
Generative Models ◽  
Latent Variable Model ◽  
Variable Model ◽  
Variational Autoencoder ◽  
Latent Representations ◽  
Low Dimensional ◽  
Better Than

This paper describes a deep latent variable model of speech power spectrograms and its application to semi-supervised speech enhancement with a deep speech prior. By integrating two major deep generative models, a variational autoencoder (VAE) and a normalizing flow (NF), in a mutually-beneficial manner, we formulate a flexible latent variable model called the NF-VAE that can extract low-dimensional latent representations from high-dimensional observations, akin to the VAE, and does not need to explicitly represent the distribution of the observations, akin to the NF. In this paper, we consider a variant of NF called the generative flow (GF a.k.a. Glow) and formulate a latent variable model called the GF-VAE. We experimentally show that the proposed GF-VAE is better than the standard VAE at capturing fine-structured harmonics of speech spectrograms, especially in the high-frequency range. A similar finding is also obtained when the GF-VAE and the VAE are used to generate speech spectrograms from latent variables randomly sampled from the standard Gaussian distribution. Lastly, when these models are used as speech priors for statistical multichannel speech enhancement, the GF-VAE outperforms the VAE and the GF.

Multi-Modal Word Synset Induction

2017 ◽  
Author(s):  
Jesse Thomason ◽  
Raymond J. Mooney
Keyword(s):  
Natural Language ◽  
Noun Phrase ◽  
Noun Phrases ◽  
Word Sense ◽  
Word Sense Induction ◽  
Multiple Meanings ◽  
The Senses ◽  
Polysemous Words ◽  
Word Senses

A word in natural language can be polysemous, having multiple meanings, as well as synonymous, meaning the same thing as other words. Word sense induction attempts to find the senses of polysemous words. Synonymy detection attempts to find when two words are interchangeable. We combine these tasks, first inducing word senses and then detecting similar senses to form word-sense synonym sets (synsets) in an unsupervised fashion. Given pairs of images and text with noun phrase labels, we perform synset induction to produce collections of underlying concepts described by one or more noun phrases. We find that considering multi-modal features from both visual and textual context yields better induced synsets than using either context alone. Human evaluations show that our unsupervised, multi-modally induced synsets are comparable in quality to annotation-assisted ImageNet synsets, achieving about 84% of ImageNet synsets' approval.

Unsupervised Acquisition of Predominant Word Senses

2007 ◽  
Vol 33 (4) ◽  
pp. 553-590 ◽  
Author(s):  
Diana McCarthy ◽  
Rob Koeling ◽  
Julie Weeds ◽  
John Carroll
Keyword(s):  
Word Sense Disambiguation ◽  
Semantic Interpretation ◽  
Word Sense ◽  
Domain Specific ◽  
Sense Disambiguation ◽  
Word Sense Ambiguity ◽  
Word Token ◽  
Low Coverage ◽  
Word Senses ◽  
Better Than

There has been a great deal of recent research into word sense disambiguation, particularly since the inception of the Senseval evaluation exercises. Because a word often has more than one meaning, resolving word sense ambiguity could benefit applications that need some level of semantic interpretation of language input. A major problem is that the accuracy of word sense disambiguation systems is strongly dependent on the quantity of manually sense-tagged data available, and even the best systems, when tagging every word token in a document, perform little better than a simple heuristic that guesses the first, or predominant, sense of a word in all contexts. The success of this heuristic is due to the skewed nature of word sense distributions. Data for the heuristic can come from either dictionaries or a sample of sense-tagged data. However, there is a limited supply of the latter, and the sense distributions and predominant sense of a word can depend on the domain or source of a document. (The first sense of “star” for example would be different in the popular press and scientific journals). In this article, we expand on a previously proposed method for determining the predominant sense of a word automatically from raw text. We look at a number of different data sources and parameterizations of the method, using evaluation results and error analyses to identify where the method performs well and also where it does not. In particular, we find that the method does not work as well for verbs and adverbs as nouns and adjectives, but produces more accurate predominant sense information than the widely used SemCor corpus for nouns with low coverage in that corpus. We further show that the method is able to adapt successfully to domains when using domain specific corpora as input and where the input can either be hand-labeled for domain or automatically classified.

scholarly journals A Flow-Based Deep Latent Variable Model for Speech Spectrogram Modeling and Enhancement

2020 ◽  
Author(s):  
Aditya Arie Nugraha ◽  
Kouhei Sekiguchi ◽  
Kazuyoshi Yoshii
Keyword(s):  
Speech Enhancement ◽  
Latent Variables ◽  
Latent Variable ◽  
Generative Models ◽  
Latent Variable Model ◽  
Variable Model ◽  
Variational Autoencoder ◽  
Latent Representations ◽  
Low Dimensional ◽  
Better Than

This paper describes a deep latent variable model of speech power spectrograms and its application to semi-supervised speech enhancement with a deep speech prior. By integrating two major deep generative models, a variational autoencoder (VAE) and a normalizing flow (NF), in a mutually-beneficial manner, we formulate a flexible latent variable model called the NF-VAE that can extract low-dimensional latent representations from high-dimensional observations, akin to the VAE, and does not need to explicitly represent the distribution of the observations, akin to the NF. In this paper, we consider a variant of NF called the generative flow (GF a.k.a. Glow) and formulate a latent variable model called the GF-VAE. We experimentally show that the proposed GF-VAE is better than the standard VAE at capturing fine-structured harmonics of speech spectrograms, especially in the high-frequency range. A similar finding is also obtained when the GF-VAE and the VAE are used to generate speech spectrograms from latent variables randomly sampled from the standard Gaussian distribution. Lastly, when these models are used as speech priors for statistical multichannel speech enhancement, the GF-VAE outperforms the VAE and the GF.

scholarly journals ALaSca: an Automated approach for Large-Scale Lexical Substitution

2021 ◽  
Author(s):  
Caterina Lacerra ◽  
Tommaso Pasini ◽  
Rocco Tripodi ◽  
Roberto Navigli
Keyword(s):  
Large Scale ◽  
Full Potential ◽  
Word Sense ◽  
Practical Applications ◽  
Word Sense Induction ◽  
Text Simplification ◽  
Novel Approach ◽  
Potential Benefits ◽  
Lexical Substitution ◽  
Better Than

The lexical substitution task aims at finding suitable replacements for words in context. It has proved to be useful in several areas, such as word sense induction and text simplification, as well as in more practical applications such as writing-assistant tools. However, the paucity of annotated data has forced researchers to apply mainly unsupervised approaches, limiting the applicability of large pre-trained models and thus hampering the potential benefits of supervised approaches to the task. In this paper, we mitigate this issue by proposing ALaSca, a novel approach to automatically creating large-scale datasets for English lexical substitution. ALaSca allows examples to be produced for potentially any word in a language vocabulary and to cover most of the meanings it lists. Thanks to this, we can unleash the full potential of neural architectures and finetune them on the lexical substitution task. Indeed, when using our data, a transformer-based model performs substantially better than when using manually annotated data only. We release ALaSca at https://sapienzanlp.github.io/alasca/.

scholarly journals Document Representation with Statistical Word Senses in Cross-Lingual Document Clustering

2015 ◽  
Vol 29 (02) ◽  
pp. 1559003 ◽  
Author(s):  
Guoyu Tang ◽  
Yunqing Xia ◽  
Erik Cambria ◽  
Peng Jin ◽  
Thomas Fang Zheng
Keyword(s):  
Latent Dirichlet Allocation ◽  
State Of The Art ◽  
Document Clustering ◽  
Word Sense ◽  
Document Representation ◽  
Word Sense Induction ◽  
Translation Ambiguity ◽  
Cross Lingual ◽  
Word Senses ◽  
Induction Model

Cross-lingual document clustering is the task of automatically organizing a large collection of multi-lingual documents into a few clusters, depending on their content or topic. It is well known that language barrier and translation ambiguity are two challenging issues for cross-lingual document representation. To this end, we propose to represent cross-lingual documents through statistical word senses, which are automatically discovered from a parallel corpus through a novel cross-lingual word sense induction model and a sense clustering method. In particular, the former consists in a sense-based vector space model and the latter leverages on a sense-based latent Dirichlet allocation. Evaluation on the benchmarking datasets shows that the proposed models outperform two state-of-the-art methods for cross-lingual document clustering.

scholarly journals Clustering and Diversifying Web Search Results with Graph-Based Word Sense Induction

2013 ◽  
Vol 39 (3) ◽  
pp. 709-754 ◽  
Author(s):  
Antonio Di Marco ◽  
Roberto Navigli
Keyword(s):  
Information Search ◽  
Web Search ◽  
Word Sense ◽  
Clustering Methods ◽  
Search Results ◽  
Search Result ◽  
Word Sense Induction ◽  
Novel Approach ◽  
Web Clustering ◽  
Word Senses

Web search result clustering aims to facilitate information search on the Web. Rather than the results of a query being presented as a flat list, they are grouped on the basis of their similarity and subsequently shown to the user as a list of clusters. Each cluster is intended to represent a different meaning of the input query, thus taking into account the lexical ambiguity (i.e., polysemy) issue. Existing Web clustering methods typically rely on some shallow notion of textual similarity between search result snippets, however. As a result, text snippets with no word in common tend to be clustered separately even if they share the same meaning, whereas snippets with words in common may be grouped together even if they refer to different meanings of the input query. In this article we present a novel approach to Web search result clustering based on the automatic discovery of word senses from raw text, a task referred to as Word Sense Induction. Key to our approach is to first acquire the various senses (i.e., meanings) of an ambiguous query and then cluster the search results based on their semantic similarity to the word senses induced. Our experiments, conducted on data sets of ambiguous queries, show that our approach outperforms both Web clustering and search engines.

scholarly journals The Multilayer Network Approach in the Study of Personality Neuroscience

2020 ◽  
Vol 10 (12) ◽  
pp. 915
Author(s):  
Dora Brooks ◽  
Hanneke E. Hulst ◽  
Leon de Bruin ◽  
Gerrit Glas ◽  
Jeroen J. G. Geurts ◽  
...  
Keyword(s):  
Complex System ◽  
Individual Differences ◽  
Latent Variable ◽  
Network Approach ◽  
Top Down ◽  
Multilayer Network ◽  
Variable Model ◽  
Psychological Traits ◽  
Fine Grained ◽  
Cognition And Emotion

It has long been understood that a multitude of biological systems, from genetics, to brain networks, to psychological factors, all play a role in personality. Understanding how these systems interact with each other to form both relatively stable patterns of behaviour, cognition and emotion, but also vast individual differences and psychiatric disorders, however, requires new methodological insight. This article explores a way in which to integrate multiple levels of personality simultaneously, with particular focus on its neural and psychological constituents. It does so first by reviewing the current methodology of studies used to relate the two levels, where psychological traits, often defined with a latent variable model are used as higher-level concepts to identify the neural correlates of personality (NCPs). This is known as a top-down approach, which though useful in revealing correlations, is not able to include the fine-grained interactions that occur at both levels. As an alternative, we discuss the use of a novel complex system approach known as a multilayer network, a technique that has recently proved successful in revealing veracious interactions between networks at more than one level. The benefits of the multilayer approach to the study of personality neuroscience follow from its well-founded theoretical basis in network science. Its predictive and descriptive power may surpass that of statistical top-down and latent variable models alone, potentially allowing the discernment of more complete descriptions of individual differences, and psychiatric and neurological changes that accompany disease. Though in its infancy, and subject to a number of methodological unknowns, we argue that the multilayer network approach may contribute to an understanding of personality as a complex system comprised of interrelated psychological and neural features.

Sign in / Sign up

Export Citation Format

Share Document