Towards an Embedding-Based Approach for the Geolocation of Texts and Users on Social Networks

2021 ◽  
pp. 206-234
Author(s):  
Sarra Hasni
Keyword(s):  
Neural Networks ◽  
Social Networks ◽  
Spatial Analysis ◽  
Recurrent Neural Networks ◽  
Contextual Information ◽  
Language Models ◽  
New Approach ◽  
Textual Data ◽  
Word Forms ◽  
Statistical Language Models

The geolocation task of textual data shared on social networks like Twitter attracts a progressive attention. Since those data are supported by advanced geographic information systems for multipurpose spatial analysis, new trends to extend the paradigm of geolocated data become more emergent. Differently from statistical language models that are widely adopted in prior works, the authors propose a new approach that is adopted to the geolocation of both tweets and users through the application of embedding models. The authors boost the geolocation strategy with a sequential modelling using recurrent neural networks to delimit the importance of words in tweets with respect to contextual information. They evaluate the power of this strategy in order to determine locations of unstructured texts that reflect unlimited user's writing styles. Especially, the authors demonstrate that semantic proprieties and word forms can be effective to geolocate texts without specifying local words or topics' descriptions per region.

scholarly journals Persian sentences to phoneme sequences conversion based on recurrent neural networks

2016 ◽  
Vol 6 (1) ◽  
pp. 219-225 ◽  
Author(s):  
Yasser Mohseni Behbahani ◽  
Bagher Babaali ◽  
Mussa Turdalyuly
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Contextual Information ◽  
Short Term ◽  
Persian Language ◽  
Short Vowels ◽  
Sequential Labeling ◽  
Long Short Term Memory

AbstractGrapheme to phoneme conversion is one of the main subsystems of Text-to-Speech (TTS) systems. Converting sequence of written words to their corresponding phoneme sequences for the Persian language is more challenging than other languages; because in the standard orthography of this language the short vowels are omitted and the pronunciation ofwords depends on their positions in a sentence. Common approaches used in the Persian commercial TTS systems have several modules and complicated models for natural language processing and homograph disambiguation that make the implementation harder as well as reducing the overall precision of system. In this paper we define the grapheme-to-phoneme conversion as a sequential labeling problem; and use the modified Recurrent Neural Networks (RNN) to create a smart and integrated model for this purpose. The recurrent networks are modified to be bidirectional and equipped with Long-Short Term Memory (LSTM) blocks to acquire most of the past and future contextual information for decision making. The experiments conducted in this paper show that in addition to having a unified structure the bidirectional RNN-LSTM has a good performance in recognizing the pronunciation of the Persian sentences with the precision more than 98 percent.

scholarly journals General methods of forecasting nonlinear nonstationary processes based on mathematical models using statistical data

2021 ◽  
pp. 79-86
Author(s):  
Oleg Belas ◽  
Andrii Belas
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Mathematical Models ◽  
Recurrent Neural Networks ◽  
Statistical Data ◽  
Economic Systems ◽  
Nonstationary Processes ◽  
General Technique ◽  
New Approach ◽  
Advantages And Disadvantages

The article considers the problem of forecasting nonlinear nonstationary processes, presented in the form of time series, which can describe the dynamics of processes in both technical and economic systems. The general technique of analysis of such data and construction of corresponding mathematical models based on autoregressive models and recurrent neural networks is described in detail. The technique is applied on practical examples while performing the comparative analysis of models of forecasting of quantity of channels of service of cellular subscribers for a given station and revealing advantages and disadvantages of each method. The need to improve the existing methodology and develop a new approach is formulated.

Combining recurrent neural networks and factored language models during decoding of code-Switching speech

2014 ◽  
Author(s):  
Heike Adel ◽  
Dominic Telaar ◽  
Ngoc Thang Vu ◽  
Katrin Kirchhoff ◽  
Tanja Schultz
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Code Switching ◽  
Language Models

Control of Recurrent Neural Networks Using Differential Minimax Game: The Stochastic Case

2010 ◽  
Author(s):  
Ziqian Liu ◽  
Nirwan Ansari
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
New Approach ◽  
Numerical Example ◽  
Inverse Optimality ◽  
Stochastic Case ◽  
Stochastic Input ◽  
Minimax Game ◽  
Unknown Covariance ◽  
Lyapunov Technique

As a continuation of our study, this paper extends our research results of optimality-oriented stabilization from deterministic recurrent neural networks to stochastic recurrent neural networks, and presents a new approach to achieve optimally stochastic input-to-state stabilization in probability for stochastic recurrent neural networks driven by noise of unknown covariance. This approach is developed by using stochastic differential minimax game, Hamilton-Jacobi-Isaacs (HJI) equation, inverse optimality, and Lyapunov technique. A numerical example is given to demonstrate the effectiveness of the proposed approach.

scholarly journals Representation of Linguistic Form and Function in Recurrent Neural Networks

2017 ◽  
Vol 43 (4) ◽  
pp. 761-780 ◽  
Author(s):  
Ákos Kádár ◽  
Grzegorz Chrupała ◽  
Afra Alishahi
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Language Model ◽  
Visual Pathway ◽  
Recurrent Network ◽  
Point Of View ◽  
Language Models ◽  
Grammatical Function ◽  
Form And Function

We present novel methods for analyzing the activation patterns of recurrent neural networks from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a standard standalone language model, and a multi-task gated recurrent network architecture consisting of two parallel pathways with shared word embeddings: The Visual pathway is trained on predicting the representations of the visual scene corresponding to an input sentence, and the Textual pathway is trained to predict the next word in the same sentence. We propose a method for estimating the amount of contribution of individual tokens in the input to the final prediction of the networks. Using this method, we show that the Visual pathway pays selective attention to lexical categories and grammatical functions that carry semantic information, and learns to treat word types differently depending on their grammatical function and their position in the sequential structure of the sentence. In contrast, the language models are comparatively more sensitive to words with a syntactic function. Further analysis of the most informative n-gram contexts for each model shows that in comparison with the Visual pathway, the language models react more strongly to abstract contexts that represent syntactic constructions.

scholarly journals Gene Expression Analysis using Markov Chains extracted from RNNs

2007 ◽  
Vol 10 (2) ◽  
Author(s):  
Igor Lorenzato Almeida ◽  
Denise Regina Pechmann ◽  
Adelmo Luis Cechin
Keyword(s):  
Gene Expression ◽  
Neural Networks ◽  
Markov Chains ◽  
Microarray Data ◽  
Recurrent Neural Networks ◽  
Gene Expression Analysis ◽  
Data Sets ◽  
New Approach ◽  
Several Variables ◽  
Gene Expression Levels

This paper present a new approach for the analysis of gene expres- sion, by extracting a Markov Chain from trained Recurrent Neural Networks (RNNs). A lot of microarray data is being generated, since array technologies have been widely used to monitor simultaneously the expression pattern of thou- sands of genes. Microarray data is highly specialized, involves several variables in which are complex to express and analyze. The challenge is to discover how to extract useful information from these data sets. So this work proposes the use of RNNs for data modeling, due to their ability to learn complex temporal non-linear data. Once a model is obtained for the data, it is possible to ex- tract the acquired knowledge and to represent it through Markov Chains model. Markov Chains are easily visualized in the form of states graphs, which show the influences among the gene expression levels and their changes in time

Sign in / Sign up

Export Citation Format

Share Document