scholarly journals Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems

2019 ◽  
Vol 10 (1) ◽  
pp. 1-19
Author(s):  
Matthieu Riou ◽  
Bassam Jabaian ◽  
Stéphane Huet ◽  
Fabrice Lefèvre
Keyword(s):  
Natural Language ◽  
Short Term Memory ◽  
Network Models ◽  
Dialogue Systems ◽  
Neural Network Models ◽  
Spoken Dialogue Systems ◽  
Spoken Dialogue ◽  
Automatic Transcription ◽  
Long Short Term Memory ◽  
Expected Payoffs

Following some recent propositions to handle natural language generation in spoken dialogue systems with long short-term memory recurrent neural network models~\citep{Wen2016a} we first investigate a variant thereof with the objective of a better integration of the attention subnetwork. Then our next objective is to propose and evaluate a framework to adapt the NLG module online through direct interactions with the users. When doing so the basic way is to ask the user to utter an alternative sentence to express a particular dialogue act. But then the system has to decide between using an automatic transcription or to ask for a manual transcription. To do so a reinforcement learning approach based on an adversarial bandit scheme is retained. We show that by defining appropriately the rewards as a linear combination of expected payoffs and costs of acquiring the new data provided by the user, a system design can balance between improving the system's performance towards a better match with the user's preferences and the burden associated with it. Then the actual benefits of this system is assessed with a human evaluation, showing that the addition of more diverse utterances allows to produce sentences more satisfying for the user.

Deep Neural Models and Retrofitting for Arabic Text Categorization

2020 ◽  
Vol 16 (2) ◽  
pp. 74-86 ◽  
Author(s):  
Fatima-Zahra El-Alami ◽  
Said Ouatik El Alaoui ◽  
Noureddine En-Nahnahi
Keyword(s):  
Text Categorization ◽  
Short Term Memory ◽  
Network Models ◽  
Arabic Text ◽  
Short Term ◽  
Lexical Resources ◽  
Neural Network Models ◽  
Online Data ◽  
Long Short Term Memory ◽  
Vector Representations

Arabic text categorization is an important task in text mining particularly with the fast-increasing quantity of the Arabic online data. Deep neural network models have shown promising performance and indicated great data modeling capacities in managing large and substantial datasets. This article investigates convolution neural networks (CNNs), long short-term memory (LSTM) and their combination for Arabic text categorization. This work additionally handles the morphological variety of Arabic words by exploring the word embeddings model using position weights and subword information. To guarantee the nearest vector representations for connected words, this article adopts a strategy for refining Arabic vector space representations using semantic information embedded in lexical resources. Several experiments utilizing different architectures have been conducted on the OSAC dataset. The obtained results show the effectiveness of CNN-LSTM without and with retrofitting for Arabic text categorization in comparison with major competing methods.

scholarly journals Exploring the Sensitivity of Recurrent Neural Network Models for Forecasting Land Cover Change

Land ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 282
Author(s):  
Alysha van Duynhoven ◽  
Suzana Dragićević
Keyword(s):  
Land Cover ◽  
Short Term Memory ◽  
Network Models ◽  
Optimization Techniques ◽  
Neural Network Models ◽  
Method Performance ◽  
Timeseries Analysis ◽  
Method Optimization ◽  
Long Short Term Memory ◽  
Real World Datasets

Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) architectures, have obtained successful outcomes in timeseries analysis tasks. While RNNs demonstrated favourable performance for Land Cover (LC) change analyses, few studies have explored or quantified the geospatial data characteristics required to utilize this method. Likewise, many studies utilize overall measures of accuracy rather than metrics accounting for the slow or sparse changes of LC that are typically observed. Therefore, the main objective of this study is to evaluate the performance of LSTM models for forecasting LC changes by conducting a sensitivity analysis involving hypothetical and real-world datasets. The intent of this assessment is to explore the implications of varying temporal resolutions and LC classes. Additionally, changing these input data characteristics impacts the number of timesteps and LC change rates provided to the respective models. Kappa variants are selected to explore the capacity of LSTM models for forecasting transitions or persistence of LC. Results demonstrate the adverse effects of coarser temporal resolutions and high LC class cardinality on method performance, despite method optimization techniques applied. This study suggests various characteristics of geospatial datasets that should be present before considering LSTM methods for LC change forecasting.

scholarly journals Comparing Neural Network Models for Photovoltaic Power Generation Prediction

2021 ◽  
Author(s):  
Carlos Henrique Torres Andrade ◽  
Tiago Figueiredo Vieira ◽  
Ícaro Bezzera Queiroz Araújo ◽  
Gustavo Costa Gomes Melo ◽  
Erick de Andrade Barboza ◽  
...  
Keyword(s):  
Alternative Energy ◽  
Fossil Fuels ◽  
Short Term Memory ◽  
Network Models ◽  
Optimization Approach ◽  
Neural Network Models ◽  
Alternative Energy Sources ◽  
Comparison Results ◽  
Long Short Term Memory ◽  
The City

Research on alternative energy sources has been increasing for the past years due to environmental concerns and the depletion of fossil fuels. Since photovoltaic generation is intermittent, one needs to predict solar incidence to alleviate problems due to demand surges in conventional distribution systems.Many works have used Long Short-Term Memory (LSTMs) to predict generation. However, to minimize computational costs related to retraining and inference, LSTMs might not be optimal. Therefore, in this work, we compare the performance of MLP (Multilayer Perceptron), Recurrent Neural Networks (RNNs), and LSTMs for the task mentioned above. We used the solar radiance measured throughout 2020 in the city of Maceió (Brazil), taking into account periods of 2 hours for training to predict the next 5-minutes. Hyperparameters were fine-tuned using an optimization approach based on Bayesian inference to promote a fair comparison. Results showed that the MLP has satisfactory performance, requiring much less time to train and forecast. Such results can contribute, for example, to improving response time in embedded systems.

scholarly journals Variational Cross-domain Natural Language Generation for Spoken Dialogue Systems

2018 ◽  
Author(s):  
Bo-Hsiang Tseng ◽  
Florian Kreyssig ◽  
Paweł Budzianowski ◽  
Iñigo Casanueva ◽  
Yen-Chen Wu ◽  
...  
Keyword(s):  
Natural Language ◽  
Natural Language Generation ◽  
Dialogue Systems ◽  
Spoken Dialogue Systems ◽  
Spoken Dialogue ◽  
Language Generation ◽  
Cross Domain

scholarly journals Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems

2015 ◽  
Author(s):  
Tsung-Hsien Wen ◽  
Milica Gasic ◽  
Nikola Mrkšić ◽  
Pei-Hao Su ◽  
David Vandyke ◽  
...  
Keyword(s):  
Natural Language ◽  
Natural Language Generation ◽  
Dialogue Systems ◽  
Spoken Dialogue Systems ◽  
Spoken Dialogue ◽  
Language Generation
Sign in / Sign up

Export Citation Format

Share Document