scholarly journals Echo state network models for nonlinear Granger causality

2021 ◽  
Vol 379 (2212) ◽  
Author(s):  
Andrea Duggento ◽  
Maria Guerrisi ◽  
Nicola Toschi
Keyword(s):  
Granger Causality ◽  
Biological Networks ◽  
Short Term Memory ◽  
Real Life ◽  
Network Models ◽  
Nonlinear Behaviour ◽  
Echo State Networks ◽  
Human Connectome Project ◽  
Challenges And Opportunities ◽  
Long Short Term Memory

While Granger causality (GC) has been often employed in network neuroscience, most GC applications are based on linear multivariate autoregressive (MVAR) models. However, real-life systems like biological networks exhibit notable nonlinear behaviour, hence undermining the validity of MVAR-based GC (MVAR-GC). Most nonlinear GC estimators only cater for additive nonlinearities or, alternatively, are based on recurrent neural networks or long short-term memory networks, which present considerable training difficulties and tailoring needs. We reformulate the GC framework in terms of echo-state networks-based models for arbitrarily complex networks, and characterize its ability to capture nonlinear causal relations in a network of noisy Duffing oscillators, showing a net advantage of echo state GC (ES-GC) in detecting nonlinear, causal links. We then explore the structure of ES-GC networks in the human brain employing functional MRI data from 1003 healthy subjects drawn from the human connectome project, demonstrating the existence of previously unknown directed within-brain interactions. In addition, we examine joint brain-heart signals in 15 subjects where we explore directed interaction between brain networks and central vagal cardiac control in order to investigate the so-called central autonomic network in a causal manner. This article is part of the theme issue ‘Advanced computation in cardiovascular physiology: new challenges and opportunities’.

scholarly journals Echo State Network models for nonlinear Granger causality

2019 ◽  
Author(s):  
Andrea Duggento ◽  
Maria Guerrisi ◽  
Nicola Toschi
Keyword(s):  
Granger Causality ◽  
Biological Networks ◽  
Short Term Memory ◽  
Real Life ◽  
Network Models ◽  
Echo State Networks ◽  
Linear Behavior ◽  
Novel Approach ◽  
Human Connectome Project ◽  
Random Initialization

AbstractWhile Granger Causality (GC) has been often employed in network neuroscience, most GC applications are based on linear multivariate autoregressive (MVAR) models. However, real-life systems like biological networks exhibit notable non-linear behavior, hence undermining the validity of MVAR-based GC (MVAR-GC). Current nonlinear GC estimators only cater for additive nonlinearities or, alternatively, are based on recurrent neural networks (RNN) or Long short-term memory (LSTM) networks, which present considerable training difficulties and tailoring needs. We define a novel approach to estimating nonlinear, directed within-network interactions through a RNN class termed echo-state networks (ESN), where training is replaced by random initialization of an internal basis based on orthonormal matrices. We reformulate the GC framework in terms of ESN-based models, our ESN-based Granger Causality (ES-GC) estimator in a network of noisy Duffing oscillators, showing a net advantage of ES-GC in detecting nonlinear, causal links. We then explore the structure of ES-GC networks in the human brain employing functional MRI data from 1003 healthy subjects drawn from the human connectome project, demonstrating the existence of previously unknown directed within-brain interactions. ES-GC performs better than commonly used and recently developed GC approaches, making it a valuable tool for the analysis of e.g. multivariate biological networks.

scholarly journals Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems

2019 ◽  
Vol 10 (1) ◽  
pp. 1-19
Author(s):  
Matthieu Riou ◽  
Bassam Jabaian ◽  
Stéphane Huet ◽  
Fabrice Lefèvre
Keyword(s):  
Natural Language ◽  
Short Term Memory ◽  
Network Models ◽  
Dialogue Systems ◽  
Neural Network Models ◽  
Spoken Dialogue Systems ◽  
Spoken Dialogue ◽  
Automatic Transcription ◽  
Long Short Term Memory ◽  
Expected Payoffs

Following some recent propositions to handle natural language generation in spoken dialogue systems with long short-term memory recurrent neural network models~\citep{Wen2016a} we first investigate a variant thereof with the objective of a better integration of the attention subnetwork. Then our next objective is to propose and evaluate a framework to adapt the NLG module online through direct interactions with the users. When doing so the basic way is to ask the user to utter an alternative sentence to express a particular dialogue act. But then the system has to decide between using an automatic transcription or to ask for a manual transcription. To do so a reinforcement learning approach based on an adversarial bandit scheme is retained. We show that by defining appropriately the rewards as a linear combination of expected payoffs and costs of acquiring the new data provided by the user, a system design can balance between improving the system's performance towards a better match with the user's preferences and the burden associated with it. Then the actual benefits of this system is assessed with a human evaluation, showing that the addition of more diverse utterances allows to produce sentences more satisfying for the user.

scholarly journals SEO Ministration : Content Related Tag Suggestion Using One vs Rest

2021 ◽  
pp. 245-253
Author(s):  
Dyapa Sravan Reddy ◽  
Lakshmi Prasanna Reddy ◽  
Kandibanda Sai Santhosh ◽  
Virrat Devaser
Keyword(s):  
Machine Learning ◽  
Short Term Memory ◽  
Learning Algorithms ◽  
Real Life ◽  
Machine Learning Algorithms ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

SEO Analyst pays a lot of time finding relevant tags for their articles and in some cases, they are unaware of the content topics. The current proposed ML model will recommend content-related tags so that the Content writers/SEO analyst will be having an overview regarding the content and minimizes their time spent on unknown articles. Machine Learning algorithms have a plethora of applications and the extent of their real-life implementations cannot be estimated. Using algorithms like One vs Rest (OVR), Long Short-Term Memory (LSTM), this study has analyzed how Machine Learning can be useful for tag suggestions for a topic. The training of the model with One vs Rest turned out to deliver more accurate results than others. This Study certainly answers how One vs Rest is used for tag suggestions that are needed to promote a website and further studies are required to suggest keywords required.

scholarly journals DLI: A Deep Learning-Based Granger Causality Inference

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-6 ◽  
Author(s):  
Wei Peng
Keyword(s):  
Deep Learning ◽  
Granger Causality ◽  
Prediction Accuracy ◽  
Short Term Memory ◽  
Target Variable ◽  
Short Term ◽  
Term Memory ◽  
P Index ◽  
Causality Inference ◽  
Long Short Term Memory

Integrating autoencoder (AE), long short-term memory (LSTM), and convolutional neural network (CNN), we propose an interpretable deep learning architecture for Granger causality inference, named deep learning-based Granger causality inference (DLI). Two contributions of the proposed DLI are to reveal the Granger causality between the bitcoin price and S&P index and to forecast the bitcoin price and S&P index with a higher accuracy. Experimental results demonstrate that there is a bidirectional but asymmetric Granger causality between the bitcoin price and S&P index. And the DLI performs a superior prediction accuracy by integrating variables that have causalities with the target variable into the prediction process.

scholarly journals Robust Brain State Decoding using Bidirectional Long Short Term Memory Networks in functional MRI

2021 ◽  
Author(s):  
Anant Mittal ◽  
Priya Aggarwal ◽  
Luiz Pessoa ◽  
Anubha Gupta
Keyword(s):  
Short Term Memory ◽  
Fmri Data ◽  
Brain State ◽  
Short Term ◽  
Term Memory ◽  
Feature Representations ◽  
Human Connectome Project ◽  
Brain States ◽  
Long Short Term Memory ◽  
The Brain

Decoding brain states of the underlying cognitive processes via learning discriminative feature representations has recently gained a lot of interest in brain imaging studies. Particularly, there has been an impetus to encode the dynamics of brain functioning by analyzing temporal information avail- able in the fMRI data. Long short term memory (LSTM), a class of machine learning model possessing a "memory" component, is increasingly being observed to perform well in various applications with dynamic temporal behavior, including brain state decoding. Because of the dynamics and inherent latency in fMRI BOLD responses, future temporal context is crucial. However, it is neither encoded nor captured by the conventional LSTM model. This paper performs robust brain state decoding via information encapsulation from both the past and future instances of fMRI data via bidirectional LSTM. This allows for explicitly modeling the dynamics of BOLD response without any delay adjustment. The two hidden activations of forward and reverse directions in bi-LSTM are collated to build the "memory" of the model and are used to robustly predict the brain states at every time instance. Working memory data from the Human Connectome Project (HCP) is utilized for validation and was observed to perform 18 percent better than it's unidirectional counterpart in terms of accuracy in predicting the brain states.

Generalized Boolean Networks

2010 ◽  
pp. 429-449 ◽  
Author(s):  
Christian Darabos ◽  
Mario Giacobini ◽  
Marco Tomassini
Keyword(s):  
Biological Networks ◽  
Regulatory Networks ◽  
Dynamical Behavior ◽  
Real Life ◽  
Network Models ◽  
Cell Types ◽  
Boolean Networks ◽  
Genetic Regulatory Networks ◽  
Point Of View ◽  
Scale Free

Random Boolean Networks (RBN) have been introduced by Kauffman more than thirty years ago as a highly simplified model of genetic regulatory networks. This extremely simple and abstract model has been studied in detail and has been shown capable of extremely interesting dynamical behavior. First of all, as some parameters are varied such as the network’s connectivity, or the probability of expressing a gene, the RBN can go through a phase transition, going from an ordered regime to a chaotic one. Kauffman’s suggestion is that cell types correspond to attractors in the RBN phase space, and only those attractors that are short and stable under perturbations will be of biological interest. Thus, according to Kauffman, RBN lying at the edge between the ordered phase and the chaotic phase can be seen as abstract models of genetic regulatory networks. The original view of Kauffman, namely that these models may be useful for understanding real-life cell regulatory networks, is still valid, provided that the model is updated to take into account present knowledge about the topology of real gene regulatory networks, and the timing of events, without loosing its attractive simplicity. According to present data, many biological networks, including genetic regulatory networks, seem, in fact, to be of the scale-free type. From the point of view of the timing of events, standard RBN update their state synchronously. This assumption is open to discussion when dealing with biologically plausible networks. In particular, for genetic regulatory networks, this is certainly not the case: genes seem to be expressed in different parts of the network at different times, according to a strict sequence, which depends on the particular network under study. The expression of a gene depends on several transcription factors, the synthesis of which appear to be neither fully synchronous nor instantaneous. Therefore, we have recently proposed a new, more biologically plausible model. It assumes a scale-free topology of the networks and we define a suitable semi-synchronous dynamics that better captures the presence of an activation sequence of genes linked to the topological properties of the network. By simulating statistical ensembles of networks, we discuss the attractors of the dynamics, showing that they are compatible with theoretical biological network models. Moreover, the model demonstrates interesting scaling abilities as the size of the networks is increased.

Deep Neural Models and Retrofitting for Arabic Text Categorization

2020 ◽  
Vol 16 (2) ◽  
pp. 74-86 ◽  
Author(s):  
Fatima-Zahra El-Alami ◽  
Said Ouatik El Alaoui ◽  
Noureddine En-Nahnahi
Keyword(s):  
Text Categorization ◽  
Short Term Memory ◽  
Network Models ◽  
Arabic Text ◽  
Short Term ◽  
Lexical Resources ◽  
Neural Network Models ◽  
Online Data ◽  
Long Short Term Memory ◽  
Vector Representations

Arabic text categorization is an important task in text mining particularly with the fast-increasing quantity of the Arabic online data. Deep neural network models have shown promising performance and indicated great data modeling capacities in managing large and substantial datasets. This article investigates convolution neural networks (CNNs), long short-term memory (LSTM) and their combination for Arabic text categorization. This work additionally handles the morphological variety of Arabic words by exploring the word embeddings model using position weights and subword information. To guarantee the nearest vector representations for connected words, this article adopts a strategy for refining Arabic vector space representations using semantic information embedded in lexical resources. Several experiments utilizing different architectures have been conducted on the OSAC dataset. The obtained results show the effectiveness of CNN-LSTM without and with retrofitting for Arabic text categorization in comparison with major competing methods.

scholarly journals Exploring the Sensitivity of Recurrent Neural Network Models for Forecasting Land Cover Change

Land ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 282
Author(s):  
Alysha van Duynhoven ◽  
Suzana Dragićević
Keyword(s):  
Land Cover ◽  
Short Term Memory ◽  
Network Models ◽  
Optimization Techniques ◽  
Neural Network Models ◽  
Method Performance ◽  
Timeseries Analysis ◽  
Method Optimization ◽  
Long Short Term Memory ◽  
Real World Datasets

Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) architectures, have obtained successful outcomes in timeseries analysis tasks. While RNNs demonstrated favourable performance for Land Cover (LC) change analyses, few studies have explored or quantified the geospatial data characteristics required to utilize this method. Likewise, many studies utilize overall measures of accuracy rather than metrics accounting for the slow or sparse changes of LC that are typically observed. Therefore, the main objective of this study is to evaluate the performance of LSTM models for forecasting LC changes by conducting a sensitivity analysis involving hypothetical and real-world datasets. The intent of this assessment is to explore the implications of varying temporal resolutions and LC classes. Additionally, changing these input data characteristics impacts the number of timesteps and LC change rates provided to the respective models. Kappa variants are selected to explore the capacity of LSTM models for forecasting transitions or persistence of LC. Results demonstrate the adverse effects of coarser temporal resolutions and high LC class cardinality on method performance, despite method optimization techniques applied. This study suggests various characteristics of geospatial datasets that should be present before considering LSTM methods for LC change forecasting.

scholarly journals Prediction of Amusement Intensity Based on Brain Activity

2021 ◽  
Author(s):  
Gabrielle Toupin ◽  
Mohamed S. Benlamine ◽  
Claude Frasson
Keyword(s):  
Positive Emotions ◽  
Cross Validation ◽  
Short Term Memory ◽  
Brain Activity ◽  
Real Life ◽  
Training Phase ◽  
Real Life Setting ◽  
Long Short Term Memory ◽  
Very High ◽  
Different Levels

Amusement can help modulate psychological disorders and cognitive functions. Unfortunately, algorithms classifying emotions still combine multiple positive emotions into a unique emotion, namely joy, making it hard to use amusement in a real-life setting. Here we train a Long-Short-Term-Memory (LSTM) on electroencephalography (EEG) to predict amusement on a categorical scale. Participants (n=10) watched and rated 120 videos with various funniness levels while their brain activity was recorded with an Emotiv Headset. Participant’s ratings were divided into four bins of amusement (low, medium, high & very high) based on the participant’s ranking’s percentile. Nested cross-validation was used to validate the models. We first left out one video from each participant for the final model’s validation and a leave-one-group-out technique was used to test the model on an unseen participant during the training phase. The nested cross-validation was tested on sixteen different videos. We created an LSTM model with five hidden layers, vatch size of 256 and an input layer of 14 x 128 (number of electrodes x 1 sec of recording) and four nodes representing the different levels of amusement. The best model obtained during the training phase was tested on the unseen video. While the level of accuracy between the validation videos varies slightly (mean=57.3%, std=13.7%), our best model obtained an accuracy of 82,4%. This high accuracy supports the use of brain activity to predict amusement. Moreover, the validation process we design conveys that models using this technique are transferable across participants and videos.

Sign in / Sign up

Export Citation Format

Share Document