scholarly journals Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Cédric Foucault ◽  
Florent Meyniel
Keyword(s):  
Latent Variables ◽  
Network Architecture ◽  
Real Life ◽  
Optimal Solution ◽  
Neural Network Architecture ◽  
Internal Representations ◽  
Sequence Prediction ◽  
Effective Learning ◽  
Multiple Levels ◽  
The Brain

From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. This architecture relies on three mechanisms: gating, lateral connections, and recurrent weight training. Like the optimal solution and the human brain, such networks develop internal representations of their changing environment (including estimates of the environment's latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.

scholarly journals Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments

2021 ◽  
Author(s):  
Cedric Foucault ◽  
Florent Meyniel
Keyword(s):  
Latent Variables ◽  
Network Architecture ◽  
Real Life ◽  
Latent Structure ◽  
Neural Network Architecture ◽  
Internal Representations ◽  
Sequence Prediction ◽  
Effective Learning ◽  
Multiple Levels ◽  
The Brain

From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. To this end, a set of three mechanisms suffices: gating, lateral connections, and recurrent weight tuning. Like the human brain, such networks develop internal representations of their changing environment (including estimates of the environment's latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.

Connectivity and Condensation in Dreaming

1992 ◽  
Vol 40 (4) ◽  
pp. 1139-1159 ◽  
Author(s):  
Stanley R. Palombo
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Selection Process ◽  
Psychoanalytic Theory ◽  
Distributed Representation ◽  
Mental Functioning ◽  
Neural Network Architecture ◽  
New Developments ◽  
The Neural Network ◽  
The Brain

Converging developments in the cognitive- and neurosciences have brought Freud's hope of a bridge between psychoanalysis and psychophysiology nearer to hand. This paper concerns the relation between dream construction and memory in terms of these new developments. The neural network architecture of memory structures in the brain is described and illustrated with simple examples. We see how a network is connected and how connection weights vary with experience. The distributed representation stored by the network and its crucial properties for mental functioning are discussed. These concepts are used to explain how particular memories of past events are selected for inclusion in the dream. The properties of the neural network suggest that images of distinct past events are conflated at times during the selection process. The appearance of these conflated images may complicate the matching of day residues with representations of past events in the dream itself. Some likely implications for psychoanalytic theory are explored.

scholarly journals Generative Adversarial Phonology: Modeling unsupervised allophonic learning with neural networks

2019 ◽  
Author(s):  
Gasper Begus
Keyword(s):  
Neural Networks ◽  
Latent Variables ◽  
Network Architecture ◽  
Speech Disorders ◽  
Levels Of Abstraction ◽  
Generative Adversarial Network ◽  
Internal Representations ◽  
Adversarial Network ◽  
Training Interval ◽  
Speech Data

Training deep neural networks on well-understood dependencies in speech data can provide new insights into how they learn internal representations. This paper argues that acquisition of speech can be modeled as a dependency between random space and generated speech data in the Generative Adversarial Network architecture and proposes a methodology to uncover the network's internal representations that correspond to phonetic and phonological properties. The Generative Adversarial architecture is uniquely appropriate for modeling phonetic and phonological learning because the network is trained on unannotated raw acoustic data and learning is unsupervised without any language-specific assumptions or pre-assumed levels of abstraction. A Generative Adversarial Network was trained on an allophonic distribution in English, in which voiceless stops surface as aspirated word-initially before stressed vowels, except if preceded by a sibilant [s]. The network successfully learns the allophonic alternation: the network's generated speech signal contains the conditional distribution of aspiration duration. The paper proposes a technique for establishing the network's internal representations that identifies latent variables that correspond to, for example, presence of [s] and its spectral properties. By manipulating these variables, we actively control the presence of [s] and its frication amplitude in the generated outputs. This suggests that the network learns to use latent variables as an approximation of phonetic and phonological representations. Crucially, we observe that the dependencies learned in training extend beyond the training interval, which allows for additional exploration of learning representations. The paper also discusses how the network's architecture and innovative outputs resemble and differ from linguistic behavior in language acquisition, speech disorders, and speech errors, and how well-understood dependencies in speech data can help us interpret how neural networks learn their representations.

scholarly journals Learning Hierarchical Representations with Spike-and-Slab Inception Network

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6382
Author(s):  
Weizheng Qiao ◽  
Xiaojun Bi
Keyword(s):  
Latent Variables ◽  
Network Architecture ◽  
Recognition Task ◽  
Model Parameters ◽  
Neural Network Architecture ◽  
Deep Convolutional Neural Networks ◽  
Parameter Redundancy ◽  
Feature Mining ◽  
Feature Expression ◽  
Hierarchical Representations

Recently, deep convolutional neural networks (CNN) with inception modules have attracted much attention due to their excellent performances on diverse domains. Nevertheless, the basic CNN can only capture a univariate feature, which is essentially linear. It leads to a weak ability in feature expression, further resulting in insufficient feature mining. In view of this issue, researchers incessantly deepened the network, bringing parameter redundancy and model over-fitting. Hence, whether we can employ this efficient deep neural network architecture to improve CNN and enhance the capacity of image recognition task still remains unknown. In this paper, we introduce spike-and-slab units to the modified inception module, enabling our model to capture dual latent variables and the average and covariance information. This operation further enhances the robustness of our model to variations of image intensity without increasing the model parameters. The results of several tasks demonstrated that dual variable operations can be well-integrated into inception modules, and excellent results have been achieved.

scholarly journals Artificial Recurrent Neural Network Architecture in Customer Consumption Prediction for Business Development

2020 ◽  
Vol 2 (2) ◽  
pp. 111-120
Author(s):  
Dr. Karrupusamy P.
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Association Rule ◽  
Network Architecture ◽  
Short Term Memory ◽  
Real Life ◽  
Consumption Pattern ◽  
Neural Network Architecture ◽  
Competitive Edge ◽  
Frame Work

The customer consumption pattern prediction has become one of a significant role in developing the business and taking it to a competitive edge. For forecasting the behaviors of the consumers the paper engages an artificial recurrent neural network architecture the long short-term memory an improvement of recurrent neural network. The mechanism laid out to predict the pattern of the consumption, uses the information’s about the consumption of products based on the age and the gender. The information essential are extracted and described with the prefix-span procedure based association rule. Utilizing the information about the day to day products purchase pattern as input a frame work to predict the customer daily essentials was designed, the designed frame was capable enough to learn the dissimilarities across the predicted and the original miscalculation rates. The frame work devised was tested using real life applications and the results observed demonstrated that the proposed LSTM based prediction with the prefix span association rule to acquire the day today consumption details is compatible for forecasting the customer consumption over time accurately.

scholarly journals Application of Deep Learning for Emotion Recognition

YMER Digital ◽  
2021 ◽  
Vol 20 (12) ◽  
pp. 834-840
Author(s):  
Varsha R Toshniwal ◽  
◽  
Pooja S Puri ◽  
Keyword(s):  
Network Architecture ◽  
Emotional State ◽  
Human Perception ◽  
Brain Regions ◽  
Eeg Signals ◽  
Neural Network Architecture ◽  
Computer Interfaces ◽  
Alpha Beta ◽  
Electroencephalogram Eeg ◽  
The Brain

The electroencephalogram (EEG) gained a lot of importance in recent years because of its property to depict the nature and actions of human perception. EEG signals are good at capturing the emotional state of a person by measuring the neuronal activities in different regions of the brain. Lots of EEG-based brain-computer interfaces with a different number of channels ( 62 channels, 32 channels, etc.) are being used to capture neuronal activities which can be segmented into different frequency ranges (delta, theta, alpha. beta and gamma). This paper puts forward a neural network architecture for the recognition of emotion from EEG signals and a study providing the set of brain regions and the frequency type associated with the corresponding brain region which contributes most for the detection of emotion though EEG signals. For experimentation, SEED-IV dataset has been used

scholarly journals Persistent frustration-induced reconfigurations of brain networks predict individual differences in irritability

2021 ◽  
Author(s):  
Julia Olivia Linke ◽  
Simone P Haller ◽  
Ellie Xu ◽  
Lynn Nguyen ◽  
Amanda Chue ◽  
...  
Keyword(s):  
Resting State ◽  
Network Architecture ◽  
Goal Attainment ◽  
Recovery Period ◽  
Brain Networks ◽  
Brain Network ◽  
Neural Network Architecture ◽  
Unseen Data ◽  
Network Process ◽  
The Brain

Background. Frustration, the response to blocked goal attainment, is a universal affective experience, but how the brain embodies frustration is not known. Understanding brain network dynamics during frustration may provide insight into pediatric irritability, one of the most frequent reasons for psychiatric consultation in youth and a risk factor for affective disorders and suicidality. Methods. Using fMRI, we investigated changes in neural network architecture from a baseline resting-state, through a task that included frustrative nonreward (FNR) and anticipation of new feedback following FNR (FNR+1), to a post-task resting-state in a transdiagnostic sample of 66 youth (33 female, mean age 14 years). Using a train/test/held-out procedure, we aimed to predict past-week irritability from the global efficiency (i.e., Eglob, capacity for parallel information processing) of brain networks before, during, and after frustration. Results. Compared to pre-task resting state, FNR+1 and the post-state resting state were uniquely associated with a more segregated brain network organization. Nodes that were originally affiliated with the default-mode-temporal-limbic and fronto-parietal networks contributed most to this reconfiguration. Solely Eglob of brain networks that emerged after the frustrating task predicted self- and observer-rated irritability in previously unseen data. Self-reported irritability was predicted by Eglob of a fronto-temporal-limbic module, while observer-rated irritability was predicted by Eglob of motor-parietal and ventral-prefrontal-subcortical modules. Discussion. We characterize frustration as an evolving brain network process and demonstrate the importance of the post-frustration recovery period for the pathophysiology of irritability; an insight that, if replicated, suggests specific intervention targets for irritability.

scholarly journals Connectome development and a novel extension to the neurodevelopmental model of schizophrenia

2018 ◽  
Vol 20 (2) ◽  
pp. 101-110 ◽  
Keyword(s):  
Complex Network ◽  
Network Architecture ◽  
Adaptive System ◽  
Brain Network ◽  
Biological Processes ◽  
System A ◽  
Neurodevelopmental Model ◽  
Multiple Levels ◽  
The Brain ◽  
Abnormal Brain

The brain is the ultimate adaptive system, a complex network organized across multiple levels of spatial and temporal resolution that is sculpted over several decades via its interactions with the environment. This review sets out to examine how fundamental biological processes in early and late neurodevelopment, in interaction with environmental inputs, guide the formation of the brain’s network and its ongoing reorganization throughout the course of development. Moreover, we explore how disruptions in these processes could lead to abnormal brain network architecture and organization and thereby give rise to schizophrenia. Arguing that the neurodevelopmental trajectory leading up to the manifestation of psychosis may best be understood from the sequential trajectory of connectome formation and maturation, we propose a novel extension to the neurodevelopmental model of the illness that posits that schizophrenia is a disorder of connectome development

scholarly journals Multi-national and Multi-language License Plate Detection using Convolutional Neural Networks

2020 ◽  
Vol 10 (4) ◽  
pp. 5979-5985
Author(s):  
M. Salemdeeb ◽  
S. Erturk
Keyword(s):  
Network Architecture ◽  
Real Life ◽  
Low Complexity ◽  
License Plate ◽  
Neural Network Architecture ◽  
New Approach ◽  
Computer Vision Applications ◽  
License Plate Detection ◽  
Language Detection ◽  
Detection And Recognition

Many real-life machine and computer vision applications are focusing on object detection and recognition. In recent years, deep learning-based approaches gained increasing interest due to their high accuracy levels. License Plate (LP) detection and classification have been studied extensively over the last decades. However, more accurate and language-independent approaches are still required. This paper presents a new approach to detect LPs and recognize their country, language, and layout. Furthermore, a new LP dataset for both multi-national and multi-language detection, with either one-line or two-line layouts is presented. The YOLOv2 detector with ResNet feature extraction core was utilized for LP detection, and a new low complexity convolutional neural network architecture was proposed to classify LPs. Results show that the proposed approach achieves an average detection precision of 99.57%, whereas the country, language, and layout classification accuracy is 99.33%.

scholarly journals Preprocessing Pipelines including Block-Matching Convolutional Neural Network for Image Denoising to Robustify Deep Reidentification against Evasion Attacks

Entropy ◽  
2021 ◽  
Vol 23 (10) ◽  
pp. 1304
Author(s):  
Marek Pawlicki ◽  
Ryszard S. Choraś
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Image Denoising ◽  
Network Architecture ◽  
Real Life ◽  
Block Matching ◽  
Use Case ◽  
Neural Network Architecture

Artificial neural networks have become the go-to solution for computer vision tasks, including problems of the security domain. One such example comes in the form of reidentification, where deep learning can be part of the surveillance pipeline. The use case necessitates considering an adversarial setting—and neural networks have been shown to be vulnerable to a range of attacks. In this paper, the preprocessing defences against adversarial attacks are evaluated, including block-matching convolutional neural network for image denoising used as an adversarial defence. The benefit of using preprocessing defences comes from the fact that it does not require the effort of retraining the classifier, which, in computer vision problems, is a computationally heavy task. The defences are tested in a real-life-like scenario of using a pre-trained, widely available neural network architecture adapted to a specific task with the use of transfer learning. Multiple preprocessing pipelines are tested and the results are promising.

Sign in / Sign up

Export Citation Format

Share Document