scholarly journals Neural network models of the tactile system develop first-order units with spatially complex receptive fields

2017 ◽  
Author(s):  
Charlie W. Zhao ◽  
Mark J. Daley ◽  
J. Andrew Pruszynski
Keyword(s):  
Neural Network ◽  
Network Performance ◽  
Receptive Fields ◽  
Network Models ◽  
Network Architectures ◽  
Learning Tools ◽  
Neural Network Models ◽  
First Order ◽  
Wide Range ◽  
Tactile System

AbstractFirst-order tactile neurons have spatially complex receptive fields. Here we use machine learning tools to show that such complexity arises for a wide range of training sets and network architectures, and benefits network performance, especially on more difficult tasks and in the presence of noise. Our work suggests that spatially complex receptive fields are normatively good given the biological constraints of the tactile periphery.

scholarly journals NeuRiPP: Neural network identification of RiPP precursor peptides

2019 ◽  
Author(s):  
Emmanuel L.C. de los Santos
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Network Models ◽  
Gene Clusters ◽  
Learning Tools ◽  
Neural Network Models ◽  
Data Set ◽  
The Rich ◽  
Tailoring Enzymes ◽  
Rich Data

ABSTRACTSignificant progress has been made in the past few years on the computational identification biosynthetic gene clusters (BGCs) that encode ribosomally synthesized and post-translationally modified peptides (RiPPs). This is done by identifying both RiPP tailoring enzymes (RTEs) and RiPP precursor peptides (PPs). However, identification of PPs, particularly for novel RiPP classes remains challenging. To address this, machine learning has been used to accurately identify PP sequences. However, current machine learning tools have limitations, since they are specific to the RiPP-class they are trained for, and are context-dependent, requiring information about the surrounding genetic environment of the putative PP sequences. NeuRiPP overcomes these limitations. It does this by leveraging the rich data set of high-confidence putative PP sequences from existing programs, along with experimentally verified PPs from RiPP databases. NeuRiPP uses neural network models that are suitable for peptide classification with weights trained on PP datasets. It is able to identify known PP sequences, and sequences that are likely PPs. When tested on existing RiPP BGC datasets, NeuRiPP is able to identify PP sequences in significantly more putative RiPP clusters than current tools, while maintaining the same HMM hit accuracy. Finally, NeuRiPP was able to successfully identify PP sequences from novel RiPP classes that are recently characterized experimentally, highlighting its utility in complementing existing bioinformatics tools.

Neural Network Models of Perceptual Learning of Angle Discrimination

1996 ◽  
Vol 8 (2) ◽  
pp. 270-299 ◽  
Author(s):  
G. Mato ◽  
H. Sompolinsky
Keyword(s):  
Neural Network ◽  
Perceptual Learning ◽  
Network Models ◽  
Angular Width ◽  
Network Architectures ◽  
Neural Network Models ◽  
Tuning Curves ◽  
Partial Transfer ◽  
Gating Network ◽  
Angle Discrimination

We study neural network models of discriminating between stimuli with two similar angles, using the two-alternative forced choice (2AFC) paradigm. Two network architectures are investigated: a two-layer perceptron network and a gating network. In the two-layer network all hidden units contribute to the decision at all angles, while in the other architecture the gating units select, for each stimulus, the appropriate hidden units that will dominate the decision. We find that both architectures can perform the task reasonably well for all angles. Perceptual learning has been modeled by training the networks to perform the task, using unsupervised Hebb learning algorithms with pairs of stimuli at fixed angles θ and δθ. Perceptual transfer is studied by measuring the performance of the network on stimuli with θ′ ≠ θ. The two-layer perceptron shows a partial transfer for angles that are within a distance a from θ, where a is the angular width of the input tuning curves. The change in performance due to learning is positive for angles close to θ, but for |θ − θ′| ≈ a it is negative, i.e., its performance after training is worse than before. In contrast, negative transfer can be avoided in the gating network by limiting the effects of learning to hidden units that are optimized for angles that are close to the trained angle.

scholarly journals Learning the synaptic and intrinsic membrane dynamics underlying working memory in spiking neural network models

2020 ◽  
Author(s):  
Yinghao Li ◽  
Robert Kim ◽  
Terrence J. Sejnowski
Keyword(s):  
Neural Network ◽  
Working Memory ◽  
Complex Dynamics ◽  
Network Models ◽  
Membrane Dynamics ◽  
Membrane Properties ◽  
Cognitive Tasks ◽  
Neural Network Models ◽  
Wide Range ◽  
Synaptic Dynamics

SummaryRecurrent neural network (RNN) model trained to perform cognitive tasks is a useful computational tool for understanding how cortical circuits execute complex computations. However, these models are often composed of units that interact with one another using continuous signals and overlook parameters intrinsic to spiking neurons. Here, we developed a method to directly train not only synaptic-related variables but also membrane-related parameters of a spiking RNN model. Training our model on a wide range of cognitive tasks resulted in diverse yet task-specific synaptic and membrane parameters. We also show that fast membrane time constants and slow synaptic decay dynamics naturally emerge from our model when it is trained on tasks associated with working memory (WM). Further dissecting the optimized parameters revealed that fast membrane properties and slow synaptic dynamics are important for encoding stimuli and WM maintenance, respectively. This approach offers a unique window into how connectivity patterns and intrinsic neuronal properties contribute to complex dynamics in neural populations.

Neural Network Models of Religious Belief

1995 ◽  
Vol 38 (4) ◽  
pp. 483-495 ◽  
Author(s):  
William Sims Bainbridge
Keyword(s):  
Neural Network ◽  
New Technology ◽  
Network Models ◽  
Eternal Life ◽  
Neural Net ◽  
Neural Network Models ◽  
Social Scientists ◽  
Theory Of Religion ◽  
Wide Range ◽  
Neural Network Technology

This paper applies neural network technology, a standard approach in computer science that has been unaccountably ignored by sociologists, to the problem of developing rigorous sociological theories. A simulation program employing a “varimax” model of human learning and decision-making models central elements of the Stark-Bainbridge theory of religion. Individuals in a micro-society of 24 simulated people learn which categories of potential exchange partners to seek for each of four material rewards which in fact can be provided by other actors in the society. However, when they seek eternal life, they are unable to find suitable human exchange partners who can provide it to them, so they postulate the existence of supernatural exchange partners as substitutes. The explanation of how the particular neural net works, including reference to modulo arithmetic, introduces some aspects of this new technology to sociology, and this paper invites readers to explore the wide range of other neural net techniques that may be of value for social scientists

scholarly journals An Empirical Study on Deep Neural Network Models for Chinese Dialogue Generation

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1756
Author(s):  
Zhe Li ◽  
Mieradilijiang Maimaiti ◽  
Jiabao Sheng ◽  
Zunwang Ke ◽  
Wushour Silamu ◽  
...  
Keyword(s):  
Neural Network ◽  
Empirical Study ◽  
Deep Neural Network ◽  
Question Answering ◽  
State Of The Art ◽  
Network Models ◽  
Superior Performance ◽  
Research Progress ◽  
Neural Network Models ◽  
Wide Range

The task of dialogue generation has attracted increasing attention due to its diverse downstream applications, such as question-answering systems and chatbots. Recently, the deep neural network (DNN)-based dialogue generation models have achieved superior performance against conventional models utilizing statistical machine learning methods. However, despite that an enormous number of state-of-the-art DNN-based models have been proposed, there lacks detailed empirical comparative analysis for them on the open Chinese corpus. As a result, relevant researchers and engineers might find it hard to get an intuitive understanding of the current research progress. To address this challenge, we conducted an empirical study for state-of-the-art DNN-based dialogue generation models in various Chinese corpora. Specifically, extensive experiments were performed on several well-known single-turn and multi-turn dialogue corpora, including KdConv, Weibo, and Douban, to evaluate a wide range of dialogue generation models that are based on the symmetrical architecture of Seq2Seq, RNNSearch, transformer, generative adversarial nets, and reinforcement learning respectively. Moreover, we paid special attention to the prevalent pre-trained model for the quality of dialogue generation. Their performances were evaluated by four widely-used metrics in this area: BLEU, pseudo, distinct, and rouge. Finally, we report a case study to show example responses generated by these models separately.

scholarly journals Error Detection for Arabic Text Using Neural Sequence Labeling

2020 ◽  
Vol 10 (15) ◽  
pp. 5279
Author(s):  
Nora Madi ◽  
Hend Al-Khalifa
Keyword(s):  
Neural Network ◽  
Error Detection ◽  
English Language ◽  
Network Models ◽  
Network Architectures ◽  
Arabic Text ◽  
Neural Network Models ◽  
Sequence Labeling ◽  
Microsoft Word ◽  
Neural Network Architectures

The English language has, thus far, received the most attention in research concerning automatic grammar error correction and detection. However, these tasks have been less investigated for other languages. In this paper, we present the first experiments using neural network models for the task of error detection for Modern Standard Arabic (MSA) text. We investigate several neural network architectures and report the evaluation results acquired by applying cross-validation on the data. All experiments involve a corpus we created and augmented. The corpus has 494 sentences and 620 sentences after augmentation. Our models achieved a maximum precision of 78.09%, recall of 83.95%, and F0.5 score of 79.62% in the error detection task using SimpleRNN. Using an LSTM, we achieved a maximum precision of 79.21%, recall of 93.8%, and F0.5 score of 79.16%. Finally, the best results were achieved using a BiLSTM with a maximum precision of 80.74%, recall of 85.73%, and F0.5 score of 81.55%. We compared the results of the three models to a baseline, which is a commercially available Arabic grammar checker (Microsoft Word 2007). LSTM, BiLSTM, and SimpleRNN all outperformed the baseline in precision and F0.5. Our work shows preliminary results, demonstrating that neural network architectures for error detection through sequence labeling can successfully be applied to Arabic text.

scholarly journals Generation of scale-invariant sequential activity in linear recurrent networks

2019 ◽  
Author(s):  
Yue Liu ◽  
Marc W. Howard
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Initial Conditions ◽  
A Priori ◽  
Network Models ◽  
Neural Mechanism ◽  
Natural World ◽  
Neural Network Models ◽  
Scale Invariant ◽  
Wide Range

AbstractSequential neural activity has been observed in many parts of the brain and has been proposed as a neural mechanism for memory. The natural world expresses temporal relationships at a wide range of scales. Because we cannot know the relevant scales a priori it is desirable that memory, and thus the generated sequences, are scale-invariant. Although recurrent neural network models have been proposed as a mechanism for generating sequences, the requirements for scale-invariant sequences are not known. This paper reports the constraints that enable a linear recurrent neural network model to generate scale-invariant sequential activity. A straightforward eigendecomposition analysis results in two independent conditions that are required for scaleinvariance for connectivity matrices with real, distinct eigenvalues. First, the eigenvalues of the network must be geometrically spaced. Second, the eigenvectors must be related to one another via translation. These constraints are easily generalizable for matrices that have complex and distinct eigenvalues. Analogous albeit less compact constraints hold for matrices with degenerate eigenvalues. These constraints, along with considerations on initial conditions, provide a general recipe to build linear recurrent neural networks that support scale-invariant sequential activity.

scholarly journals Computation of the electroencephalogram (EEG) from network models of point neurons

2020 ◽  
Author(s):  
Pablo Martínez-Cañada ◽  
Torbjørn V. Ness ◽  
Gaute T. Einevoll ◽  
Tommaso Fellin ◽  
Stefano Panzeri
Keyword(s):  
Neural Network ◽  
Brain Function ◽  
Network Models ◽  
Ground Truth ◽  
Recording Electrode ◽  
Neuron Network ◽  
Neural Network Models ◽  
Synaptic Currents ◽  
Wide Range ◽  
Electroencephalogram Eeg

AbstractThe electroencephalogram (EEG) is one of the main tools for non-invasively studying brain function and dysfunction. To better interpret EEGs in terms of neural mechanisms, it is important to compare experimentally recorded EEGs with the output of neural network models. Most current neural network models use networks of simple point neurons. They capture important properties of cortical dynamics, and are numerically or analytically tractable. However, point neuron networks cannot directly generate an EEG, since EEGs are generated by spatially separated transmembrane currents. Here, we explored how to compute an accurate approximation of the EEG with a combination of quantities defined in point-neuron network models. We constructed several different candidate approximations (or proxies) of the EEG that can be computed from networks of leaky integrate-and-fire (LIF) point neurons, such as firing rates, membrane potentials, and specific combinations of synaptic currents. We then evaluated how well each proxy reconstructed a realistic ground-truth EEG obtained when the synaptic input currents of the LIF network were fed into a three-dimensional (3D) network model of multi-compartmental neurons with realistic cell morphologies. We found that a new class of proxies, based on an optimized linear combination of time-shifted AMPA and GABA currents, provided the most accurate estimate of the EEG over a wide range of network states of the LIF point-neuron network. The new linear proxies explained most of the variance (85-95%) of the ground-truth EEG for a wide range of cell morphologies, distributions of presynaptic inputs, and position of the recording electrode. Non-linear proxies, obtained using a convolutional neural network (CNN) to predict the EEG from synaptic currents, increased proxy performance by a further 2-8%. Our proxies can be used to easily calculate a biologically realistic EEG signal directly from point-neuron simulations and thereby allow a quantitative comparison between computational models and experimental EEG recordings.Author summaryNetworks of point neurons are widely used to model neural dynamics. Their output, however, cannot be directly compared to the electroencephalogram (EEG), which is one of the most used tools to non-invasively measure brain activity. To allow a direct integration between neural network theory and empirical EEG data, here we derived a new mathematical expression, termed EEG proxy, which estimates with high accuracy the EEG based simply on the variables available from simulations of point-neuron network models. To compare and validate these EEG proxies, we computed a realistic ground-truth EEG produced by a network of simulated neurons with realistic 3D morphologies that receive the same spikes of the simpler network of point neurons. The new obtained EEG proxies outperformed previous approaches and worked well under a wide range of simulated configurations of cell morphologies, distribution of presynaptic inputs, and position of the recording electrode. The new proxies approximated well both EEG spectra and EEG evoked potentials. Our work provides important mathematical tools that allow a better interpretation of experimentally measured EEGs in terms of neural models of brain function.

Sign in / Sign up

Export Citation Format

Share Document