How Optimal Stimuli for Sensory Neurons Are Constrained by Network Architecture

2008 ◽  
Vol 20 (3) ◽  
pp. 668-708 ◽  
Author(s):  
Christopher DiMattina ◽  
Kechen Zhang
Keyword(s):  
Firing Rate ◽  
Network Architecture ◽  
Single Neuron ◽  
Network Models ◽  
Synaptic Connections ◽  
Trial And Error ◽  
Neural Network Models ◽  
Stimulus Response ◽  
The Relationship ◽  
Topological Boundary

Identifying the optimal stimuli for a sensory neuron is often a difficult process involving trial and error. By analyzing the relationship between stimuli and responses in feedforward and stable recurrent neural network models, we find that the stimulus yielding the maximum firing rate response always lies on the topological boundary of the collection of all allowable stimuli, provided that individual neurons have increasing input-output relations or gain functions and that the synaptic connections are convergent between layers with nondegenerate weight matrices. This result suggests that in neurophysiological experiments under these conditions, only stimuli on the boundary need to be tested in order to maximize the response, thereby potentially reducing the number of trials needed for finding the most effective stimuli. Even when the gain functions allow firing rate cutoff or saturation, a peak still cannot exist in the stimulus-response relation in the sense that moving away from the optimum stimulus always reduces the response. We further demonstrate that the condition for nondegenerate synaptic connections also implies that proper stimuli can independently perturb the activities of all neurons in the same layer. One example of this type of manipulation is changing the activity of a single neuron in a given processing layer while keeping that of all others constant. Such stimulus perturbations might help experimentally isolate the interactions of selected neurons within a network.

Spatial Variability Aware Deep Neural Networks (SVANN): A General Approach

2021 ◽  
Vol 12 (6) ◽  
pp. 1-21
Author(s):  
Jayant Gupta ◽  
Carl Molnar ◽  
Yiqun Xie ◽  
Joe Knight ◽  
Shashi Shekhar
Keyword(s):  
Neural Network ◽  
Spatial Variability ◽  
Network Architecture ◽  
Network Models ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
Climatic Zones ◽  
The Neural Network ◽  
Plant Hardiness ◽  
Interpretation Model

Spatial variability is a prominent feature of various geographic phenomena such as climatic zones, USDA plant hardiness zones, and terrestrial habitat types (e.g., forest, grasslands, wetlands, and deserts). However, current deep learning methods follow a spatial-one-size-fits-all (OSFA) approach to train single deep neural network models that do not account for spatial variability. Quantification of spatial variability can be challenging due to the influence of many geophysical factors. In preliminary work, we proposed a spatial variability aware neural network (SVANN-I, formerly called SVANN ) approach where weights are a function of location but the neural network architecture is location independent. In this work, we explore a more flexible SVANN-E approach where neural network architecture varies across geographic locations. In addition, we provide a taxonomy of SVANN types and a physics inspired interpretation model. Experiments with aerial imagery based wetland mapping show that SVANN-I outperforms OSFA and SVANN-E performs the best of all.

scholarly journals A data-driven neural network architecture for sentiment analysis

2019 ◽  
Vol 53 (1) ◽  
pp. 2-19 ◽  
Author(s):  
Erion Çano ◽  
Maurizio Morisio
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Network Architecture ◽  
Network Models ◽  
Data Sets ◽  
Feature Maps ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
Content Type ◽  
Max Pooling

Purpose The fabulous results of convolution neural networks in image-related tasks attracted attention of text mining, sentiment analysis and other text analysis researchers. It is, however, difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. The purpose of this paper is to present the creation steps of two big data sets of song emotions. The authors also explore usage of convolution and max-pooling neural layers on song lyrics, product and movie review text data sets. Three variants of a simple and flexible neural network architecture are also compared. Design/methodology/approach The intention was to spot any important patterns that can serve as guidelines for parameter optimization of similar models. The authors also wanted to identify architecture design choices which lead to high performing sentiment analysis models. To this end, the authors conducted a series of experiments with neural architectures of various configurations. Findings The results indicate that parallel convolutions of filter lengths up to 3 are usually enough for capturing relevant text features. Also, max-pooling region size should be adapted to the length of text documents for producing the best feature maps. Originality/value Top results the authors got are obtained with feature maps of lengths 6–18. An improvement on future neural network models for sentiment analysis could be generating sentiment polarity prediction of documents using aggregation of predictions on smaller excerpt of the entire text.

scholarly journals Data-to-Text Generation with Content Selection and Planning

2019 ◽  
Vol 33 ◽  
pp. 6908-6915 ◽  
Author(s):  
Ratish Puduppully ◽  
Li Dong ◽  
Mirella Lapata
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Large Scale ◽  
Network Models ◽  
Text Generation ◽  
Neural Network Models ◽  
Generation Task ◽  
Content Selection ◽  
End To End ◽  
Two Stages

Recent advances in data-to-text generation have led to the use of large-scale datasets and neural network models which are trained end-to-end, without explicitly modeling what to say and in what order. In this work, we present a neural network architecture which incorporates content selection and planning without sacrificing end-to-end training. We decompose the generation task into two stages. Given a corpus of data records (paired with descriptive documents), we first generate a content plan highlighting which information should be mentioned and in which order and then generate the document while taking the content plan into account. Automatic and human-based evaluation experiments show that our model1 outperforms strong baselines improving the state-of-the-art on the recently released RotoWIRE dataset.

Variability in somatosensory cortical neuron discharge: effects on capacity to signal different stimulus conditions using a mean rate code

1978 ◽  
Vol 41 (2) ◽  
pp. 338-349 ◽  
Author(s):  
R. C. Schreiner ◽  
G. K. Essick ◽  
B. L. Whitsel
Keyword(s):  
Firing Rate ◽  
Linear Equation ◽  
Cortical Neuron ◽  
Single Neuron ◽  
Rate Code ◽  
Firing Rates ◽  
Discrimination Criterion ◽  
The Mean ◽  
Change In Mean ◽  
The Relationship

1. The present study is based on the demonstration (8, 9) that the relationship between mean interval (MI) and standard deviation (SD) for stimulus-driven activity recorded from SI neurons is well fitted by the linear equation SD = a X MI + b and on the observations that the values of the slope (a) and y intercept (b) parameters of this relationship are independent of stimulus conditions and may vary widely from one neuron to the next (8). 2. A criterion for the discriminability of two different mean firing rates requiring that the mean intervals of their respective interspike interval (ISI) distributions be separated by a fixed interval (expressed in SD units) is developed and, on the basis of this criterion, a graphical display of the capacity of a neuron with a known SD-MI relationship to reflect a change in stimulus conditions with a change in mean firing rate is derived. Using this graphical approach, it is shown that the parameters of the SD-MI relationship for a single neuron determine a range of firing frequencies, within which that neuron exhibits the greatest capacity to signal differences in stimulus conditions using a frequency code. 3. The discrimination criterion is modified to incorporate the changes in the symmetry of the ISI distribution observed to accompany changes in mean firing rate. It is shown that, although the observed symmetry changes do influence the capacity of a cortical neuron to signal a change in stimulus conditions with a change in mean firing rate, they do not alter the range of firing rates (determined by the parameters of the SD-MI relationship) within which the capacity for discrimination is maximal. 4. The maximal number of firing levels that can be distinguished by a somatosensory cortical neuron (using the same discrimination criterion described above) discharging within a specified range of mean frequencies also is demonstrated to depend on the parameters of the linear equation which relates SD to MI. 5. Two approaches based on the t test for differences between two means are developed in an attempt to ascertain the minimum separation of the mean intervals of the ISI distributions necessary for two different mean firing rates to be discriminated with 80% certainty.

A Multi-Scale Modeling Method Based on Big Data

2015 ◽  
Vol 734 ◽  
pp. 447-450 ◽  
Author(s):  
Ji Wei Liu
Keyword(s):  
Big Data ◽  
Network Models ◽  
Modeling Method ◽  
Scale Model ◽  
Neural Network Models ◽  
Multi Scale ◽  
Scale Modeling ◽  
Input And Output ◽  
Multi Scale Modeling ◽  
The Relationship

A multi-scale modeling method based on big data was proposed to establish neural network models for complex plant. Wavelet transform was used to decompose input and output parameters into different scales. The relationship between these parameters were researched in every scale. Then models in each scale were established and added together to form a multi-scale model. A model of coal mill current in power plant was established using the multi-scale modeling method based on big data. The result shows that, the method is effective.

Unsupervised Mutual Information Criterion for Elimination of Overtraining in Supervised Multilayer Networks

1995 ◽  
Vol 7 (1) ◽  
pp. 86-107 ◽  
Author(s):  
G. Deco ◽  
W. Finnoff ◽  
H. G. Zimmermann
Keyword(s):  
Mutual Information ◽  
Network Architecture ◽  
Synthetic Data ◽  
Network Models ◽  
Information Criterion ◽  
Small Data ◽  
Economic Time Series ◽  
Neural Network Models ◽  
Data Set ◽  
Penalty Term

Controlling the network complexity in order to prevent overfitting is one of the major problems encountered when using neural network models to extract the structure from small data sets. In this paper we present a network architecture designed for use with a cost function that includes a novel complexity penalty term. In this architecture the outputs of the hidden units are strictly positive and sum to one, and their outputs are defined as the probability that the actual input belongs to a certain class formed during learning. The penalty term expresses the mutual information between the inputs and the extracted classes. This measure effectively describes the network complexity with respect to the given data in an unsupervised fashion. The efficiency of this architecture/penalty-term when combined with backpropagation training, is demonstrated on a real world economic time series forecasting problem. The model was also applied to the benchmark sunspot data and to a synthetic data set from the statistics community.

scholarly journals Developing a seq2seq neural network using visual attention to transform mathematical expressions from images to LaTeX.

Doklady BGUIR ◽  
2022 ◽  
Vol 19 (8) ◽  
pp. 40-44
Author(s):  
P. A. Vyaznikov ◽  
I. D. Kotilevets
Keyword(s):  
Neural Network ◽  
Visual Attention ◽  
Network Architecture ◽  
Network Models ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
The Neural Network ◽  
Mathematical Expressions ◽  
Series Of Experiments ◽  
Visual Attention Mechanism

The paper presents the methods of development and the results of research on the effectiveness of the seq2seq neural network architecture using Visual Attention mechanism to solve the im2latex problem. The essence of the task is to create a neural network capable of converting an image with mathematical expressions into a similar expression in the LaTeX markup language. This problem belongs to the Image Captioning type: the neural network scans the image and, based on the extracted features, generates a description in natural language. The proposed solution uses the seq2seq architecture, which contains the Encoder and Decoder mechanisms, as well as Bahdanau Attention. A series of experiments was conducted on training and measuring the effectiveness of several neural network models.

Mutual Inhibition with Few Inhibitory Cells via Nonlinear Inhibitory Synaptic Interaction

2019 ◽  
Vol 31 (11) ◽  
pp. 2252-2265
Author(s):  
Felix Weissenberger ◽  
Marcelo Matheus Gauy ◽  
Xun Zou ◽  
Angelika Steger
Keyword(s):  
Neural Network ◽  
Nonlinear Interaction ◽  
Network Models ◽  
Inhibitory Neurons ◽  
Inhibitory Synapses ◽  
Synaptic Connections ◽  
Mutual Inhibition ◽  
Neural Network Models ◽  
Excitatory Neurons ◽  
Inhibitory Population

In computational neural network models, neurons are usually allowed to excite some and inhibit other neurons, depending on the weight of their synaptic connections. The traditional way to transform such networks into networks that obey Dale's law (i.e., a neuron can either excite or inhibit) is to accompany each excitatory neuron with an inhibitory one through which inhibitory signals are mediated. However, this requires an equal number of excitatory and inhibitory neurons, whereas a realistic number of inhibitory neurons is much smaller. In this letter, we propose a model of nonlinear interaction of inhibitory synapses on dendritic compartments of excitatory neurons that allows the excitatory neurons to mediate inhibitory signals through a subset of the inhibitory population. With this construction, the number of required inhibitory neurons can be reduced tremendously.

A systematic approach for segmenting voiced/unvoiced signals using fuzzy-logic system and general fusion of neural network models for phonemes-based speech recognition

2020 ◽  
Vol 39 (5) ◽  
pp. 7411-7429
Author(s):  
Sathees Kumar Nataraj ◽  
M. P. Paulraj ◽  
Ahmad Nazri Bin Abdullah ◽  
Sazali Bin Yaacob
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Spectral Band ◽  
False Negative ◽  
Network Models ◽  
Neural Network Models ◽  
Robust Model ◽  
The Neural Network ◽  
Total Data ◽  
Trained Network

In this paper, a speech-to-text translation model has been developed for Malaysian speakers based on 41 classes of Phonemes. A simple data acquisition algorithm has been used to develop a MATLAB graphical user interface (GUI) for recording the isolated word speech signals from 35 non-native Malaysian speakers. The collected database consists of 86 words with 41 classes of phoneme based on Affricatives, Diphthongs, Fricatives, Liquid, Nasals, Semivowels and Glides, Stop and Vowels. The speech samples are preprocessed to eliminate the undesirable artifacts and the fuzzy voice classifier has been employed to classify the samples into voiced sequence and unvoiced sequence. The voiced sequences are divided into frame segments and for each frame, the Linear Predictive co-efficients features are obtained from the voiced sequence. Then the feature sets are formed by deriving the LPC features from all the extracted voiced sequences, and used for classification. The isolated words chosen based on the phonemes are associated with the extracted features to establish classification system input-output mapping. The data are then normalized and randomized to rearrange the values into definite range. The Multilayer Neural Network (MLNN) model has been developed with four combinations of input and hidden activation functions. The neural network models are trained with 60%, 70% and 80% of the total data samples. The neural network architecture was aimed at creating a robust model with 60%, 70%, and 80% of the feature set with 25 trials. The trained network model is validated by simulating the network with the remaining 40%, 30%, and 20% of the set. The reliability of trained network models were compared by measuring true-positive, false-negative, and network classification accuracy. The LPC features show better discrimination and the MLNN neural network models trained using the LPC spectral band features gives better recognition.

scholarly journals Research of Vehicle Rear-End Collision Model considering Multiple Factors

2020 ◽  
Vol 2020 ◽  
pp. 1-11 ◽  
Author(s):  
Qiang Luo ◽  
Xiaodong Zang ◽  
Jie Yuan ◽  
Xinqiang Chen ◽  
Junheng Yang ◽  
...  
Keyword(s):  
Traffic Safety ◽  
Traffic Accident ◽  
Model Performance ◽  
Network Models ◽  
Neural Network Models ◽  
Collision Model ◽  
Car Following ◽  
Research Findings ◽  
Two Parameters ◽  
The Relationship

The accuracy of the rear-end collision models is crucial for the early warning of potential traffic accident identification, and thus analyzes of the main factors influencing the rear-end collision relevant models is an active topic in the field. The previous studies have tried to determine the single factor influence on the rear-end collision model performance. Less attention was paid to exploit mutual influences on the model performance. To bridge the gap, we proposed an improved vehicle rear-end collision model by integrating varied factors which influence two parameters (i.e., response time and road adhesion coefficient). The two parameters were solved with the integrated weighting and neural network models, respectively. After that we analyzed the relationship between varied factors and the minimum car-following distance. The research findings support both the theoretical and practical guidance for transportation regulations to release more reasonable minimum headway distance to enhance the roadway traffic safety.

Sign in / Sign up

Export Citation Format

Share Document