scholarly journals Neural oligarchy: how synaptic plasticity breeds neurons with extreme influence

2018 ◽  
Author(s):  
Florence I. Kleberg ◽  
Jochen Triesch
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Synaptic Plasticity ◽  
Network Model ◽  
Neural Network Model ◽  
Network Dynamics ◽  
Inhibitory Neurons ◽  
Inhibitory Synapses ◽  
Firing Rates ◽  
Long Tailed Distribution

AbstractSynapses between cortical neurons are subject to constant modifications through synaptic plasticity mechanisms, which are believed to underlie learning and memory formation. The strengths of excitatory and inhibitory synapses in the cortex follow a right-skewed long-tailed distribution. Similarly, the firing rates of excitatory and inhibitory neurons also follow a right-skewed long-tailed distribution. How these distributions come about and how they maintain their shape over time is currently not well understood. Here we propose a spiking neural network model that explains the origin of these distributions as a consequence of the interaction of spike-timing dependent plasticity (STDP) of excitatory and inhibitory synapses and a multiplicative form of synaptic normalisation. Specifically, we show that the combination of additive STDP and multiplicative normalisation leads to lognormal-like distributions of excitatory and inhibitory synaptic efficacies as observed experimentally. The shape of these distributions remains stable even if spontaneous fluctuations of synaptic efficacies are added. In the same network, lognormal-like distributions of the firing rates of excitatory and inhibitory neurons result from small variability in the spiking thresholds of individual neurons. Interestingly, we find that variation in firing rates is strongly coupled to variation in synaptic efficacies: neurons with the highest firing rates develop very strong connections onto other neurons. Finally, we define an impact measure for individual neurons and demonstrate the existence of a small group of neurons with an exceptionally strong impact on the network, that arise as a result of synaptic plasticity. In summary, synaptic plasticity and small variability in neuronal parameters underlie a neural oligarchy in recurrent neural networks.Author summaryOur brain’s neural networks are composed of billions of neurons that exchange signals via trillions of synapses. Are these neurons created equal, or do they contribute in similar ways to the network dynamics? Or do some neurons wield much more power than others? Recent experiments have shown that some neurons are much more active than the average neuron and that some synaptic connections are much stronger than the average synaptic connection. However, it is still unclear how these properties come about in the brain. Here we present a neural network model that explains these findings as a result of the interaction of synaptic plasticity mechanisms that modify synapses’ efficacies. The model reproduces recent findings on the statistics of neuronal firing rates and synaptic efficacies and predicts a small class of neurons with exceptionally high impact on the network dynamics. Such neurons may play a key role in brain disorders such as epilepsy.

Dual Information Representation with Stable Firing Rates and Chaotic Spatiotemporal Spike Patterns in a Neural Network Model

2001 ◽  
Vol 13 (12) ◽  
pp. 2799-2822 ◽  
Author(s):  
Osamu Araki ◽  
Kazuyuki Aihara
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Dynamics ◽  
Emergent Properties ◽  
Recent Experimental Data ◽  
Information Representation ◽  
Firing Rates ◽  
Properties Of Information ◽  
Spike Patterns

Although various means of information representation in the cortex have been considered, the fundamental mechanism for such representation is not well understood. The relation between neural network dynamics and properties of information representation needs to be examined. We examined spatial pattern properties of mean firing rates and spatiotemporal spikes in an interconnected spiking neural network model. We found that whereas the spatiotemporal spike patterns are chaotic and unstable, the spatial patterns of mean firing rates (SPMFR) are steady and stable, reflecting the internal structure of synaptic weights. Interestingly, the chaotic instability contributes to fast stabilization of the SPMFR. Findings suggest that there are two types of network dynamics behind neuronal spiking: internally-driven dynamics and externally driven dynamics. When the internally driven dynamics dominate, spikes are relatively more chaotic and independent of external inputs; the SPMFR are steady and stable. When the externally driven dynamics dominate, the spiking patterns are relatively more dependent on the spatiotemporal structure of external inputs. These emergent properties of information representation imply that the brain may adopt a dual coding system. Recent experimental data suggest that internally driven and externally driven dynamics coexist and work together in the cortex.

scholarly journals Dynamic versus static neural network model for rainfall forecasting at Klang River Basin, Malaysia

2012 ◽  
Vol 16 (4) ◽  
pp. 1151-1169 ◽  
Author(s):  
A. El-Shafie ◽  
A. Noureldin ◽  
M. Taha ◽  
A. Hussain ◽  
M. Mukhlisin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Rainfall Time Series ◽  
Hydrological Process ◽  
Multi Layer Perceptron ◽  
Rainfall Forecasting ◽  
Unseen Data

Abstract. Rainfall is considered as one of the major components of the hydrological process; it takes significant part in evaluating drought and flooding events. Therefore, it is important to have an accurate model for rainfall forecasting. Recently, several data-driven modeling approaches have been investigated to perform such forecasting tasks as multi-layer perceptron neural networks (MLP-NN). In fact, the rainfall time series modeling involves an important temporal dimension. On the other hand, the classical MLP-NN is a static and has a memoryless network architecture that is effective for complex nonlinear static mapping. This research focuses on investigating the potential of introducing a neural network that could address the temporal relationships of the rainfall series. Two different static neural networks and one dynamic neural network, namely the multi-layer perceptron neural network (MLP-NN), radial basis function neural network (RBFNN) and input delay neural network (IDNN), respectively, have been examined in this study. Those models had been developed for the two time horizons for monthly and weekly rainfall forecasting at Klang River, Malaysia. Data collected over 12 yr (1997–2008) on a weekly basis and 22 yr (1987–2008) on a monthly basis were used to develop and examine the performance of the proposed models. Comprehensive comparison analyses were carried out to evaluate the performance of the proposed static and dynamic neural networks. Results showed that the MLP-NN neural network model is able to follow trends of the actual rainfall, however, not very accurately. RBFNN model achieved better accuracy than the MLP-NN model. Moreover, the forecasting accuracy of the IDNN model was better than that of static network during both training and testing stages, which proves a consistent level of accuracy with seen and unseen data.

Credit Scoring Using Supervised and Unsupervised Neural Networks

2002 ◽  
pp. 154-166 ◽  
Author(s):  
David West ◽  
Cornelius Muchineuta
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Support ◽  
Network Model ◽  
Neural Network Model ◽  
Credit Scoring ◽  
Self Organizing Map ◽  
Subprime Lending ◽  
The Neural Network ◽  
Combined Use

Some of the concerns that plague developers of neural network decision support systems include: (a) How do I understand the underlying structure of the problem domain; (b) How can I discover unknown imperfections in the data which might detract from the generalization accuracy of the neural network model; and (c) What variables should I include to obtain the best generalization properties in the neural network model? In this paper we explore the combined use of unsupervised and supervised neural networks to address these concerns. We develop and test a credit-scoring application using a self-organizing map and a multilayered feedforward neural network. The final product is a neural network decision support system that facilitates subprime lending and is flexible and adaptive to the needs of e-commerce applications.

Estimation of Polypropylene Melt Index through the Application of Stacked Neural Networks

2011 ◽  
Vol 187 ◽  
pp. 411-415
Author(s):  
Lu Yue Xia ◽  
Hai Tian Pan ◽  
Meng Fei Zhou ◽  
Yi Jun Cai ◽  
Xiao Fang Sun
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Model Performance ◽  
Measurement Interval ◽  
Melt Index ◽  
Polypropylene Melt ◽  
Single Neural Network

Melt index is the most important parameter in determining the polypropylene grade. Since the lack of proper on-line instruments, its measurement interval and delay are both very long. This makes the quality control quite difficult. A modeling approach based on stacked neural networks is proposed to estimation the polypropylene melt index. Single neural network model generalization capability can be significantly improved by using stacked neural networks model. Proper determination of the stacking weights is essential for good stacked neural networks model performance, so determination of appropriate weights for combining individual networks using the criteria about minimization of sum of absolute prediction error is proposed. Application to real industrial data demonstrates that the polypropylene melt index can be successfully estimated using stacked neural networks. The results obtained demonstrate significant improvements in model accuracy, as a result of using stacked neural networks model, compared to using single neural network model.

APPLYING NEURAL NETWORKS TO SOFTWARE RELIABILITY ASSESSMENT

2010 ◽  
Vol 17 (04) ◽  
pp. 313-329 ◽  
Author(s):  
NORMAN SCHNEIDEWIND
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Software Reliability ◽  
Likelihood Estimation ◽  
Time To Failure ◽  
The Neural Network ◽  
Method Of Maximum Likelihood ◽  
Network Method

We adapt concepts from the field of neural networks to assess the reliability of software, employing cumulative failures, reliability, remaining failures, and time to failure metrics. In addition, the risk of not achieving reliability, remaining failure, and time to failure goals are assessed. The purpose of the assessment is to compare a criterion, derived from a neural network model, for estimating the parameters of software reliability metrics, with the method of maximum likelihood estimation. To our surprise the neural network method proved superior for all the reliability metrics that were assessed by virtue of yielding lower prediction error and risk. We also found that considerable adaptation of the neural network model was necessary to be meaningful for our application – only inputs, functions, neurons, weights, activation units, and outputs were required to characterize our application.

DYNAMIC OUTPUT FEEDBACK STABILIZATION FOR NONLINEAR SYSTEMS BASED ON STANDARD NEURAL NETWORK MODELS

2006 ◽  
Vol 16 (04) ◽  
pp. 305-317 ◽  
Author(s):  
MEIQIN LIU
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Nonlinear Systems ◽  
Network Model ◽  
Neural Network Model ◽  
Output Feedback ◽  
Control Design ◽  
Feedback Stabilization ◽  
Dynamic Output Feedback ◽  
Dynamic Output

A neural-model-based control design for some nonlinear systems is addressed. The design approach is to approximate the nonlinear systems with neural networks of which the activation functions satisfy the sector conditions. A novel neural network model termed standard neural network model (SNNM) is advanced for describing this class of approximating neural networks. Full-order dynamic output feedback control laws are then designed for the SNNMs with inputs and outputs to stabilize the closed-loop systems. The control design equations are shown to be a set of linear matrix inequalities (LMIs) which can be easily solved by various convex optimization algorithms to determine the control signals. It is shown that most neural-network-based nonlinear systems can be transformed into input-output SNNMs to be stabilization synthesized in a unified way. Finally, some application examples are presented to illustrate the control design procedures.

A NEURAL NETWORK MODEL FOR CREDIT RISK EVALUATION

2009 ◽  
Vol 19 (04) ◽  
pp. 285-294 ◽  
Author(s):  
ADNAN KHASHMAN
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Credit Risk ◽  
Network Model ◽  
Neural Network Model ◽  
Risk Evaluation ◽  
Evaluation System ◽  
Learning Algorithm ◽  
Back Propagation ◽  
Learning Schemes

Credit scoring is one of the key analytical techniques in credit risk evaluation which has been an active research area in financial risk management. This paper presents a credit risk evaluation system that uses a neural network model based on the back propagation learning algorithm. We train and implement the neural network to decide whether to approve or reject a credit application, using seven learning schemes and real world credit applications from the Australian credit approval datasets. A comparison of the system performance under the different learning schemes is provided, furthermore, we compare the performance of two neural networks; with one and two hidden layers following the ideal learning scheme. Experimental results suggest that neural networks can be effectively used in automatic processing of credit applications.

Establish an Artificial Neural Networks Model to Make Quantitative Analysis about the Capillary Electrophoresis Spectrum

2012 ◽  
Vol 452-453 ◽  
pp. 1116-1120
Author(s):  
Hong Ping Li ◽  
Hong Li
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Capillary Electrophoresis ◽  
Artificial Neural Networks ◽  
Quantitative Analysis ◽  
Network Model ◽  
Neural Network Model ◽  
Network Training ◽  
Dissociation Degree ◽  
Artificial Neural

Simulating the overlapping capillary electrophoresis spectrogram under the dissimilar conditions by the computer system , Choosing the overlapping capillary electrophoresis spectrogram simulated under the different conditions , processing the data to compose a neural network training regulations, Applying the artificial neural networks method to make a quantitative analysis about the multi-component in the overlapping capillary electrophoresis spectrogram,Using: Radial direction primary function neural network model and multi-layered perceptron neural network model. The findings indicated that, along with the increasing of the capillary electrophoresis spectrogram noise level, the related components’ ability of the two kinds of the overlapping capillary electrophoresis spectrogram by neural network model quantitative analysis drop down. Along with the increasing of the capillary electrophoresis spectrogram’s total dissociation degree, the multi-layered perceptron neural network model to the related components’ ability of the overlapping capillary electrophoresis spectum by quantitative analysis raise up.

scholarly journals Exploring the limits of learning: segregation of information integration and response selection is required for learning a serial reversal task

2017 ◽  
Author(s):  
Camilo J. Mininni ◽  
B. Silvano Zanutto
Keyword(s):  
Neural Network ◽  
Synaptic Plasticity ◽  
Network Model ◽  
Neural Network Model ◽  
Response Selection ◽  
Neural Population ◽  
Spiking Neural Network ◽  
Reversal Task ◽  
Spiking Neural Network Model ◽  
Serial Reversal

AbstractAnimals are proposed to learn the latent rules governing their environment in order to maximize their chances of survival. However, rules may change without notice, forcing animals to keep a memory of which one is currently at work. Rule switching can lead to situations in which the same stimulus/response pairing is positively and negatively rewarded in the long run, depending on variables that are not accessible to the animal. This fact rises questions on how neural systems are capable of reinforcement learning in environments where the reinforcement is inconsistent. Here we address this issue by asking about which aspects of connectivity, neural excitability and synaptic plasticity are key for a very general, stochastic spiking neural network model to solve a task in which rules change without being cued, taking the serial reversal task (SRT) as paradigm. Contrary to what could be expected, we found strong limitations for biologically plausible networks to solve the SRT. Especially, we proved that no network of neurons can learn a SRT if it is a single neural population that integrates stimuli information and at the same time is responsible of choosing the behavioural response. This limitation is independent of the number of neurons, neuronal dynamics or plasticity rules, and arises from the fact that plasticity is locally computed at each synapse, and that synaptic changes and neuronal activity are mutually dependent processes. We propose and characterize a spiking neural network model that solves the SRT, which relies on separating the functions of stimuli integration and response selection. The model suggests that experimental efforts to understand neural function should focus on the characterization of neural circuits according to their connectivity, neural dynamics, and the degree of modulation of synaptic plasticity with reward.

scholarly journals Neural network method as means of processing experimental data on grain crop yields

2020 ◽  
Vol 161 ◽  
pp. 01031
Author(s):  
Aleksandr Nikiforov ◽  
Aleksei Kuchumov ◽  
Sergei Terentev ◽  
Inessa Karamulina ◽  
Iraida Romanova ◽  
...  
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Crop Yields ◽  
Winter Triticale ◽  
The Neural Network ◽  
Grain Crop ◽  
Scientific Results

In the work based on agroecological and technological testing of varieties of grain crops of domestic and foreign breeding, winter triticale in particular, conducted on the experimental field of the Smolensk State Agricultural Academy between 2015 and 2019, we present the methodology and results of processing the experimental data used for constructing the neural network model. Neural networks are applicable for solving tasks that are difficult for computers of traditional design and humans alike. Those are processing large volumes of experimental data, automation of image recognition, approximation of functions and prognosis. Neural networks include analyzing subject areas and weight coefficients of neurons, detecting conflict samples and outliers, normalizing data, determining the number of samples required for teaching a neural network and increasing the learning quality when their number is insufficient, as well as selecting the neural network type and decomposition based on the number of input neurons. We consider the technology of initial data processing and selecting the optimal neural network structure that allows to significantly reduce modeling errors in comparison with neural networks created with unprepared source data. Our accumulated experience of working with neural networks has demonstrated encouraging results, which indicates the prospects of this area, especially when describing processes with large amounts of variables. In order to verify the resulting neural network model, we have carried out a computational experiment, which showed the possibility of applying scientific results in practice.

Sign in / Sign up

Export Citation Format

Share Document