scholarly journals Synaptic Dynamics as Convolutional Units

2020 ◽  
Author(s):  
Julian Rossbroich ◽  
Daniel Trotter ◽  
Katalin Tóth ◽  
Richard Naud
Keyword(s):  
Neural Networks ◽  
Action Potentials ◽  
Computational Models ◽  
Mathematical Framework ◽  
Challenging Problem ◽  
Maximum Likelihood Approach ◽  
Different Types ◽  
Likelihood Approach ◽  
Synaptic Dynamics ◽  
Nonlinear Operation

AbstractSynaptic dynamics differ markedly across connections and strongly regulate how action potentials are being communicated. To model the range of synaptic dynamics observed in experiments, we develop a flexible mathematical framework based on a linear-nonlinear operation. This model can capture various experimentally observed features of synaptic dynamics and different types of heteroskedasticity. Despite its conceptual simplicity, we show it is more adaptable than previous models. Combined with a standard maximum likelihood approach, synaptic dynamics can be accurately and efficiently characterized using naturalistic stimulation patterns. These results make explicit that synaptic processing bears algorithmic similarities with information processing in convolutional neural networks.Author summaryUnderstanding how information is transmitted relies heavily on knowledge of the underlying regulatory synaptic dynamics. Existing computational models for capturing such dynamics are often either very complex or too restrictive. As a result, effectively capturing the different types of dynamics observed experimentally remains a challenging problem. Here, we propose a mathematically flexible linear-nonlinear model that is capable of efficiently characterizing synaptic dynamics. We demonstrate the ability of this model to capture different features of experimentally observed data.

scholarly journals Linear-nonlinear cascades capture synaptic dynamics

2021 ◽  
Vol 17 (3) ◽  
pp. e1008013
Author(s):  
Julian Rossbroich ◽  
Daniel Trotter ◽  
John Beninger ◽  
Katalin Tóth ◽  
Richard Naud
Keyword(s):  
Neural Networks ◽  
Maximum Likelihood ◽  
Action Potentials ◽  
Mathematical Framework ◽  
Short Term ◽  
Maximum Likelihood Approach ◽  
Different Types ◽  
Likelihood Approach ◽  
Synaptic Dynamics ◽  
Nonlinear Operation

Short-term synaptic dynamics differ markedly across connections and strongly regulate how action potentials communicate information. To model the range of synaptic dynamics observed in experiments, we have developed a flexible mathematical framework based on a linear-nonlinear operation. This model can capture various experimentally observed features of synaptic dynamics and different types of heteroskedasticity. Despite its conceptual simplicity, we show that it is more adaptable than previous models. Combined with a standard maximum likelihood approach, synaptic dynamics can be accurately and efficiently characterized using naturalistic stimulation patterns. These results make explicit that synaptic processing bears algorithmic similarities with information processing in convolutional neural networks.

Simulator for neural networks and action potentials: description and application

1994 ◽  
Vol 71 (1) ◽  
pp. 294-308 ◽  
Author(s):  
I. Ziv ◽  
D. A. Baxter ◽  
J. H. Byrne
Keyword(s):  
Neural Networks ◽  
Central Pattern Generator ◽  
Action Potentials ◽  
Modular Design ◽  
Second Messenger ◽  
Pattern Generator ◽  
Slow Inward Current ◽  
Synaptic Connections ◽  
Different Types ◽  
Voltage Dependent

1. We describe a simulator for neural networks and action potentials (SNNAP) that can simulate up to 30 neurons, each with up to 30 voltage-dependent conductances, 30 electrical synapses, and 30 multicomponent chemical synapses. Voltage-dependent conductances are described by Hodgkin-Huxley type equations, and the contributions of time-dependent synaptic conductances are described by second-order differential equations. The program also incorporates equations for simulating different types of neural modulation and synaptic plasticity. 2. Parameters, initial conditions, and output options for SNNAP are passed to the program through a number of modular ASCII files. These modules can be modified by commonly available text editors that use a conventional (i.e., character based) interface or by an editor incorporated into SNNAP that uses a graphical interface. The modular design facilitates the incorporation of existing modules into new simulations. Thus libraries can be developed of files describing distinctive cell types and files describing distinctive neural networks. 3. Several different types of neurons with distinct biophysical properties and firing properties were simulated by incorporating different combinations of voltage-dependent Na+, Ca2+, and K+ channels as well as Ca(2+)-activated and Ca(2+)-inactivated channels. Simulated cells included those that respond to depolarization with tonic firing, adaptive firing, or plateau potentials as well as endogenous pacemaker and bursting cells. 4. Several types of simple neural networks were simulated that included feed-forward excitatory and inhibitory chemical synaptic connections, a network of electrically coupled cells, and a network with feedback chemical synaptic connections that simulated rhythmic neural activity. In addition, with the use of the equations describing electrical coupling, current flow in a branched neuron with 18 compartments was simulated. 5. Enhancement of excitability and enhancement of transmitter release, produced by modulatory transmitters, were simulated by second-messenger-induced modulation of K+ currents. A depletion model for synaptic depression was also simulated. 6. We also attempted to simulate the features of a more complicated central pattern generator, inspired by the properties of neurons in the buccal ganglia of Aplysia. Dynamic changes in the activity of this central pattern generator were produced by a second-messenger-induced modulation of a slow inward current in one of the neurons.

scholarly journals MEASURING CLASSIFICATION ACCURACIES USING IMPROVED THERMAL INFRARED DATA

2014 ◽  
pp. 137-143
Author(s):  
Venkateswarulu Cheruku ◽  
Sumanth Yenduri ◽  
S. S. Iyengar
Keyword(s):  
Neural Networks ◽  
Image Analysis ◽  
Classification Accuracy ◽  
Digital Image Analysis ◽  
Remotely Sensed Data ◽  
Maximum Likelihood Approach ◽  
Final Note ◽  
Thermal Data ◽  
Likelihood Approach ◽  
Infrared Data

Image classification is one of the major aspects in digital image analysis of remotely sensed data. In this paper, we present the effects on classification accuracy if improved thermal data are used instead of raw thermal data. We use two methods, Artificial Neural Networks (ANN) and Maximum Likelihood Approach (MLH) to demonstrate our purpose. Using each method different combinations of raw and improved data are tested to classify in order to compare the accuracies. As a final note, the findings are discussed.

Probability Density Estimation Using Entropy Maximization

1998 ◽  
Vol 10 (7) ◽  
pp. 1925-1938 ◽  
Author(s):  
Gad Miller ◽  
David Horn
Keyword(s):  
Neural Networks ◽  
Probability Density ◽  
Probability Density Functions ◽  
Conditional Density ◽  
Density Functions ◽  
Probability Density Estimation ◽  
Entropy Maximization ◽  
Maximum Likelihood Approach ◽  
Likelihood Approach ◽  
Stochastic Gradient Ascent

We propose a method for estimating probability density functions and conditional density functions by training on data produced by such distributions. The algorithm employs new stochastic variables that amount to coding of the input, using a principle of entropy maximization. It is shown to be closely related to the maximum likelihood approach. The encoding step of the algorithm provides an estimate of the probability distribution. The decoding step serves as a generative mode, producing an ensemble of data with the desired distribution. The algorithm is readily implemented by neural networks, using stochastic gradient ascent to achieve entropy maximization.

An Extension Approach for Neural Networks by Introducing a Nearest Neighbor Algorithm in Relative Coordinates

2010 ◽  
Vol 14 (4) ◽  
pp. 325-343 ◽  
Author(s):  
Hirofumi Suzaki ◽  
◽  
Satoru Kuhara ◽  
Keyword(s):  
Neural Networks ◽  
Computational Models ◽  
Neuron Model ◽  
Nearest Neighbor ◽  
Distributed Processing ◽  
Machine Learning Algorithms ◽  
Whole Body ◽  
Nearest Neighbor Algorithm ◽  
Relative Coordinates ◽  
Different Types

Computational models known as neural networks discriminate among different types of nonlinear data, enabling the design of flexible calculation through machine-learning algorithms. Thanks to the simplicity of calculation, the nearest neighbor algorithm is a well-studied classification method. If the nearest neighbor algorithm inference is shown by a network model consisting of a neuron model representing data, it may become deterministic with adjustable parameters. We propose a new neuron model using the generalized mean and have designed a practical neural network framework based on the nearest neighbor algorithm. Because our proposed parallel distributed processing model is not simply a distance comparison between two points, it uses information from a whole body of data. This makes our classification superior to the nearest neighbor algorithm for algorithmic principles.

On the Computational Power of Winner-Take-All

2000 ◽  
Vol 12 (11) ◽  
pp. 2519-2535 ◽  
Author(s):  
Wolfgang Maass
Keyword(s):  
Neural Networks ◽  
Theoretical Analysis ◽  
Computational Models ◽  
Continuous Functions ◽  
Weighted Sums ◽  
Computational Power ◽  
Positive Weights ◽  
Winner Take All ◽  
Nonlinear Operation ◽  
Take All

This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-takeall in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity.

Squeak and rattle noise classification using radial basis function neural networks

2020 ◽  
Vol 68 (4) ◽  
pp. 283-293
Author(s):  
Oleksandr Pogorilyi ◽  
Mohammad Fard ◽  
John Davy ◽  
Mechanical and Automotive Engineering, School ◽  
Mechanical and Automotive Engineering, School ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
High Accuracy ◽  
Training Method ◽  
Vehicle Interior ◽  
Trained Classifier ◽  
Different Types ◽  
Noise Classification ◽  
Automatic Tool ◽  
Multi Class Classification

In this article, an artificial neural network is proposed to classify short audio sequences of squeak and rattle (S&R) noises. The aim of the classification is to see how accurately the trained classifier can recognize different types of S&R sounds. Having a high accuracy model that can recognize audible S&R noises could help to build an automatic tool able to identify unpleasant vehicle interior sounds in a matter of seconds from a short audio recording of the sounds. In this article, the training method of the classifier is proposed, and the results show that the trained model can identify various classes of S&R noises: simple (binary clas- sification) and complex ones (multi class classification).

A dual U-Net algorithm for automating feature extraction from satellite imagery

2021 ◽  
pp. 154851292098354
Author(s):  
Samuel Humphries ◽  
Trevor Parker ◽  
Bryan Jonas ◽  
Bryan Adams ◽  
Nicholas J Clark
Keyword(s):  
Neural Networks ◽  
Object Detection ◽  
Satellite Images ◽  
Detection Algorithm ◽  
Military Operations ◽  
Detection Algorithms ◽  
Us Military ◽  
Standard Product ◽  
Different Types ◽  
Road Intersections

Quick identification of building and roads is critical for execution of tactical US military operations in an urban environment. To this end, a gridded, referenced, satellite images of an objective, often referred to as a gridded reference graphic or GRG, has become a standard product developed during intelligence preparation of the environment. At present, operational units identify key infrastructure by hand through the work of individual intelligence officers. Recent advances in Convolutional Neural Networks, however, allows for this process to be streamlined through the use of object detection algorithms. In this paper, we describe an object detection algorithm designed to quickly identify and label both buildings and road intersections present in an image. Our work leverages both the U-Net architecture as well the SpaceNet data corpus to produce an algorithm that accurately identifies a large breadth of buildings and different types of roads. In addition to predicting buildings and roads, our model numerically labels each building by means of a contour finding algorithm. Most importantly, the dual U-Net model is capable of predicting buildings and roads on a diverse set of test images and using these predictions to produce clean GRGs.

Sign in / Sign up

Export Citation Format

Share Document