Temporal Coding in Realistic Neural Networks

1995 ◽  
Vol 5 (10) ◽  
pp. 1367-1374
Author(s):  
S. M. Gerasyuta ◽  
D. V. Ivanov
Keyword(s):  
Neural Networks ◽  
Temporal Coding

Feature-Linking Model for Image Enhancement

2016 ◽  
Vol 28 (6) ◽  
pp. 1072-1100 ◽  
Author(s):  
Kun Zhan ◽  
Jicai Teng ◽  
Jinhui Shi ◽  
Qiaoqiao Li ◽  
Mingying Wang
Keyword(s):  
Neural Networks ◽  
Image Enhancement ◽  
Human Visual System ◽  
Input Image ◽  
Temporal Coding ◽  
Essential Dimension ◽  
Neural Representations ◽  
Gamma Band Oscillations ◽  
Processing Mechanisms ◽  
Enhancement Algorithm

Inspired by gamma-band oscillations and other neurobiological discoveries, neural networks research shifts the emphasis toward temporal coding, which uses explicit times at which spikes occur as an essential dimension in neural representations. We present a feature-linking model (FLM) that uses the timing of spikes to encode information. The first spiking time of FLM is applied to image enhancement, and the processing mechanisms are consistent with the human visual system. The enhancement algorithm achieves boosting the details while preserving the information of the input image. Experiments are conducted to demonstrate the effectiveness of the proposed method. Results show that the proposed method is effective.

Temporal coding in neural networks

1993 ◽  
Author(s):  
William E. Faller ◽  
Scott J. Schreck ◽  
M. W. Luttges
Keyword(s):  
Neural Networks ◽  
Temporal Coding

Design and Implementation of a Spiking Neural Network with Integrate-and-Fire Neuron Model for Pattern Recognition

2020 ◽  
pp. 2050073
Author(s):  
Parvaneh Rashvand ◽  
Mohammad Reza Ahmadzadeh ◽  
Farzaneh Shayegh
Keyword(s):  
Neural Networks ◽  
Neuron Model ◽  
Learning Algorithm ◽  
Algorithm Design ◽  
Receptive Fields ◽  
Structural Features ◽  
Temporal Coding ◽  
Neuron Models ◽  
Integrate And Fire ◽  
Encoding Method

In contrast to the previous artificial neural networks (ANNs), spiking neural networks (SNNs) work based on temporal coding approaches. In the proposed SNN, the number of neurons, neuron models, encoding method, and learning algorithm design are described in a correct and pellucid fashion. It is also discussed that optimizing the SNN parameters based on physiology, and maximizing the information they pass leads to a more robust network. In this paper, inspired by the “center-surround” structure of the receptive fields in the retina, and the amount of overlap that they have, a robust SNN is implemented. It is based on the Integrate-and-Fire (IF) neuron model and uses the time-to-first-spike coding to train the network by a newly proposed method. The Iris and MNIST datasets were employed to evaluate the performance of the proposed network whose accuracy, with 60 input neurons, was 96.33% on the Iris dataset. The network was trained in only 45 iterations indicating its reasonable convergence rate. For the MNIST dataset, when the gray level of each pixel was considered as input to the network, 600 input neurons were required, and the accuracy of the network was 90.5%. Next, 14 structural features were used as input. Therefore, the number of input neurons decreased to 210, and accuracy increased up to 95%, meaning that an SNN with fewer input neurons and good skill was implemented. Also, the ABIDE1 dataset is applied to the proposed SNN. Of the 184 data, 79 are used for healthy people and 105 for people with autism. One of the characteristics that can differentiate between these two classes is the entropy of the existing data. Therefore, Shannon entropy is used for feature extraction. Applying these values to the proposed SNN, an accuracy of 84.42% was achieved by only 120 iterations, which is a good result compared to the recent results.

scholarly journals TDSNN: From Deep Neural Networks to Deep Spike Neural Networks with Temporal-Coding

2019 ◽  
Vol 33 ◽  
pp. 1319-1326 ◽  
Author(s):  
Lei Zhang ◽  
Shengyuan Zhou ◽  
Tian Zhi ◽  
Zidong Du ◽  
Yunji Chen
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Neural Model ◽  
Temporal Coding ◽  
Energy Costs ◽  
Large Networks ◽  
Convolutional Networks ◽  
Rate Coding ◽  
Novel Method ◽  
Embedded Applications

Continuous-valued deep convolutional networks (DNNs) can be converted into accurate rate-coding based spike neural networks (SNNs). However, the substantial computational and energy costs, which is caused by multiple spikes, limit their use in mobile and embedded applications. And recent works have shown that the newly emerged temporal-coding based SNNs converted from DNNs can reduce the computational load effectively. In this paper, we propose a novel method to convert DNNs to temporal-coding SNNs, called TDSNN. Combined with the characteristic of the leaky integrate-andfire (LIF) neural model, we put forward a new coding principle Reverse Coding and design a novel Ticking Neuron mechanism. According to our evaluation, our proposed method achieves 42% total operations reduction on average in large networks comparing with DNNs with no more than 0.5% accuracy loss. The evaluation shows that TDSNN may prove to be one of the key enablers to make the adoption of SNNs widespread.

scholarly journals Spiking Autoencoders With Temporal Coding

2021 ◽  
Vol 15 ◽  
Author(s):  
Iulia-Maria Comşa ◽  
Luca Versari ◽  
Thomas Fischbacher ◽  
Jyrki Alakuijala
Keyword(s):  
Neural Networks ◽  
Single Layer ◽  
Building Blocks ◽  
Classification Performance ◽  
Temporal Coding ◽  
Negative Evidence ◽  
Coding Schemes ◽  
Learning Tasks ◽  
Compact Representations ◽  
Output Spike

Spiking neural networks with temporal coding schemes process information based on the relative timing of neuronal spikes. In supervised learning tasks, temporal coding allows learning through backpropagation with exact derivatives, and achieves accuracies on par with conventional artificial neural networks. Here we introduce spiking autoencoders with temporal coding and pulses, trained using backpropagation to store and reconstruct images with high fidelity from compact representations. We show that spiking autoencoders with a single layer are able to effectively represent and reconstruct images from the neuromorphically-encoded MNIST and FMNIST datasets. We explore the effect of different spike time target latencies, data noise levels and embedding sizes, as well as the classification performance from the embeddings. The spiking autoencoders achieve results similar to or better than conventional non-spiking autoencoders. We find that inhibition is essential in the functioning of the spiking autoencoders, particularly when the input needs to be memorised for a longer time before the expected output spike times. To reconstruct images with a high target latency, the network learns to accumulate negative evidence and to use the pulses as excitatory triggers for producing the output spikes at the required times. Our results highlight the potential of spiking autoencoders as building blocks for more complex biologically-inspired architectures. We also provide open-source code for the model.

A temporal coding hardware implementation for spiking neural networks

2010 ◽  
Vol 38 (4) ◽  
pp. 2-7 ◽  
Author(s):  
Marco Nuño-Maganda ◽  
Cesar Torres-Huitzil
Keyword(s):  
Neural Networks ◽  
Hardware Implementation ◽  
Temporal Coding ◽  
Spiking Neural Networks

scholarly journals Optimization of Spiking Neural Networks Based on Binary Streamed Rate Coding

Electronics ◽  
2020 ◽  
Vol 9 (10) ◽  
pp. 1599
Author(s):  
Ali A. Al-Hamid ◽  
HyungWon Kim
Keyword(s):  
Neural Networks ◽  
Hardware Implementation ◽  
Structure Optimization ◽  
Optimization Methods ◽  
Optimization Method ◽  
Neural System ◽  
Temporal Coding ◽  
Spiking Neural Networks ◽  
Coding Scheme ◽  
Rate Coding

Spiking neural networks (SNN) increasingly attract attention for their similarity to the biological neural system. Hardware implementation of spiking neural networks, however, remains a great challenge due to their excessive complexity and circuit size. This work introduces a novel optimization method for hardware friendly SNN architecture based on a modified rate coding scheme called Binary Streamed Rate Coding (BSRC). BSRC combines the features of both rate and temporal coding. In addition, by employing a built-in randomizer, the BSRC SNN model provides a higher accuracy and faster training. We also present SNN optimization methods including structure optimization and weight quantization. Extensive evaluations with MNIST SNNs demonstrate that the structure optimization of SNN (81-30-20-10) provides 183.19 times reduction in hardware compared with SNN (784-800-10), while providing an accuracy of 95.25%, a small loss compared with 98.89% and 98.93% reported in the previous works. Our weight quantization reduces 32-bit weights to 4-bit integers leading to further hardware reduction of 4 times with only 0.56% accuracy loss. Overall, the SNN model (81-30-20-10) optimized by our method shrinks the SNN’s circuit area from 3089.49 mm2 for SNN (784-800-10) to 4.04 mm2—a reduction of 765 times.

Additional tests of Amit's attractor neural networks

1995 ◽  
Vol 18 (4) ◽  
pp. 634-635
Author(s):  
Ralph E. Hoffman
Keyword(s):  
Neural Networks ◽  
False Positive ◽  
Biological Systems ◽  
Temporal Coding ◽  
Attractor Networks ◽  
Memory Overload

AbstractFurther tests of Amit's model are indicated. One strategy is to use the apparent coding sparseness of the model to make predictions about coding sparseness in Miyashita's network. A second approach is to use memory overload to induce false positive responses in modules and biological systems. In closing, the importance of temporal coding and timing requirements in developing biologically plausible attractor networks is mentioned.

Temporal coding in realistic neural networks

2002 ◽  
Author(s):  
S.M. Gerasyuta ◽  
D.V. Ivanov
Keyword(s):  
Neural Networks ◽  
Temporal Coding
Sign in / Sign up

Export Citation Format

Share Document