scholarly journals New Efficient Multi-Spike Learning for Fast Processing and Robust Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 4650-4657
Author(s):  
Shenglan Li ◽  
Qiang Yu
Keyword(s):  
Neural Networks ◽  
Single Neuron ◽  
Learning Methods ◽  
Processing Efficiency ◽  
Robust Learning ◽  
Learning Rules ◽  
Challenging Tasks ◽  
Fast Processing ◽  
Classification Tasks ◽  
Neuron Dynamics

Spiking neural networks (SNNs) are considered to be more biologically plausible and lower power consuming than traditional artificial neural networks (ANNs). SNNs use discrete spikes as input and output, but how to process and learn these discrete spikes efficiently and accurately still remains a challenging task. Moreover, most existing learning methods are inefficient with complicated neuron dynamics and learning procedures being involved. In this paper, we propose efficient alternatives by firstly introducing a simplified and efficient neuron model. Based on it, we develop two new multi-spike learning rules together with an event-driven scheme being presented to improve the processing efficiency. We show that, with the as-proposed rules, a single neuron can be trained to successfully perform challenging tasks such as multi-category classification and feature extraction. Our learning methods demonstrate a significant robustness against various strong noises. Moreover, experimental results on some real-world classification tasks show that our approaches yield higher efficiency with less requirement on computation resource, highlighting the advantages and potential of spike-based processing and driving more efforts towards neuromorphic computing.

scholarly journals A comparison of conventional and deep learning methods of image classification

2021 ◽  
Vol 21 ◽  
pp. 303-308
Author(s):  
Maryna Dovbnych ◽  
Małgorzata Plechawska–Wójcik
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Image Classification ◽  
Classification Performance ◽  
Multi Layer Perceptron ◽  
Learning Methods ◽  
Training Outcome ◽  
Performance Training ◽  
Classification Tasks ◽  
Research Experiment

The aim of the research is to compare traditional and deep learning methods in image classification tasks. The conducted research experiment covers the analysis of five different models of neural networks: two models of multi–layer perceptron architecture: MLP with two hidden layers, MLP with three hidden layers; and three models of convolutional architecture: the three VGG blocks model, AlexNet and GoogLeNet. The models were tested on two different datasets: CIFAR–10 and MNIST and have been applied to the task of image classification. They were tested for classification performance, training speed, and the effect of the complexity of the dataset on the training outcome.

scholarly journals Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Florian Stelzer ◽  
André Röhm ◽  
Raul Vicente ◽  
Ingo Fischer ◽  
Serhiy Yanchuk
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Single Neuron ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Local Network ◽  
Multiple Time ◽  
Learning Tools ◽  
Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

scholarly journals Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 223
Author(s):  
Yen-Ling Tai ◽  
Shin-Jhe Huang ◽  
Chien-Chang Chen ◽  
Henry Horng-Shing Lu
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Computational Complexity ◽  
High Performance ◽  
Low Cost ◽  
Structural Complexity ◽  
Correction Function ◽  
Computational Time ◽  
Learning Methods ◽  
Band Theory

Nowadays, deep learning methods with high structural complexity and flexibility inevitably lean on the computational capability of the hardware. A platform with high-performance GPUs and large amounts of memory could support neural networks having large numbers of layers and kernels. However, naively pursuing high-cost hardware would probably drag the technical development of deep learning methods. In the article, we thus establish a new preprocessing method to reduce the computational complexity of the neural networks. Inspired by the band theory of solids in physics, we map the image space into a noninteraction physical system isomorphically and then treat image voxels as particle-like clusters. Then, we reconstruct the Fermi–Dirac distribution to be a correction function for the normalization of the voxel intensity and as a filter of insignificant cluster components. The filtered clusters at the circumstance can delineate the morphological heterogeneity of the image voxels. We used the BraTS 2019 datasets and the dimensional fusion U-net for the algorithmic validation, and the proposed Fermi–Dirac correction function exhibited comparable performance to other employed preprocessing methods. By comparing to the conventional z-score normalization function and the Gamma correction function, the proposed algorithm can save at least 38% of computational time cost under a low-cost hardware architecture. Even though the correction function of global histogram equalization has the lowest computational time among the employed correction functions, the proposed Fermi–Dirac correction function exhibits better capabilities of image augmentation and segmentation.

Nonlinear Resonance in Neuron Dynamics

1995 ◽  
Vol 50 (8) ◽  
pp. 718-726 ◽  
Author(s):  
Scott Rader ◽  
Diek W. Wheeler ◽  
W.C. Schieve ◽  
Pranab Das
Keyword(s):  
Neural Networks ◽  
Energy Transfer ◽  
Power Spectrum ◽  
Nonlinear Resonance ◽  
Nonlinear Oscillators ◽  
Hopfield Neural Networks ◽  
Neuron Dynamics

Abstract Hübler's technique using aperiodic forces to drive nonlinear oscillators to resonance is analyzed. The oscillators being examined are effective neurons that model Hopfield neural networks. The method is shown to be valid under several different circumstances. It is verified through analysis of the power spectrum, force, resonance, and energy transfer of the system.

Human Skin Detection in Color Images Using Deep Learning

2015 ◽  
Vol 5 (2) ◽  
pp. 1-13 ◽  
Author(s):  
Mohammadreza Hajiarbabi ◽  
Arvin Agah
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Human Skin ◽  
Skin Color ◽  
Color Image ◽  
Gaussian Model ◽  
Color Images ◽  
Skin Detection ◽  
Learning Methods ◽  
Rule Based

Human skin detection is an important and challenging problem in computer vision. Skin detection can be used as the first phase in face detection when using color images. The differences in illumination and ranges of skin colors have made skin detection a challenging task. Gaussian model, rule based methods, and artificial neural networks are methods that have been used for human skin color detection. Deep learning methods are new techniques in learning that have shown improved classification power compared to neural networks. In this paper the authors use deep learning methods in order to enhance the capabilities of skin detection algorithms. Several experiments have been performed using auto encoders and different color spaces. The proposed technique is evaluated compare with other available methods in this domain using two color image databases. The results show that skin detection utilizing deep learning has better results compared to other methods such as rule-based, Gaussian model and feed forward neural network.

scholarly journals Deep Scattering Spectra with Deep Neural Networks for Acoustic Scene Classification Tasks

2019 ◽  
Vol 28 (6) ◽  
pp. 1177-1183
Author(s):  
Pengyuan Zhang ◽  
Hangting Chen ◽  
Haichuan Bai ◽  
Qingsheng Yuan
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Scene Classification ◽  
Scattering Spectra ◽  
Classification Tasks
Sign in / Sign up

Export Citation Format

Share Document