scholarly journals BEHAVIORAL FUNCTIONS IMPLEMENTATION ON SPIKING NEURAL NETWORKS

2021 ◽  
Vol 20 (3) ◽  
pp. 591-622
Author(s):  
Anton Korsakov ◽  
Aleksandr Bakhshiev ◽  
Lyubov Astapova ◽  
Lev Stankevich
Keyword(s):  
Neural Networks ◽  
Conditioned Reflex ◽  
Negative Reinforcement ◽  
Input Pulse ◽  
Compartmental Models ◽  
Structural Adaptation ◽  
Spike Model ◽  
Current State ◽  
Proposed Model ◽  
Spiking Model

The question of behavioral functions modeling of animals (in particular, the modeling and implementation of the conditioned reflex) is considered. The analysis of the current state of neural networks with the possibility of structural reconfiguration is carried out. The modeling is carried out by means of neural networks, which are built on the basis of a compartmental spiking model of a neuron with the possibility of structural adaptation to the input pulse pattern. The compartmental spike model of a neuron is able to change its structure (the size of the cell body, the number and length of dendrites, the number of synapses) depending on the incoming pulse pattern at its inputs. A brief description of the compartmental spiking model of a neuron is given, and its main features are noted in terms of the possibility of its structural reconfiguration. The method of structural adaptation of the compartmental spiking model of the neuron to the input pulse pattern is described. To study the work of the proposed model of a neuron in a network, the choice of a conditioned reflex as a special case of the formation of associative connections is justified as an example. The structural scheme and algorithm of formation of a conditioned reflex with both positive and negative reinforcement are described. The article presents a step-by-step description of experiments on the associative connection’s formation in general and conditioned reflex (both with positive and negative reinforcement), in particular. The conclusion is made about the prospects of using spiking compartmental models of neurons to improve the efficiency of the implementation of behavioral functions in neuromorphic control systems. Further promising directions for the development of neuromorphic systems based on spiking compartmental models of the neuron are considered.

scholarly journals Neural Network Approach for Global Solar Irradiance Prediction at Extremely Short-Time-Intervals Using Particle Swarm Optimization Algorithm

Energies ◽  
2021 ◽  
Vol 14 (4) ◽  
pp. 1213
Author(s):  
Ahmed Aljanad ◽  
Nadia M. L. Tan ◽  
Vassilios G. Agelidis ◽  
Hussain Shareef
Keyword(s):  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Solar Irradiance ◽  
Particle Swarm ◽  
Time Interval ◽  
Time Intervals ◽  
Swarm Optimization ◽  
Backpropagation Neural Networks ◽  
Proposed Model ◽  
Short Time

Hourly global solar irradiance (GSR) data are required for sizing, planning, and modeling of solar photovoltaic farms. However, operating and controlling such farms exposed to varying environmental conditions, such as fast passing clouds, necessitates GSR data to be available for very short time intervals. Classical backpropagation neural networks do not perform satisfactorily when predicting parameters within short intervals. This paper proposes a hybrid backpropagation neural networks based on particle swarm optimization. The particle swarm algorithm is used as an optimization algorithm within the backpropagation neural networks to optimize the number of hidden layers and neurons used and its learning rate. The proposed model can be used as a reliable model in predicting changes in the solar irradiance during short time interval in tropical regions such as Malaysia and other regions. Actual global solar irradiance data of 5-s and 1-min intervals, recorded by weather stations, are applied to train and test the proposed algorithm. Moreover, to ensure the adaptability and robustness of the proposed technique, two different cases are evaluated using 1-day and 3-days profiles, for two different time intervals of 1-min and 5-s each. A set of statistical error indices have been introduced to evaluate the performance of the proposed algorithm. From the results obtained, the 3-days profile’s performance evaluation of the BPNN-PSO are 1.7078 of RMSE, 0.7537 of MAE, 0.0292 of MSE, and 31.4348 of MAPE (%), at 5-s time interval, where the obtained results of 1-min interval are 0.6566 of RMSE, 0.2754 of MAE, 0.0043 of MSE, and 1.4732 of MAPE (%). The results revealed that proposed model outperformed the standalone backpropagation neural networks method in predicting global solar irradiance values for extremely short-time intervals. In addition to that, the proposed model exhibited high level of predictability compared to other existing models.

scholarly journals A Review of Algorithms and Hardware Implementations for Spiking Neural Networks

2021 ◽  
Vol 11 (2) ◽  
pp. 23
Author(s):  
Duy-Anh Nguyen ◽  
Xuan-Tu Tran ◽  
Francesca Iacopi
Keyword(s):  
Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Training Algorithms ◽  
Current State ◽  
Hardware Implementations ◽  
Neuromorphic Hardware ◽  
High Level ◽  
Event Based ◽  
Hardware Platforms

Deep Learning (DL) has contributed to the success of many applications in recent years. The applications range from simple ones such as recognizing tiny images or simple speech patterns to ones with a high level of complexity such as playing the game of Go. However, this superior performance comes at a high computational cost, which made porting DL applications to conventional hardware platforms a challenging task. Many approaches have been investigated, and Spiking Neural Network (SNN) is one of the promising candidates. SNN is the third generation of Artificial Neural Networks (ANNs), where each neuron in the network uses discrete spikes to communicate in an event-based manner. SNNs have the potential advantage of achieving better energy efficiency than their ANN counterparts. While generally there will be a loss of accuracy on SNN models, new algorithms have helped to close the accuracy gap. For hardware implementations, SNNs have attracted much attention in the neuromorphic hardware research community. In this work, we review the basic background of SNNs, the current state and challenges of the training algorithms for SNNs and the current implementations of SNNs on various hardware platforms.

A Method of Environmental Sound Classification Based on Residual Networks and Data Augmentation

2021 ◽  
pp. 2150018
Author(s):  
Jinfang Zeng ◽  
Youming Li ◽  
Yu Zhang ◽  
Da Chen
Keyword(s):  
Neural Networks ◽  
Data Augmentation ◽  
Machine Learning Techniques ◽  
Deep Convolutional Neural Networks ◽  
Mel Frequency Cepstral Coefficients ◽  
Environmental Sound ◽  
Sound Classification ◽  
Learning Techniques ◽  
Proposed Model ◽  
Audio Data

Environmental sound classification (ESC) is a challenging problem due to the complexity of sounds. To date, a variety of signal processing and machine learning techniques have been applied to ESC task, including matrix factorization, dictionary learning, wavelet filterbanks and deep neural networks. It is observed that features extracted from deeper networks tend to achieve higher performance than those extracted from shallow networks. However, in ESC task, only the deep convolutional neural networks (CNNs) which contain several layers are used and the residual networks are ignored, which lead to degradation in the performance. Meanwhile, a possible explanation for the limited exploration of CNNs and the difficulty to improve on simpler models is the relative scarcity of labeled data for ESC. In this paper, a residual network called EnvResNet for the ESC task is proposed. In addition, we propose to use audio data augmentation to overcome the problem of data scarcity. The experiments will be performed on the ESC-50 database. Combined with data augmentation, the proposed model outperforms baseline implementations relying on mel-frequency cepstral coefficients and achieves results comparable to other state-of-the-art approaches in terms of classification accuracy.

scholarly journals What executive function network is that? An image-based meta-analysis of network labels

2020 ◽  
Author(s):  
Suzanne T. Witt ◽  
Helene van Ettinger-Veenstra ◽  
Taylor Salo ◽  
Michael C. Riedel ◽  
Angela R. Laird
Keyword(s):  
Neural Networks ◽  
Executive Function ◽  
Meta Analysis ◽  
Control Network ◽  
Attention Network ◽  
Frontoparietal Network ◽  
Current State ◽  
The Neural Networks ◽  
Task Positive Network ◽  
Spatial Topography

AbstractThe current state of label conventions used to describe brain networks related to executive functions is highly inconsistent, leading to confusion among researchers regarding network labels. Visually similar networks are referred to by different labels, yet these same labels are used to distinguish networks within studies. We performed a literature review of fMRI studies and identified nine frequently-used labels that are used to describe topographically or functionally similar neural networks: central executive network (CEN), cognitive control network (CCN), dorsal attention network (DAN), executive control network (ECN), executive network (EN), frontoparietal network (FPN), working memory network (WMN), task positive network (TPN), and ventral attention network (VAN). Our aim was to meta-analytically determine consistency of network topography within and across these labels. We hypothesized finding considerable overlap in the spatial topography among the neural networks associated with these labels. An image-based meta-analysis was performed on 166 individual statistical maps (SPMs) received from authors of 72 papers listed on PubMed. Our results indicated that there was very little consistency in the SPMs labeled with a given network name. We identified four clusters of SPMs representing four spatially distinct executive function networks. We provide recommendations regarding label nomenclature and propose that authors looking to assign labels to executive function networks adopt this template set for labeling networks.

scholarly journals Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax

2019 ◽  
Author(s):  
Yinfei Yang ◽  
Gustavo Hernandez Abrego ◽  
Steve Yuan ◽  
Mandy Guo ◽  
Qinlan Shen ◽  
...  
Keyword(s):  
United Nations ◽  
State Of The Art ◽  
Cosine Similarity ◽  
Retrieval Task ◽  
Parallel Corpus ◽  
Similar Performance ◽  
Second Stage ◽  
Current State ◽  
Proposed Model ◽  
Document Level

In this paper, we present an approach to learn multilingual sentence embeddings using a bi-directional dual-encoder with additive margin softmax. The embeddings are able to achieve state-of-the-art results on the United Nations (UN) parallel corpus retrieval task. In all the languages tested, the system achieves P@1 of 86% or higher. We use pairs retrieved by our approach to train NMT models that achieve similar performance to models trained on gold pairs. We explore simple document-level embeddings constructed by averaging our sentence embeddings. On the UN document-level retrieval task, document embeddings achieve around 97% on P@1 for all experimented language pairs. Lastly, we evaluate the proposed model on the BUCC mining task. The learned embeddings with raw cosine similarity scores achieve competitive results compared to current state-of-the-art models, and with a second-stage scorer we achieve a new state-of-the-art level on this task.

scholarly journals Design of Cascaded CNN for Medical Image Super Resolution

2021 ◽  
Vol 7 (3) ◽  
pp. 22-29
Author(s):  
Kajol Singh ◽  
Manish Saxena
Keyword(s):  
Neural Networks ◽  
Medical Image ◽  
Research Work ◽  
Super Resolution ◽  
Primary Objective ◽  
Lighting Conditions ◽  
Proposed Model ◽  
Filter Size ◽  
Medical Image Reconstruction ◽  
Image Super Resolution

The images captured through a camera usually belong to over or under exposed conditions. The reason may be inappropriate lighting conditions or camera resolution. Hence, it is of utmost importance to have a few enhancement techniques that could make these artefacts look better. Hence, the primary objective pertaining to the adjustment and enhancement techniques is to enhance the characteristics of an image. The initial numeric values related to an image get distorted when an image is enhanced. Therefore, enhancement techniques should be designed in such a way that the image quality isn’t compromised. This research work is focused on proposed a network design for deep convolution neural networks for application of super resolution techniques. To improve the complexity of existing techniques this work is intended towards network designs, different filter size and CNN architecture. The CNN model is most effective model for detection and segmentation in image. This model will improve the efficiency of medical image reconstruction from LR to HR. The proposed model showed its efficiency not only PET medical images but also on retinal database and achieved advance results as compared to existing works.

scholarly journals Stock Pattern Classification from Charts using Deep Learning Algorithms

2020 ◽  
Vol 3 (1) ◽  
pp. 445-454
Author(s):  
Celal Buğra Kaya ◽  
Alperen Yılmaz ◽  
Gizem Nur Uzun ◽  
Zeynep Hilal Kilimci
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Pattern Classification ◽  
Short Term Memory ◽  
Stock Exchange ◽  
Learning Techniques ◽  
Proposed Model ◽  
Istanbul Stock Exchange

Pattern classification is related with the automatic finding of regularities in dataset through the utilization of various learning techniques. Thus, the classification of the objects into a set of categories or classes is provided. This study is undertaken to evaluate deep learning methodologies to the classification of stock patterns. In order to classify patterns that are obtained from stock charts, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long-short term memory networks (LSTMs) are employed. To demonstrate the efficiency of proposed model in categorizing patterns, hand-crafted image dataset is constructed from stock charts in Istanbul Stock Exchange and NASDAQ Stock Exchange. Experimental results show that the usage of convolutional neural networks exhibits superior classification success in recognizing patterns compared to the other deep learning methodologies.

scholarly journals Soft Threshold Ternary Networks

2020 ◽  
Author(s):  
Weixiang Xu ◽  
Xiangyu He ◽  
Tianli Zhao ◽  
Qinghao Hu ◽  
Peisong Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Mobile Devices ◽  
State Of The Art ◽  
Performance Gap ◽  
Training Time ◽  
Current State ◽  
The Arts ◽  
Soft Threshold ◽  
And Storage ◽  
Selection Of

Large neural networks are difficult to deploy on mobile devices because of intensive computation and storage. To alleviate it, we study ternarization, a balance between efficiency and accuracy that quantizes both weights and activations into ternary values. In previous ternarized neural networks, a hard threshold Δ is introduced to determine quantization intervals. Although the selection of Δ greatly affects the training results, previous works estimate Δ via an approximation or treat it as a hyper-parameter, which is suboptimal. In this paper, we present the Soft Threshold Ternary Networks (STTN), which enables the model to automatically determine quantization intervals instead of depending on a hard threshold. Concretely, we replace the original ternary kernel with the addition of two binary kernels at training time, where ternary values are determined by the combination of two corresponding binary values. At inference time, we add up the two binary kernels to obtain a single ternary kernel. Our method dramatically outperforms current state-of-the-arts, lowering the performance gap between full-precision networks and extreme low bit networks. Experiments on ImageNet with AlexNet (Top-1 55.6%), ResNet-18 (Top-1 66.2%) achieves new state-of-the-art.

scholarly journals AI-driven deep CNN approach for multi-label pathology classification using chest X-Rays

2021 ◽  
Vol 7 ◽  
pp. e495
Author(s):  
Saleh Albahli ◽  
Hafiz Tayyab Rauf ◽  
Abdulelah Algosaibi ◽  
Valentina Emilia Balas
Keyword(s):  
Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Synthetic Data ◽  
X Rays ◽  
Deep Convolutional Neural Networks ◽  
Current State ◽  
Pathology Classification ◽  
Wide Range ◽  
Multi Class Classification

Artificial intelligence (AI) has played a significant role in image analysis and feature extraction, applied to detect and diagnose a wide range of chest-related diseases. Although several researchers have used current state-of-the-art approaches and have produced impressive chest-related clinical outcomes, specific techniques may not contribute many advantages if one type of disease is detected without the rest being identified. Those who tried to identify multiple chest-related diseases were ineffective due to insufficient data and the available data not being balanced. This research provides a significant contribution to the healthcare industry and the research community by proposing a synthetic data augmentation in three deep Convolutional Neural Networks (CNNs) architectures for the detection of 14 chest-related diseases. The employed models are DenseNet121, InceptionResNetV2, and ResNet152V2; after training and validation, an average ROC-AUC score of 0.80 was obtained competitive as compared to the previous models that were trained for multi-class classification to detect anomalies in x-ray images. This research illustrates how the proposed model practices state-of-the-art deep neural networks to classify 14 chest-related diseases with better accuracy.

scholarly journals Switching dynamics of single and coupled VO2-based oscillators as elements of neural networks

2017 ◽  
Vol 31 (02) ◽  
pp. 1650261 ◽  
Author(s):  
Andrey Velichko ◽  
Maksim Belyaev ◽  
Vadim Putrolaynen ◽  
Alexander Pergament ◽  
Valentin Perminov
Keyword(s):  
Neural Networks ◽  
Strong Coupling ◽  
Capacitive Coupling ◽  
Chaotic Oscillations ◽  
Physical Mechanisms ◽  
Coupling Capacitance ◽  
Switching Dynamics ◽  
Proposed Model ◽  
Coupling Mode ◽  
Phase Differences

In the present paper, we report on the switching dynamics of both single and coupled VO2-based oscillators, with resistive and capacitive coupling, and explore the capability of their application in oscillatory neural networks. Based on these results, we further select an adequate SPICE model to describe the modes of operation of coupled oscillator circuits. Physical mechanisms influencing the time of forward and reverse electrical switching, that determine the applicability limits of the proposed model, are identified. For the resistive coupling, it is shown that synchronization takes place at a certain value of the coupling resistance, though it is unstable and a synchronization failure occurs periodically. For the capacitive coupling, two synchronization modes, with weak and strong coupling, are found. The transition between these modes is accompanied by chaotic oscillations. A decrease in the width of the spectrum harmonics in the weak-coupling mode, and its increase in the strong-coupling one, is detected. The dependences of frequencies and phase differences of the coupled oscillatory circuits on the coupling capacitance are found. Examples of operation of coupled VO2 oscillators as a central pattern generator are demonstrated.

Sign in / Sign up

Export Citation Format

Share Document