scholarly journals Stochastic Markov gradient descent and training low-bit neural networks

2021 ◽  
Vol 19 (2) ◽  
Author(s):  
Jonathan Ashbrock ◽  
Alexander M. Powell
Keyword(s):  
Neural Networks ◽  
Gradient Descent ◽  
And Training

scholarly journals Optical Recognition of Handwritten Logic Formulas Using Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2761
Author(s):  
Vaios Ampelakiotis ◽  
Isidoros Perikos ◽  
Ioannis Hatzilygeroudis ◽  
George Tsihrintzis
Keyword(s):  
Neural Networks ◽  
Character Recognition ◽  
Gradient Descent ◽  
Feedforward Neural Networks ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Gradient Descent Algorithm ◽  
Two Stages ◽  
And Training

In this paper, we present a handwritten character recognition (HCR) system that aims to recognize first-order logic handwritten formulas and create editable text files of the recognized formulas. Dense feedforward neural networks (NNs) are utilized, and their performance is examined under various training conditions and methods. More specifically, after three training algorithms (backpropagation, resilient propagation and stochastic gradient descent) had been tested, we created and trained an NN with the stochastic gradient descent algorithm, optimized by the Adam update rule, which was proved to be the best, using a trainset of 16,750 handwritten image samples of 28 × 28 each and a testset of 7947 samples. The final accuracy achieved is 90.13%. The general methodology followed consists of two stages: the image processing and the NN design and training. Finally, an application has been created that implements the methodology and automatically recognizes handwritten logic formulas. An interesting feature of the application is that it allows for creating new, user-oriented training sets and parameter settings, and thus new NN models.

Cloning Safe Driving Behavior for Self-Driving Cars using Convolutional Neural Networks

2019 ◽  
Vol 12 (2) ◽  
pp. 120-127 ◽  
Author(s):  
Wael Farag
Keyword(s):  
Gradient Descent ◽  
Autonomous Driving ◽  
Driving Behavior ◽  
Training Data ◽  
Stochastic Gradient Descent ◽  
Data Set ◽  
Safe Driving ◽  
Processing Pipeline ◽  
Self Driving Cars ◽  
And Training

Background: In this paper, a Convolutional Neural Network (CNN) to learn safe driving behavior and smooth steering manoeuvring, is proposed as an empowerment of autonomous driving technologies. The training data is collected from a front-facing camera and the steering commands issued by an experienced driver driving in traffic as well as urban roads. Methods: This data is then used to train the proposed CNN to facilitate what it is called “Behavioral Cloning”. The proposed Behavior Cloning CNN is named as “BCNet”, and its deep seventeen-layer architecture has been selected after extensive trials. The BCNet got trained using Adam’s optimization algorithm as a variant of the Stochastic Gradient Descent (SGD) technique. Results: The paper goes through the development and training process in details and shows the image processing pipeline harnessed in the development. Conclusion: The proposed approach proved successful in cloning the driving behavior embedded in the training data set after extensive simulations.

scholarly journals Synthetic neural-like computing in microbial consortia for pattern recognition

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Ximing Li ◽  
Luna Rizik ◽  
Valeriia Kravchik ◽  
Maria Khoury ◽  
Netanel Korin ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Gradient Descent ◽  
Biological Systems ◽  
Logic Gates ◽  
Structural Similarity ◽  
Microbial Consortia ◽  
Promoter Strength ◽  
Chemical Inducers ◽  
Artificial Neural

AbstractComplex biological systems in nature comprise cells that act collectively to solve sophisticated tasks. Synthetic biological systems, in contrast, are designed for specific tasks, following computational principles including logic gates and analog design. Yet such approaches cannot be easily adapted for multiple tasks in biological contexts. Alternatively, artificial neural networks, comprised of flexible interactions for computation, support adaptive designs and are adopted for diverse applications. Here, motivated by the structural similarity between artificial neural networks and cellular networks, we implement neural-like computing in bacteria consortia for recognizing patterns. Specifically, receiver bacteria collectively interact with sender bacteria for decision-making through quorum sensing. Input patterns formed by chemical inducers activate senders to produce signaling molecules at varying levels. These levels, which act as weights, are programmed by tuning the sender promoter strength Furthermore, a gradient descent based algorithm that enables weights optimization was developed. Weights were experimentally examined for recognizing 3 × 3-bit pattern.

scholarly journals Exploring Optimized Spiking Neural Network Architectures for Classification Tasks on Embedded Platforms

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3240
Author(s):  
Tehreem Syed ◽  
Vijay Kakani ◽  
Xuenan Cui ◽  
Hakil Kim
Keyword(s):  
Neural Networks ◽  
Gradient Descent ◽  
Spiking Neural Networks ◽  
License Plate ◽  
Training Techniques ◽  
Neuromorphic Hardware ◽  
Private And Public ◽  
Embedded Platforms ◽  
Public Datasets ◽  
Event Based

In recent times, the usage of modern neuromorphic hardware for brain-inspired SNNs has grown exponentially. In the context of sparse input data, they are undertaking low power consumption for event-based neuromorphic hardware, specifically in the deeper layers. However, using deep ANNs for training spiking models is still considered as a tedious task. Until recently, various ANN to SNN conversion methods in the literature have been proposed to train deep SNN models. Nevertheless, these methods require hundreds to thousands of time-steps for training and still cannot attain good SNN performance. This work proposes a customized model (VGG, ResNet) architecture to train deep convolutional spiking neural networks. In this current study, the training is carried out using deep convolutional spiking neural networks with surrogate gradient descent backpropagation in a customized layer architecture similar to deep artificial neural networks. Moreover, this work also proposes fewer time-steps for training SNNs with surrogate gradient descent. During the training with surrogate gradient descent backpropagation, overfitting problems have been encountered. To overcome these problems, this work refines the SNN based dropout technique with surrogate gradient descent. The proposed customized SNN models achieve good classification results on both private and public datasets. In this work, several experiments have been carried out on an embedded platform (NVIDIA JETSON TX2 board), where the deployment of customized SNN models has been extensively conducted. Performance validations have been carried out in terms of processing time and inference accuracy between PC and embedded platforms, showing that the proposed customized models and training techniques are feasible for achieving a better performance on various datasets such as CIFAR-10, MNIST, SVHN, and private KITTI and Korean License plate dataset.

Sign in / Sign up

Export Citation Format

Share Document