A Low-Power Spike-Like Neural Network Design

Modern massively-parallel Graphics Processing Units (GPUs) and Machine Learning (ML) frameworks enable neural network implementations of unprecedented performance and sophistication. However, state-of-the-art GPU hardware platforms are extremely power-hungry, while microprocessors cannot achieve the performance requirements. Biologically-inspired Spiking Neural Networks (SNN) have inherent characteristics that lead to lower power consumption. We thus present a bit-serial SNN-like hardware architecture. By using counters, comparators, and an indexing scheme, the design effectively implements the sum-of-products inherent in neurons. In addition, we experimented with various strength-reduction methods to lower neural network resource usage. The proposed Spiking Hybrid Network (SHiNe), validated on an FPGA, has been found to achieve reasonable performance with a low resource utilization, with some trade-off with respect to hardware throughput and signal representation.

Download Full-text

Real-time simulation of a spiking neural network model of the basal ganglia circuitry using general purpose computing on graphics processing units

Neural Networks ◽

10.1016/j.neunet.2011.06.008 ◽

2011 ◽

Vol 24 (9) ◽

pp. 950-960 ◽

Cited By ~ 26

Author(s):

Jun Igarashi ◽

Osamu Shouno ◽

Tomoki Fukai ◽

Hiroshi Tsujino

Keyword(s):

Neural Network ◽

Basal Ganglia ◽

Real Time ◽

Neural Network Model ◽

Graphics Processing Units ◽

General Purpose ◽

Spiking Neural Network ◽

Time Simulation ◽

Graphics Processing ◽

Spiking Neural Network Model

Download Full-text

Autotuning Parallel Programs by Model Checking

Modeling and Analysis of Information Systems ◽

10.18255/1818-1015-2021-4-338-355 ◽

2021 ◽

Vol 28 (4) ◽

pp. 338-355

Author(s):

Natalia Olegovna Garanina ◽

Sergei Petrovich Gorlatch

Keyword(s):

Model Checking ◽

Graphics Processing Units ◽

Formal Semantics ◽

Parallel Programs ◽

Tuning Parameters ◽

Counter Example ◽

Optimality Property ◽

Execution Model ◽

Parallel Graphics ◽

Graphics Processing

The paper presents a new approach to autotuning data-parallel programs. Autotuning is a search for optimal program settings which maximize its performance. The novelty of the approach lies in the use of the model checking method to find the optimal tuning parameters by the method of counterexamples. In our work, we abstract from specific programs and specific processors by defining their representative abstract patterns. Our method of counterexamples implements the following four steps. At the first step, an execution model of an abstract program on an abstract processor is described in the language of a model checking tool. At the second step, in the language of the model checking tool, we formulate the optimality property that depends on the constructed model. At the third step, we find the optimal values of the tuning parameters by using a counterexample constructed during the verification of the optimality property. In the fourth step, we extract the information about the tuning parameters from the counter-example for the optimal parameters. We apply this approach to autotuning parallel programs written in OpenCL, a popular modern language that extends the C language for programming both standard multi-core processors (CPUs) and massively parallel graphics processing units (GPUs). As a verification tool, we use the SPIN verifier and its model representation language Promela, whose formal semantics is good for modelling the execution of parallel programs on processors with different architectures.

Download Full-text

TRAINING NEURAL NETWORK FOR TAXI PASSENGER DEMAND FORECASTING USING GRAPHICS PROCESSING UNITS

Ukrainian Journal of Information Technology ◽

10.23939/ujit2020.02.029 ◽

2020 ◽

Vol 2 (1) ◽

pp. 29-36

Author(s):

M. I. Zghoba ◽

◽

Yu. I. Hrytsiuk ◽

Keyword(s):

Neural Network ◽

Graphics Processing Units ◽

Neural Network Training ◽

Training Time ◽

Passenger Demand ◽

Network Training ◽

The Neural Network ◽

Input Dataset ◽

Speed Up ◽

Graphics Processing

The peculiarities of neural network training for forecasting taxi passenger demand using graphics processing units are considered, which allowed to speed up the training procedure for different sets of input data, hardware configurations, and its power. It has been found that taxi services are becoming more accessible to a wide range of people. The most important task for any transportation company and taxi driver is to minimize the waiting time for new orders and to minimize the distance from drivers to passengers on order receiving. Understanding and assessing the geographical passenger demand that depends on many factors is crucial to achieve this goal. This paper describes an example of neural network training for predicting taxi passenger demand. It shows the importance of a large input dataset for the accuracy of the neural network. Since the training of a neural network is a lengthy process, parallel training was used to speed up the training. The neural network for forecasting taxi passenger demand was trained using different hardware configurations, such as one CPU, one GPU, and two GPUs. The training times of one epoch were compared along with these configurations. The impact of different hardware configurations on training time was analyzed in this work. The network was trained using a dataset containing 4.5 million trips within one city. The results of this study show that the training with GPU accelerators doesn't necessarily improve the training time. The training time depends on many factors, such as input dataset size, splitting of the entire dataset into smaller subsets, as well as hardware and power characteristics.

Download Full-text