scholarly journals Compact yet efficient hardware architecture for multilayer-perceptron neural networks

2011 ◽  
Vol 22 (6) ◽  
pp. 647-663 ◽  
Author(s):  
Rodrigo Martins da Silva ◽  
Luiza de Macedo Mourelle ◽  
Nadia Nedjah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Multilayer Perceptron ◽  
Resource Sharing ◽  
Activation Function ◽  
Hardware Architecture ◽  
Weighted Sum ◽  
Combinational Circuits ◽  
Number Of Layers ◽  
Artificial Neural Network Ann

There are several neural network implementations using either software, hardware-based or a hardware/software co-design. This work proposes a hardware architecture to implement an artificial neural network (ANN), whose topology is the multilayer perceptron (MLP). In this paper, we explore the parallelism of neural networks and allow on-thefly changes of the number of inputs, number of layers and number of neurons per layer of the net. This reconfigurability characteristic permits that any application of ANNs may be implemented using the proposed hardware. In order to reduce the processing time that is spent in arithmetic computation, a real number is represented using a fraction of integers. In this way, the arithmetics is limited to integer operations, performed by fast combinational circuits. A simple state machine is required to control sums and products of fractions. Sigmoid is used as the activation function in the proposed implementation. It is approximated by polynomials, whose underlying computation requires only sums and products. A theorem is introduced and proven so as to cover the arithmetic strategy of the computation of the activation function. Thus, the arithmetic circuitry used to implement the neuron weighted sum is reused for computing the sigmoid. this resource sharing decreased drastically the total area of the system. After modeling and simulation for functionality validation, the proposed architecture synthesized using reconfigurable hardware. The results are promising.

scholarly journals Stacked Heterogeneous Neural Networks for Time Series Forecasting

2010 ◽  
Vol 2010 ◽  
pp. 1-20 ◽  
Author(s):  
Florin Leon ◽  
Mihai Horia Zaharia
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Case Studies ◽  
Multilayer Perceptron ◽  
Neural Model ◽  
Activation Function ◽  
Time Series Forecasting ◽  
The Other ◽  
Activation Functions

A hybrid model for time series forecasting is proposed. It is a stacked neural network, containing one normal multilayer perceptron with bipolar sigmoid activation functions, and the other with an exponential activation function in the output layer. As shown by the case studies, the proposed stacked hybrid neural model performs well on a variety of benchmark time series. The combination of weights of the two stack components that leads to optimal performance is also studied.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

scholarly journals Determination of the Factors Affecting King Abdul Aziz University Published Articles in ISI by Multilayer Perceptron Artificial Neural Network

Mathematics ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. 766
Author(s):  
Rashad A. R. Bantan ◽  
Ramadan A. Zeineldin ◽  
Farrukh Jamal ◽  
Christophe Chesneau
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Multilayer Perceptron ◽  
Research Study ◽  
Real Data ◽  
Factors Affecting ◽  
King Abdulaziz University ◽  
Artificial Neural ◽  
Artificial Neural Network Ann

Deanship of scientific research established by the King Abdulaziz University provides some research programs for its staff and researchers and encourages them to submit proposals in this regard. Distinct research study (DRS) is one of these programs. It is available all the year and the King Abdulaziz University (KAU) staff can submit more than one proposal at the same time up to three proposals. The rules of the DSR program are simple and easy so it contributes in increasing the international rank of KAU. The authors are offered financial and moral reward after publishing articles from these proposals in Thomson-ISI journals. In this paper, multiplayer perceptron (MLP) artificial neural network (ANN) is employed to determine the factors that have more effect on the number of ISI published articles. The proposed study used real data of the finished projects from 2011 to April 2019.

Deep neural network based on generalized neo-fuzzy neurons and its learning based on backpropagation

2021 ◽  
Vol 26 (jai2021.26(1)) ◽  
pp. 32-41
Author(s):  
Bodyanskiy Y ◽  
◽  
Antonenko T ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Process ◽  
Activation Function ◽  
Point Of View ◽  
Basic Unit ◽  
Theoretical Point ◽  
Activation Functions ◽  
Approximation Properties ◽  
Network Training

Modern approaches in deep neural networks have a number of issues related to the learning process and computational costs. This article considers the architecture grounded on an alternative approach to the basic unit of the neural network. This approach achieves optimization in the calculations and gives rise to an alternative way to solve the problems of the vanishing and exploding gradient. The main issue of the article is the usage of the deep stacked neo-fuzzy system, which uses a generalized neo-fuzzy neuron to optimize the learning process. This approach is non-standard from a theoretical point of view, so the paper presents the necessary mathematical calculations and describes all the intricacies of using this architecture from a practical point of view. From a theoretical point, the network learning process is fully disclosed. Derived all necessary calculations for the use of the backpropagation algorithm for network training. A feature of the network is the rapid calculation of the derivative for the activation functions of neurons. This is achieved through the use of fuzzy membership functions. The paper shows that the derivative of such function is a constant, and this is a reason for the statement of increasing in the optimization rate in comparison with neural networks which use neurons with more common activation functions (ReLU, sigmoid). The paper highlights the main points that can be improved in further theoretical developments on this topic. In general, these issues are related to the calculation of the activation function. The proposed methods cope with these points and allow approximation using the network, but the authors already have theoretical justifications for improving the speed and approximation properties of the network. The results of the comparison of the proposed network with standard neural network architectures are shown

scholarly journals Prediction of the chrome distribution in subarctic Noyabrsk using co-kriging, generalized regression neural network, multilayer perceptron, and hybrid technics

2019 ◽  
pp. 77-86
Author(s):  
A. G. Buevich ◽  
I. E. Subbotina ◽  
A. V. Shichkin ◽  
A. P. Sergeev ◽  
E. M. Baglaeva
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Multilayer Perceptron ◽  
Surface Soil ◽  
Soil Layer ◽  
Generalized Regression Neural Network ◽  
Residual Kriging ◽  
Surface Soil Layer ◽  
Artificial Neural ◽  
Generalized Regression

Combination of geostatistical interpolation (kriging) and machine learning (artificial neural networks, ANN) methods leads to an increase in the accuracy of forecasting. The paper considers the application of residual kriging of an artificial neural network to predicting the spatial contamination of the surface soil layer with chromium (Cr). We reviewed and compared two neural networks: the generalized regression neural network (GRNN) and multilayer perceptron (MLP), as well as the combined method: multilayer perceptron residual kriging (MLPRK). The study is based on the results of the screening of the surface soil layer in the subarctic Noyabrsk, Russia. The models are developed based on computer modeling with minimization of the RMSE. The MLPRK model showed the best prognostic accuracy.

Power Forecasting from Solar Panels Using Artificial Neural Network in UTHM Parit Raja

2021 ◽  
Vol 02 (01) ◽  
Author(s):  
Natasha Munirah Mohd Fahmi ◽  
◽  
Nor Aira Zambri ◽  
Norhafiz Salim ◽  
Sim Sy Yi ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Climatic Factors ◽  
Activation Function ◽  
Solar Panels ◽  
Ann Model ◽  
Power Characteristics ◽  
Step Procedure ◽  
Artificial Neural ◽  
Artificial Neural Network Ann

This paper presents a step-by-step procedure for the simulation of photovoltaic modules with numerical values, using MALTAB/Simulink software. The proposed model is developed based on the mathematical model of PV module, which based on PV solar cell employing one-diode equivalent circuit. The output current and power characteristics curves highly depend on some climatic factors such as radiation and temperature, are obtained by simulation of the selected module. The collected data are used in developing Artificial Neural Network (ANN) model. Multilayer Perceptron (MLP) and Radial Basis Function (RBF) are the techniques used to forecast the outputs of the PV. Various types of activation function will be applied such as Linear, Logistic Sigmoid, Hyperbolic Tangent Sigmoid and Gaussian. The simulation results show that the Logistic Sigmoid is the best technique which produce minimal root mean square error for the system.

Classification of Coronary Artery Disease Using Multilayer Perceptron Neural Network

2021 ◽  
Vol 12 (3) ◽  
pp. 35-43
Author(s):  
Pratibha Verma ◽  
Vineet Kumar Awasthi ◽  
Sanat Kumar Sahu
Keyword(s):  
Neural Network ◽  
Coronary Artery Disease ◽  
Artificial Neural Network ◽  
Coronary Artery ◽  
Multilayer Perceptron ◽  
Artificial Neural ◽  
Artery Disease ◽  
Artificial Neural Network Ann ◽  
Hidden Layer

Coronary artery disease (CAD) has been the leading cause of death worldwide over the past 10 years. Researchers have been using several data mining techniques to help healthcare professionals diagnose heart disease. The neural network (NN) can provide an excellent solution to identify and classify different diseases. The artificial neural network (ANN) methods play an essential role in recognizes diseases in the CAD. The authors proposed multilayer perceptron neural network (MLPNN) among one hidden layer neuron (MLP) and four hidden layers neurons (P-MLP)-based highly accurate artificial neural network (ANN) method for the classification of the CAD dataset. Therefore, the ten-fold cross-validation (T-FCV) method, P-MLP algorithms, and base classifiers of MLP were employed. The P-MLP algorithm yielded very high accuracy (86.47% in CAD-56 and 98.35% in CAD-59 datasets) and F1-Score (90.36% in CAD-56 and 98.83% in CAD-59 datasets) rates, which have not been reported simultaneously in the MLP.

Signal processing algorithm for neural networks with integrodifferential splines as an activation function and its particular case of image classification

2021 ◽  
Author(s):  
T.K. Biryukova
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Classification ◽  
Activation Function ◽  
Experimental Comparison ◽  
Training Time ◽  
Operation Speed ◽  
The Neural Network ◽  
Linear Algebraic Equations ◽  
Network Operation

Classic neural networks suppose trainable parameters to include just weights of neurons. This paper proposes parabolic integrodifferential splines (ID-splines), developed by author, as a new kind of activation function (AF) for neural networks, where ID-splines coefficients are also trainable parameters. Parameters of ID-spline AF together with weights of neurons are vary during the training in order to minimize the loss function thus reducing the training time and increasing the operation speed of the neural network. The newly developed algorithm enables software implementation of the ID-spline AF as a tool for neural networks construction, training and operation. It is proposed to use the same ID-spline AF for neurons in the same layer, but different for different layers. In this case, the parameters of the ID-spline AF for a particular layer change during the training process independently of the activation functions (AFs) of other network layers. In order to comply with the continuity condition for the derivative of the parabolic ID-spline on the interval (x x0, n) , its parameters fi (i= 0,...,n) should be calculated using the tridiagonal system of linear algebraic equations: To solve the system it is necessary to use two more equations arising from the boundary conditions for specific problems. For exam- ple the values of the grid function (if they are known) in the points (x x0, n) may be used for solving the system above: f f x0 = ( 0) , f f xn = ( n) . The parameters Iii+1 (i= 0,...,n−1 ) are used as trainable parameters of neural networks. The grid boundaries and spacing of the nodes of ID-spline AF are best chosen experimentally. The optimal selection of grid nodes allows improving the quality of results produced by the neural network. The formula for a parabolic ID-spline is such that the complexity of the calculations does not depend on whether the grid of nodes is uniform or non-uniform. An experimental comparison of the results of image classification from the popular FashionMNIST dataset by convolutional neural 0, x< 0 networks with the ID-spline AFs and the well-known ReLUx( ) =AF was carried out. The results reveal that the usage x x, ≥ 0 of the ID-spline AFs provides better accuracy of neural network operation than the ReLU AF. The training time for two convolutional layers network with two ID-spline AFs is just about 2 times longer than with two instances of ReLU AF. Doubling of the training time due to complexity of the ID-spline formula is the acceptable price for significantly better accuracy of the network. Wherein the difference of an operation speed of the networks with ID-spline and ReLU AFs will be negligible. The use of trainable ID-spline AFs makes it possible to simplify the architecture of neural networks without losing their efficiency. The modification of the well-known neural networks (ResNet etc.) by replacing traditional AFs with ID-spline AFs is a promising approach to increase the neural network operation accuracy. In a majority of cases, such a substitution does not require to train the network from scratch because it allows to use pre-trained on large datasets neuron weights supplied by standard software libraries for neural network construction thus substantially shortening training time.

Parallel Hardware for Artificial Neural Networks Using Fixed Floating Point Representation

2011 ◽  
pp. 295-308
Author(s):  
Nadia Nedjah ◽  
Rodrigo Martins da Silva ◽  
Luiza de Macedo Mourelle
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Activation Function ◽  
Floating Point ◽  
Weighted Sum ◽  
Design Environment ◽  
Silicon Area ◽  
Point Representation ◽  
Artificial Neural ◽  
Mathematical Computation

Artificial Neural Networks (ANNs) is a well known bio-inspired model that simulates human brain capabilities such as learning and generalization. ANNs consist of a number of interconnected processing units, wherein each unit performs a weighted sum followed by the evaluation of a given activation function. The involved computation has a tremendous impact on the implementation efficiency. Existing hardware implementations of ANNs attempt to speed up the computational process. However, these implementations require a huge silicon area that makes it almost impossible to fit within the resources available on a state-of-the-art FPGAs. In this chapter, a hardware architecture for ANNs that takes advantage of the dedicated adder blocks, commonly called MACs, to compute both the weighted sum and the activation function is devised. The proposed architecture requires a reduced silicon area considering the fact that the MACs come for free as these are FPGA’s built-in cores. Our system uses integer (fixed point) mathematics and operates with fractions to represent real numbers. Hence, floating point representation is not employed and any mathematical computation of the ANN hardware is based on combinational circuitry (performing only sums and multiplications). The hardware is fast because it is massively parallel. Besides, the proposed architecture can adjust itself on-the-fly to the user-defined configuration of the neural network, i.e., the number of layers and neurons per layer of the ANN can be settled with no extra hardware changes. This is a very nice characteristic in robot-like systems considering the possibility of the same hardware may be exploited in different tasks. The hardware also requires another system (a software) that controls the sequence of the hardware computation and provides inputs, weights and biases for the ANN in hardware. Thus, a co-design environment is necessary.

Fundamental Categories of Artificial Neural Networks

2021 ◽  
pp. 30-64
Author(s):  
Arunaben Prahladbhai Gurjar ◽  
Shitalben Bhagubhai Patel
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Artificial Neural Networks ◽  
Mathematical Operation ◽  
Artificial Neural ◽  
Classification Prediction ◽  
Artificial Neural Network Ann ◽  
Fundamental Categories

The new era of the world uses artificial intelligence (AI) and machine learning. The combination of AI and machine learning is called artificial neural network (ANN). Artificial neural network can be used as hardware or software-based components. Different topology and learning algorithms are used in artificial neural networks. Artificial neural network works similarly to the functionality of the human nervous system. ANN is working as a nonlinear computing model based on activities performed by human brain such as classification, prediction, decision making, visualization just by considering previous experience. ANN is used to solve complex, hard-to-manage problems by accruing knowledge about the environment. There are different types of artificial neural networks available in machine learning. All types of artificial neural networks work based of mathematical operation and require a set of parameters to get results. This chapter gives overview on the various types of neural networks like feed forward, recurrent, feedback, classification-predication.

Sign in / Sign up

Export Citation Format

Share Document