SIMPLEX: An Activation Function with Improved Loss Function Results in Validation

2020 ◽  
Vol 17 (1) ◽  
pp. 147-153
Author(s):  
Sarfaraz Masoood ◽  
Nida Safdar Jan
Keyword(s):  
Neural Network ◽  
Activation Function ◽  
Mathematical Function ◽  
Training Phase ◽  
Complex Relationship ◽  
Activation Functions ◽  
Testing Phase ◽  
Experimental Problem ◽  
Overall Performance ◽  
Artificial Neural

An activation function is a mathematical function used for squashing purposes in artificial neural networks, whose domain and the range are two important most features to judge its potency. Overfitting of a neural network, is an issue that has gained considerable importance. This is a consequence of a function developing some complex relationship during the training phase and then these do not show up during the testing phase due to which these relationships aren’t actually relations, but are merely a consequence of sampling noise that arises during the training phase and is absent during testing phase. This creates a significant gap in accuracy which if minimized could result in better results in terms of overall performance of an ANN (Artificial Neural Network). The activation function proposed in this work is called SIMPLEX. Over a set of experiments, it was observed, to have the least overfitting issue among the rest of the analyzed activation functions over the MNIST dataset, selected as the experimental problem.

scholarly journals Writer Identification Based on Hand Writing using Artificial Neural Network

2021 ◽  
Vol 3 (2) ◽  
pp. 257-264
Author(s):  
Rosalia Arum Kumalasanti ◽  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Social Interaction ◽  
Social Relations ◽  
Image Data ◽  
Training Phase ◽  
Testing Phase ◽  
Training Stage ◽  
Artificial Neural ◽  
Hand Writing

Humans are social beings who depend on social interaction. Social interaction that is often used is communication. Communication is one of the bridges to connect social relations between humans. Communication can be delivered in two ways, namely verbal or nonverbal. Handwriting is an example of nonverbal communication using paper and writing utensils. Each individual's writing has its own uniqueness so that handwriting often becomes the character or characteristic of the author. The handwriting pattern usually becomes a character for the writer so that people who recognize the writing will easily guess the ownership of the related handwriting. However, handwriting is often used by irresponsible people in the form of handwriting falsification. The acts of writing falcification often occur in the workplace or even in the field of education. This is one of the driving factors for creating a reliable system in tracking someone's handwriting based on their ownership. In this study, we will discuss the identification of a person's handwriting based on their ownership. The output of this research is in the form of ID from the author and accuracy in the form of percentage of system reliability in identifying. The results of this study are expected to have a good impact on all parties, in order to minimize plagiarism. Identification of handwriting to be built consists of two main processes, namely the training phase and the testing phase. At the training stage, the handwritten image is subjected to several processes, namely threshold, wavelet conversion, and then will be trained using the Backpropagation Artificial Neural Network. In the testing phase, the process is the same as in the training phase, but at the end of the process, a comparison will be made between the image data that has been stored during training with a comparison image. Backpropagation ANN can work optimally if it is trained using input data that has determined the size, learning rate, parameters, and the number of nodes on the network. It is expected that the offered method can work optimally so that it produces an accurate percentage in order to minimize handwriting falcification.

scholarly journals Agent-based Modeling for Activation Function in Enhancement Logic Programming in Hopfield Neural Network

2020 ◽  
Vol 9 (4) ◽  
pp. 1872-1879
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Logic Programming ◽  
Hamming Distance ◽  
Hopfield Neural Network ◽  
Activation Function ◽  
Agent Based Modeling ◽  
Activation Functions ◽  
Agent Based ◽  
Artificial Neural

Artificial Neural Network (ANN) uses many activation functions to update the state on neuron. The research and engineering have been used activation functions in the artificial neural network as the transfer functions. The most common reasons for using this transfer function were its unit interval boundaries, the functions and quick computability of its derivative, and several useful mathematical properties in the approximation of theory realm. Aim of this study is to figure out the best robust activation functions to accelerate HornSAT logic in the Hopfield Neural Network's context. In this paper we had developed Agent-based Modelling (ABM) assessed the performance of the Zeng Martinez Activation Function (ZMAF) and the Hyperbolic Tangent Activation Function (HTAF) beside the Wan Abdullah method to do Logic Programming (LP) in Hopfield Neural Network (HNN). These assessments are carried out on the basis of hamming distance (HD), the global minima ratio (zM), and CPU time. NETLOGO 5.3.1 software has been used for developing Agent-based Modeling (ABM) to test the proposed comparison of the efficaecy of these two activation functions HTAF and ZMAF.

scholarly journals Prediction of Stress in Power Transformer Winding Conductors Using Artificial Neural Networks: Hyperparameter Analysis

Energies ◽  
2021 ◽  
Vol 14 (14) ◽  
pp. 4242
Author(s):  
Fausto Valencia ◽  
Hugo Arcos ◽  
Franklin Quilumba
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Finite Element ◽  
Power Transformer ◽  
Network Models ◽  
Activation Function ◽  
Neural Network Models ◽  
Transformer Winding ◽  
Artificial Neural ◽  
Femm Software

The purpose of this research is the evaluation of artificial neural network models in the prediction of stresses in a 400 MVA power transformer winding conductor caused by the circulation of fault currents. The models were compared considering the training, validation, and test data errors’ behavior. Different combinations of hyperparameters were analyzed based on the variation of architectures, optimizers, and activation functions. The data for the process was created from finite element simulations performed in the FEMM software. The design of the Artificial Neural Network was performed using the Keras framework. As a result, a model with one hidden layer was the best suited architecture for the problem at hand, with the optimizer Adam and the activation function ReLU. The final Artificial Neural Network model predictions were compared with the Finite Element Method results, showing good agreement but with a much shorter solution time.

scholarly journals Using Artificial Neural Networks in Predicting the Level of Stress among Military Conscripts

Mathematics ◽  
2021 ◽  
Vol 9 (6) ◽  
pp. 626
Author(s):  
Svajone Bekesiene ◽  
Rasa Smaliukiene ◽  
Ramute Vaicaitiene
Keyword(s):  
Neural Network ◽  
Transfer Functions ◽  
Back Propagation ◽  
Armed Forces ◽  
Activation Function ◽  
Optimal Number ◽  
Ann Model ◽  
Back Propagation Algorithm ◽  
Classification Rate ◽  
Artificial Neural

The present study aims to elucidate the main variables that increase the level of stress at the beginning of military conscription service using an artificial neural network (ANN)-based prediction model. Random sample data were obtained from one battalion of the Lithuanian Armed Forces, and a survey was conducted to generate data for the training and testing of the ANN models. Using nonlinearity in stress research, numerous ANN structures were constructed and verified to limit the optimal number of neurons, hidden layers, and transfer functions. The highest accuracy was obtained by the multilayer perceptron neural network (MLPNN) with a 6-2-2 partition. A standardized rescaling method was used for covariates. For the activation function, the hyperbolic tangent was used with 20 units in one hidden layer as well as the back-propagation algorithm. The best ANN model was determined as the model that showed the smallest cross-entropy error, the correct classification rate, and the area under the ROC curve. These findings show, with high precision, that cohesion in a team and adaptation to military routines are two critical elements that have the greatest impact on the stress level of conscripts.

Deep neural network based on generalized neo-fuzzy neurons and its learning based on backpropagation

2021 ◽  
Vol 26 (jai2021.26(1)) ◽  
pp. 32-41
Author(s):  
Bodyanskiy Y ◽  
◽  
Antonenko T ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Process ◽  
Activation Function ◽  
Point Of View ◽  
Basic Unit ◽  
Theoretical Point ◽  
Activation Functions ◽  
Approximation Properties ◽  
Network Training

Modern approaches in deep neural networks have a number of issues related to the learning process and computational costs. This article considers the architecture grounded on an alternative approach to the basic unit of the neural network. This approach achieves optimization in the calculations and gives rise to an alternative way to solve the problems of the vanishing and exploding gradient. The main issue of the article is the usage of the deep stacked neo-fuzzy system, which uses a generalized neo-fuzzy neuron to optimize the learning process. This approach is non-standard from a theoretical point of view, so the paper presents the necessary mathematical calculations and describes all the intricacies of using this architecture from a practical point of view. From a theoretical point, the network learning process is fully disclosed. Derived all necessary calculations for the use of the backpropagation algorithm for network training. A feature of the network is the rapid calculation of the derivative for the activation functions of neurons. This is achieved through the use of fuzzy membership functions. The paper shows that the derivative of such function is a constant, and this is a reason for the statement of increasing in the optimization rate in comparison with neural networks which use neurons with more common activation functions (ReLU, sigmoid). The paper highlights the main points that can be improved in further theoretical developments on this topic. In general, these issues are related to the calculation of the activation function. The proposed methods cope with these points and allow approximation using the network, but the authors already have theoretical justifications for improving the speed and approximation properties of the network. The results of the comparison of the proposed network with standard neural network architectures are shown

Analysis of Fin-Tube Evaporator Performance With Limited Experimental Data Using Artificial Neural Networks

2000 ◽  
Author(s):  
Arturo Pacheco-Vega ◽  
Mihir Sen ◽  
Rodney L. McClain
Keyword(s):  
Neural Network ◽  
Heat Rate ◽  
Network Models ◽  
Activation Function ◽  
Operating Conditions ◽  
Training Data ◽  
Neural Network Models ◽  
The Neural Network ◽  
Artificial Neural ◽  
Fin Tube

Abstract In the current study we consider the problem of accuracy in heat rate estimations from artificial neural network models of heat exchangers used for refrigeration applications. The network configuration is of the feedforward type with a sigmoid activation function and a backpropagation algorithm. Limited experimental measurements from a manufacturer are used to show the capability of the neural network technique in modeling the heat transfer in these systems. Results from this exercise show that a well-trained network correlates the data with errors of the same order as the uncertainty of the measurements. It is also shown that the number and distribution of the training data are linked to the performance of the network when estimating the heat rates under different operating conditions, and that networks trained from few tests may give large errors. A methodology based on the cross-validation technique is presented to find regions where not enough data are available to construct a reliable neural network. The results from three tests show that the proposed methodology gives an upper bound of the estimated error in the heat rates.

Power Forecasting from Solar Panels Using Artificial Neural Network in UTHM Parit Raja

2021 ◽  
Vol 02 (01) ◽  
Author(s):  
Natasha Munirah Mohd Fahmi ◽  
◽  
Nor Aira Zambri ◽  
Norhafiz Salim ◽  
Sim Sy Yi ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Climatic Factors ◽  
Activation Function ◽  
Solar Panels ◽  
Ann Model ◽  
Power Characteristics ◽  
Step Procedure ◽  
Artificial Neural ◽  
Artificial Neural Network Ann

This paper presents a step-by-step procedure for the simulation of photovoltaic modules with numerical values, using MALTAB/Simulink software. The proposed model is developed based on the mathematical model of PV module, which based on PV solar cell employing one-diode equivalent circuit. The output current and power characteristics curves highly depend on some climatic factors such as radiation and temperature, are obtained by simulation of the selected module. The collected data are used in developing Artificial Neural Network (ANN) model. Multilayer Perceptron (MLP) and Radial Basis Function (RBF) are the techniques used to forecast the outputs of the PV. Various types of activation function will be applied such as Linear, Logistic Sigmoid, Hyperbolic Tangent Sigmoid and Gaussian. The simulation results show that the Logistic Sigmoid is the best technique which produce minimal root mean square error for the system.

scholarly journals Comparing Activation Functions in Modeling Shoreline Variation Using Multilayer Perceptron Neural Network

Water ◽  
2020 ◽  
Vol 12 (5) ◽  
pp. 1281
Author(s):  
Je-Chian Chen ◽  
Yu-Min Wang
Keyword(s):  
Neural Network ◽  
Multilayer Perceptron ◽  
Activation Function ◽  
Aerial Survey ◽  
Activation Functions ◽  
Shoreline Changes ◽  
Extensive Evaluation ◽  
Aerial Vehicle ◽  
Sine Functions ◽  
Mlp Model

The study has modeled shoreline changes by using a multilayer perceptron (MLP) neural network with the data collected from five beaches in southern Taiwan. The data included aerial survey maps of the Forestry Bureau for years 1982, 2002, and 2006, which served as predictors, while the unmanned aerial vehicle (UAV) surveyed data of 2019 served as the respondent. The MLP was configured using five different activation functions with the aim of evaluating their significance. These functions were Identity, Tahn, Logistic, Exponential, and Sine Functions. The results have shown that the performance of an MLP model may be affected by the choice of an activation function. Logistic and the Tahn activation functions outperformed the other models, with Logistic performing best in three beaches and Tahn having the rest. These findings suggest that the application of machine learning to shoreline changes should be accompanied by an extensive evaluation of the different activation functions.

Prediction of stress in fillet portion of spur gears using artificial neural networks

2007 ◽  
Vol 22 (1) ◽  
pp. 41-51
Author(s):  
M.S. Shunmugam ◽  
N. Siva Prasad
Keyword(s):  
Neural Network ◽  
Gear Tooth ◽  
Spur Gear ◽  
Activation Function ◽  
Training Data ◽  
Spur Gears ◽  
Pressure Angle ◽  
The Neural Network ◽  
Artificial Neural ◽  
Taguchi’S Design Of Experiments

AbstractA fillet curve is provided at the root of the spur gear tooth, as stresses are high in this portion. The fillet curve may be a trochoid or an arc of suitable size as specified by designer. The fillet stress is influenced by the fillet geometry as well as the number of teeth, modules, and the pressure angle of the gear. Because the relationship is nonlinear and complex, an artificial neural network and a backpropagation algorithm are used in the present work to predict the fillet stresses. Training data are obtained from finite element simulations that are greatly reduced using Taguchi's design of experiments. Each simulation takes around 30 min. The 4-5-1 network and a sigmoid activation function are chosen. TRAINLM function is used for training the network with a learning rate parameter of 0.01 and a momentum constant of 0.8. The neural network is able to predict the fillet stresses in 0.03 s with reasonable accuracy for spur gears having 25–125 teeth, a 1–5 mm module, a 0.05–0.45 mm fillet radius, and a 15°–25° pressure angle.

scholarly journals General Recurrent Neural Network for Solving Generalized Linear Matrix Equation

Complexity ◽  
2017 ◽  
Vol 2017 ◽  
pp. 1-7 ◽  
Author(s):  
Zhan Li ◽  
Hong Cheng ◽  
Hongliang Guo
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Neural Network Model ◽  
Matrix Equation ◽  
Activation Function ◽  
Linear Matrix ◽  
Activation Functions ◽  
State Matrix ◽  
Linear Matrix Equation

This brief proposes a general framework of the nonlinear recurrent neural network for solving online the generalized linear matrix equation (GLME) with global convergence property. If the linear activation function is utilized, the neural state matrix of the nonlinear recurrent neural network can globally and exponentially converge to the unique theoretical solution of GLME. Additionally, as compared with the case of using the linear activation function, two specific types of nonlinear activation functions are proposed for the general nonlinear recurrent neural network model to achieve superior convergence. Illustrative examples are shown to demonstrate the efficacy of the general nonlinear recurrent neural network model and its superior convergence when activated by the aforementioned nonlinear activation functions.

Sign in / Sign up

Export Citation Format

Share Document