Artificial neurovascular network (ANVN) to study the accuracy vs. efficiency trade-off in an energy dependent neural network

AbstractArtificial feedforward neural networks perform a wide variety of classification and function approximation tasks with high accuracy. Unlike their artificial counterparts, biological neural networks require a supply of adequate energy delivered to single neurons by a network of cerebral microvessels. Since energy is a limited resource, a natural question is whether the cerebrovascular network is capable of ensuring maximum performance of the neural network while consuming minimum energy? Should the cerebrovascular network also be trained, along with the neural network, to achieve such an optimum?In order to answer the above questions in a simplified modeling setting, we constructed an Artificial Neurovascular Network (ANVN) comprising a multilayered perceptron (MLP) connected to a vascular tree structure. The root node of the vascular tree structure is connected to an energy source, and the terminal nodes of the vascular tree supply energy to the hidden neurons of the MLP. The energy delivered by the terminal vascular nodes to the hidden neurons determines the biases of the hidden neurons. The “weights” on the branches of the vascular tree depict the energy distribution from the parent node to the child nodes. The vascular weights are updated by a kind of “backpropagation” of the energy demand error generated by the hidden neurons.We observed that higher performance was achieved at lower energy levels when the vascular network was also trained along with the neural network. This indicates that the vascular network needs to be trained to ensure efficient neural performance. We observed that below a certain network size, the energetic dynamics of the network in the per capita energy consumption vs. classification accuracy space approaches a fixed-point attractor for various initial conditions. Once the number of hidden neurons increases beyond a threshold, the fixed point appears to vanish, giving place to a line of attractors. The model also showed that when there is a limited resource, the energy consumption of neurons is strongly correlated to their individual contribution to the network’s performance.Author summaryThe limited availability of resources contributed to a significant role in shaping evolution. The brain is also no different. It is known to have tremendous computing power at a significantly lower cost than artificial computing systems. The artificial neural networks aim typically at minimizing output error and maximizing accuracy. A biological network like the brain has an added constraint of energy availability, which might force it to choose an optimal solution that provides the best possible accuracy while consuming minimum energy. The intricate vascular network which ensures adequate energy to the brain might be a systematically trained layout rather than a hard-wired anatomical structure. Through this work, we intend to explore how the artificial neural network would behave if it were made dependent on an energy supply network and how the training of the energy supply network would influence the performance of the neural network. Our model concluded that training of a vascular energy network is highly desirable, and when the size of the neural network is small, the energy consumed by each neuron is a direct readout on its contribution to the network performance.

Download Full-text

Modeling the energy consumption of trains by applying neural networks

Proceedings of the Institution of Mechanical Engineers Part F Journal of Rail and Rapid Transit ◽

10.1177/0954409717694522 ◽

2017 ◽

Vol 232 (3) ◽

pp. 816-823 ◽

Cited By ~ 6

Author(s):

Juan D Pineda-Jaramillo ◽

Ricardo Insa ◽

Pablo Martínez

Keyword(s):

Neural Network ◽

Neural Networks ◽

Energy Consumption ◽

Power Consumption ◽

High Accuracy ◽

Calibration And Validation ◽

The Neural Network ◽

Consumption Data ◽

Metro System

This paper presents the training of a neural network using consumption data measured in the underground network of Valencia (Spain), with the objective of estimating the energy consumption of the systems. After the calibration and validation of the neural network using part of the gathered consumption data, the results obtained show that the neural network is capable of predicting power consumption with high accuracy. Once fully trained, the network can be used to study the energy consumption of a metro system and for testing the hypothetical operation scenarios.

Download Full-text

NEURAL NETWORK FORECASTING OF ENERGY CONSUMPTION OF A METALLURGICAL ENTERPRISE

Innovative technologies and scientific solutions for industries ◽

10.30837/itssi.2021.15.014 ◽

2021 ◽

pp. 14-22

Author(s):

Anna Bakurova ◽

Olesia Yuskiv ◽

Dima Shyrokorad ◽

Anton Riabenko ◽

Elina Tereschenko

Keyword(s):

Neural Network ◽

Neural Networks ◽

Time Series ◽

Artificial Neural Networks ◽

Energy Consumption ◽

Short Term ◽

Metallurgical Enterprise ◽

Input Information ◽

The Neural Network ◽

Artificial Neural

The subject of the research is the methods of constructing and training neural networks as a nonlinear modeling apparatus for solving the problem of predicting the energy consumption of metallurgical enterprises. The purpose of this work is to develop a model for forecasting the consumption of the power system of a metallurgical enterprise and its experimental testing on the data available for research of PJSC "Dneprospetsstal". The following tasks have been solved: analysis of the time series of power consumption; building a model with the help of which data on electricity consumption for a historical period is processed; building the most accurate forecast of the actual amount of electricity for the day ahead; assessment of the forecast quality. Methods used: time series analysis, neural network modeling, short-term forecasting of energy consumption in the metallurgical industry. The results obtained: to develop a model for predicting the energy consumption of a metallurgical enterprise based on artificial neural networks, the MATLAB complex with the Neural Network Toolbox was chosen. When conducting experiments, based on the available statistical data of a metallurgical enterprise, a selection of architectures and algorithms for learning neural networks was carried out. The best results were shown by the feedforward and backpropagation network, architecture with nonlinear autoregressive and learning algorithms: Levenberg-Marquard nonlinear optimization, Bayesian Regularization method and conjugate gradient method. Another approach, deep learning, is also considered, namely the neural network with long short-term memory LSTM and the adam learning algorithm. Such a deep neural network allows you to process large amounts of input information in a short time and build dependencies with uninformative input information. The LSTM network turned out to be the most effective among the considered neural networks, for which the indicator of the maximum prediction error had the minimum value. Conclusions: analysis of forecasting results using the developed models showed that the chosen approach with experimentally selected architectures and learning algorithms meets the necessary requirements for forecast accuracy when developing a forecasting model based on artificial neural networks. The use of models will allow automating high-precision operational hourly forecasting of energy consumption in market conditions. Keywords: energy consumption; forecasting; artificial neural network; time series.

Download Full-text

Neural networks for modelling the energy consumption of metro trains

Proceedings of the Institution of Mechanical Engineers Part F Journal of Rail and Rapid Transit ◽

10.1177/0954409719861595 ◽

2019 ◽

Vol 234 (7) ◽

pp. 722-733 ◽

Cited By ~ 2

Author(s):

Pablo Martínez Fernández ◽

Pablo Salvador Zuriaga ◽

Ignacio Villalba Sanchís ◽

Ricardo Insa Franco

Keyword(s):

Neural Network ◽

Neural Networks ◽

Energy Consumption ◽

Mean Square Error ◽

Mean Square ◽

The Neural Network ◽

Train Speed ◽

Input Variables ◽

Trained Neural Network ◽

Traction Power

This paper presents the application of machine learning systems based on neural networks to model the energy consumption of electric metro trains, as a first step in a research project that aims to optimise the energy consumed for traction in the Metro Network of Valencia (Spain). An experimental dataset was gathered and used for training. Four input variables (train speed and acceleration, track slope and curvature) and one output variable (traction power) were considered. The fully trained neural network shows good agreement with the target data, with relative mean square error around 21%. Additional tests with independent datasets also give good results (relative mean square error = 16%). The neural network has been applied to five simple case studies to assess its performance – and has proven to correctly model basic consumption trends (e.g. the influence of the slope) – and to properly reproduce acceleration, holding and braking, although it tends to slightly underestimate the energy regenerated during braking. Overall, the neural network provides a consistent estimation of traction power and the global energy consumption of metro trains, and thus may be used as a modelling tool during further stages of research.

Download Full-text

Application of Neural Networks to the Interpretation of Laboratory Data in Cancer Diagnosis

Clinical Chemistry ◽

10.1093/clinchem/38.1.34 ◽

1992 ◽

Vol 38 (1) ◽

pp. 34-38 ◽

Cited By ~ 72

Author(s):

Michael L Astion ◽

Peter Wilding

Keyword(s):

Neural Network ◽

Neural Networks ◽

Discriminant Function ◽

Cross Validation ◽

Function Analysis ◽

Laboratory Data ◽

Density Lipoprotein ◽

The Neural Network ◽

Quadratic Discriminant Function ◽

Hidden Neurons

Abstract Neural networks are a relatively new method of multivariate analysis. The purpose of this study was to investigate the ability of neural networks to differentiate benign from malignant breast conditions on the basis of the pattern of nine variables: patient age, total cholesterol, high-density lipoprotein cholesterol, triglycerides, apolipoprotein A-I, apolipoprotein B, albumin, the tumor marker CA15-3, and the Fossel index (measurement of methylene and methyl line-widths in proton NMR spectra). The laboratory analyses were made with blood plasma or serum specimens. The neural network was "trained" with 57 patients: 23 patients with breast malignancies and 34 patients with benign breast conditions. A neural network with nine input neurons, 15 hidden neurons, and two output neurons correctly classified all 57 patients. The ability of the network to predict the diagnoses of patients that it had no encountered in training was tested with a separate group (cross-validation group) of 20 patients. The network correctly predicted the diagnoses for 80% of these patients. For comparison we analyzed the same sets of 57 training patients and 20 cross-validation patients by quadratic discriminant function analysis. The quadratic discriminant function, calculated from the same 57 patients used to train the neural network, correctly classified 84% of the 57 patients, and correctly diagnosed 75% of the 20 cross-validation patients. The results suggest that neural networks are a potentially useful multivariate method for optimizing the diagnostic utility of laboratory data.

Download Full-text

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

Economy of Ukraine ◽

10.15407/economyukr.2020.10.054 ◽

2020 ◽

Vol 2020 (10) ◽

pp. 54-62

Author(s):

Oleksii VASYLIEV ◽

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Statistical Data ◽

Activation Function ◽

Decision Making Process ◽

Neural Network Architecture ◽

Acceptable Accuracy ◽

The Neural Network ◽

Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

Download Full-text

Discretization and machine learning approximation of BSDEs with a constraint on the Gains-process

Monte Carlo Methods and Applications ◽

10.1515/mcma-2020-2080 ◽

2021 ◽

Vol 0 (0) ◽

Author(s):

Idris Kharroubi ◽

Thomas Lim ◽

Xavier Warin

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Differential Equations ◽

Numerical Experiments ◽

Optimization Problem ◽

Learning Approach ◽

The Neural Network ◽

Machine Learning Approach ◽

Mesh Grid

AbstractWe study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to zero. We then focus on the approximation of the discretely constrained BSDE. For that we adopt a machine learning approach. We show that the facelift can be approximated by an optimization problem over a class of neural networks under constraints on the neural network and its derivative. We then derive an algorithm converging to the discretely constrained BSDE as the number of neurons goes to infinity. We end by numerical experiments.

Download Full-text

Application of neural networks in predictions of brake wear particulate matter emission

Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering ◽

10.1177/09544070211036321 ◽

2021 ◽

pp. 095440702110363

Author(s):

Saša Vasiljević ◽

Jasna Glišović ◽

Nadica Stojanović ◽

Ivan Grujić

Keyword(s):

Neural Network ◽

Neural Networks ◽

Particulate Matter ◽

Traffic Control ◽

World Health ◽

Vehicle Speed ◽

System Pressure ◽

The Neural Network ◽

Vehicle Systems ◽

Brake Wear

According to the World Health Organization, air pollution with PM10 and PM2.5 (PM-particulate matter) is a significant problem that can have serious consequences for human health. Vehicles, as one of the main sources of PM10 and PM2.5 emissions, pollute the air and the environment both by creating particles by burning fuel in the engine, and by wearing of various elements in some vehicle systems. In this paper, the authors conducted the prediction of the formation of PM10 and PM2.5 particles generated by the wear of the braking system using a neural network (Artificial Neural Networks (ANN)). In this case, the neural network model was created based on the generated particles that were measured experimentally, while the validity of the created neural network was checked by means of a comparative analysis of the experimentally measured amount of particles and the prediction results. The experimental results were obtained by testing on an inertial braking dynamometer, where braking was performed in several modes, that is under different braking parameters (simulated vehicle speed, brake system pressure, temperature, braking time, braking torque). During braking, the concentration of PM10 and PM2.5 particles was measured simultaneously. The total of 196 measurements were performed and these data were used for training, validation, and verification of the neural network. When it comes to simulation, a comparison of two types of neural networks was performed with one output and with two outputs. For each type, network training was conducted using three different algorithms of backpropagation methods. For each neural network, a comparison of the obtained experimental and simulation results was performed. More accurate prediction results were obtained by the single-output neural network for both particulate sizes, while the smallest error was found in the case of a trained neural network using the Levenberg-Marquardt backward propagation algorithm. The aim of creating such a prediction model is to prove that by using neural networks it is possible to predict the emission of particles generated by brake wear, which can be further used for modern traffic systems such as traffic control. In addition, this wear algorithm could be applied on other vehicle systems, such as a clutch or tires.

Download Full-text

A Neural Network Decomposition Algorithm for Mapping on Crossbar-Based Computing Systems

Electronics ◽

10.3390/electronics9091526 ◽

2020 ◽

Vol 9 (9) ◽

pp. 1526 ◽

Cited By ~ 1

Author(s):

Choongmin Kim ◽

Jacob A. Abraham ◽

Woochul Kang ◽

Jaeyong Chung

Keyword(s):

Neural Network ◽

Neural Networks ◽

Predictive Performance ◽

Fixed Number ◽

Decomposition Algorithm ◽

Computing Systems ◽

Network Decomposition ◽

Von Neumann ◽

Analog Computing ◽

The Neural Network

Crossbar-based neuromorphic computing to accelerate neural networks is a popular alternative to conventional von Neumann computing systems. It is also referred as processing-in-memory and in-situ analog computing. The crossbars have a fixed number of synapses per neuron and it is necessary to decompose neurons to map networks onto the crossbars. This paper proposes the k-spare decomposition algorithm that can trade off the predictive performance against the neuron usage during the mapping. The proposed algorithm performs a two-level hierarchical decomposition. In the first global decomposition, it decomposes the neural network such that each crossbar has k spare neurons. These neurons are used to improve the accuracy of the partially mapped network in the subsequent local decomposition. Our experimental results using modern convolutional neural networks show that the proposed method can improve the accuracy substantially within about 10% extra neurons.

Download Full-text

Classification of Alloys with an Artificial Neural Network and Multivariate Calibration of Glow-Discharge Emission Spectra

Applied Spectroscopy ◽

10.1366/0003702914335238 ◽

1991 ◽

Vol 45 (10) ◽

pp. 1706-1716 ◽

Cited By ~ 17

Author(s):

Mark Glick ◽

Gary M. Hieftje

Keyword(s):

Neural Network ◽

Neural Networks ◽

Glow Discharge ◽

Multivariate Calibration ◽

Emission Spectra ◽

K Nearest Neighbor ◽

The Neural Network ◽

Calibration Sets ◽

Artificial Neural

Artificial neural networks were constructed for the classification of metal alloys based on their elemental constituents. Glow discharge-atomic emission spectra obtained with a photodiode array spectrometer were used in multivariate calibrations for 7 elements in 37 Ni-based alloys (different types) and 15 Fe-based alloys. Subsets of the two major classes formed calibration sets for stepwise multiple linear regression. The remaining samples were used to validate the calibration models. Reference data from the calibration sets were then pooled into a single set to train neural networks with different architectures and different training parameters. After the neural networks learned to discriminate correctly among alloy classes in the training set, their ability to classify samples in the testing set was measured. In general, the neural network approach performed slightly better than the K-nearest neighbor method, but it suffered from a hidden classification mechanism and nonunique solutions. The neural network methodology is discussed and compared with conventional sample-classification techniques, and multivariate calibration of glow discharge spectra is compared with conventional univariate calibration.

Download Full-text